diff options
Diffstat (limited to 'doc')
910 files changed, 28051 insertions, 19582 deletions
diff --git a/doc/.vale/gitlab/FirstPerson.yml b/doc/.vale/gitlab/FirstPerson.yml deleted file mode 100644 index e97b886b5ed..00000000000 --- a/doc/.vale/gitlab/FirstPerson.yml +++ /dev/null @@ -1,16 +0,0 @@ ---- -# Warning: gitlab.FirstPerson -# -# Checks for use of first person pronouns. -# -# For a list of all options, see https://vale.sh/docs/topics/styles/ -extends: existence -message: "Instead of '%s', speak directly to the reader. Use 'you' or re-write to remove." -level: warning -ignorecase: true -link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html -tokens: - - '\bI[ ,;:?!"]|\bI\x27.{1,2}' - - me - - myself - - mine diff --git a/doc/.vale/gitlab/Markdown_emoji.yml b/doc/.vale/gitlab/Markdown_emoji.yml index 9873fb8becd..20f3ed0f5cb 100644 --- a/doc/.vale/gitlab/Markdown_emoji.yml +++ b/doc/.vale/gitlab/Markdown_emoji.yml @@ -1,11 +1,11 @@ --- # Warning: gitlab.Markdown_emoji # -# Check for use of GLFM emoji syntax (https://docs.gitlab.com/ee/user/markdown.html#emojis), which doesn't render correctly in documentation. +# Check for use of GLFM emoji syntax (https://docs.gitlab.com/ee/user/markdown.html#emoji), which doesn't render correctly in documentation. # # For a list of all options, see https://vale.sh/docs/topics/styles/ extends: existence -message: "Replace '%s' with GitLab SVGs or Unicode emojis." +message: "Replace '%s' with GitLab SVGs or Unicode emoji." link: https://docs.gitlab.com/ee/development/documentation/styleguide/#gitlab-svg-icons level: warning scope: text diff --git a/doc/.vale/gitlab/SubstitutionWarning.yml b/doc/.vale/gitlab/SubstitutionWarning.yml index fa4a0960d9c..d9d4200e88b 100644 --- a/doc/.vale/gitlab/SubstitutionWarning.yml +++ b/doc/.vale/gitlab/SubstitutionWarning.yml @@ -18,11 +18,15 @@ swap: click: "select" code base: "codebase" config: "configuration" + confirmation box: "confirmation dialog" + confirmation dialog box: "confirmation dialog" deselect: "clear" deselected: "cleared" + dialog box: "dialog" distro: "distribution" docs: "documentation" e-mail: "email" + emojis: "emoji" ex: "for example" file name: "filename" filesystem: "file system" @@ -30,6 +34,9 @@ swap: it is recommended: "you should" logged in user: "authenticated user" logged-in user: "authenticated user" + modal dialog: "dialog" + modal window: "dialog" + modal: "dialog" n/a: "not applicable" navigate to: "go to" OAuth2: "OAuth 2.0" @@ -38,6 +45,9 @@ swap: once that: "after that" once the: "after the" once you: "after you" + pop-up window: "dialog" + pop-up: "dialog" + popup: "dialog" repo: "repository" signed in user: "authenticated user" signed-in user: "authenticated user" diff --git a/doc/.vale/gitlab/Zip.yml b/doc/.vale/gitlab/Zip.yml new file mode 100644 index 00000000000..69ff980b822 --- /dev/null +++ b/doc/.vale/gitlab/Zip.yml @@ -0,0 +1,15 @@ +--- +# Warning: gitlab.Zip +# +# Recommends all instances of something.zip be wrapped in backticks +# due to the .zip top-level domain +# +# For a list of all options, see https://vale.sh/docs/topics/styles/ +extends: existence +message: "Wrap '%s' in backticks to prevent unintentional links to .zip domain names." +link: 'https://docs.gitlab.com/ee/development/documentation/styleguide/index.md#backticks-in-markdown' +nonword: true +level: error +ignorecase: true +tokens: + - '\b\w*\.zip' diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt index 4bed441ba9d..ed505094e75 100644 --- a/doc/.vale/gitlab/spelling-exceptions.txt +++ b/doc/.vale/gitlab/spelling-exceptions.txt @@ -193,6 +193,7 @@ Consul Contentful Corosync corpuses +Cosign Coursier CPU CPUs @@ -374,6 +375,7 @@ FQDNs FreshBooks frontend Fugit +Fulcio fuzzer fuzzing Gantt @@ -516,6 +518,7 @@ Kubernetes Kubesec Kucoin Kustomize +Kustomization kwargs Laravel LaunchDarkly @@ -785,6 +788,7 @@ reindexes reindexing reinitialize reinitializing +Rekor relicensing remediations renderers @@ -879,6 +883,7 @@ Shimo Shippo Shopify Sidekiq +Sigstore Silverlight Sisense Sitespeed diff --git a/doc/administration/admin_area.md b/doc/administration/admin_area.md new file mode 100644 index 00000000000..1e103bb55c8 --- /dev/null +++ b/doc/administration/admin_area.md @@ -0,0 +1,489 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# GitLab Admin Area **(FREE SELF)** + +The Admin Area provides a web UI to manage and configure features of GitLab +self-managed instances. If you are an administrator,to access the Admin Area: + +- In GitLab 16.1 and later: on the left sidebar, expand the top-most chevron (**{chevron-down}**), then select **Admin Area**. +- In GitLab 16.0 and earlier: on the top bar, select **Main menu > Admin**. + +NOTE: +Only administrators can access the Admin Area. + +## Administering projects + +You can administer all projects in the GitLab instance from the Admin Area's Projects page. + +To access the Projects page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Projects**. +1. Select the **All**, **Private**, **Internal**, or **Public** tab to list only + projects of that criteria. + +By default, all projects are listed, in reverse order of when they were last updated. For each +project, the following information is listed: + +- Name +- Namespace +- Description +- Size, updated every 15 minutes at most + +Projects can be edited or deleted. + +To edit a project's name or description: + +1. In the Projects overview, next to the project you want to edit, select **Edit**. +1. Edit the **Project name** or **Project description**. +1. Select **Save Changes**. + +To delete a project: + +1. In the Projects overview, next to the project you want to delete, select **Delete**. + +The list of projects can be sorted by: + +- Updated date +- Last created +- Name +- Most stars +- Oldest created +- Oldest updated +- Largest repository + +A user can choose to hide or show archived projects in the list. + +In the **Filter by name** field, type the project name you want to find, and GitLab filters +them as you type. + +To filter only projects in that namespace, select from the **Namespace** dropdown list. + +You can combine the filter options. For example, to list only public projects with `score` in their name: + +1. Select the **Public** tab. +1. Enter `score` in the **Filter by name...** input box. + +## Administering users + +You can administer all users in the GitLab instance from the Admin Area's Users page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. + +To list users matching a specific criteria, select one of the following tabs on the **Users** page: + +- **Active** +- **Admins** +- **2FA Enabled** +- **2FA Disabled** +- **External** +- **[Blocked](../administration/moderate_users.md#block-a-user)** +- **[Deactivated](../administration/moderate_users.md#deactivate-a-user)** +- **Without projects** + +For each user, the following are listed: + +1. Username +1. Email address +1. Project membership count +1. Group membership count ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276215) in GitLab 13.12) +1. Date of account creation +1. Date of last activity + +To edit a user, in the user's row, select **Edit**. To delete the user, or delete the user and their contributions, select the cog dropdown list in +that user's row, and select the desired option. + +To change the sort order: + +1. Select the sort dropdown list. +1. Select the desired order. + +By default the sort dropdown list shows **Name**. + +To search for users, enter your criteria in the search field. The user search is case +insensitive, and applies partial matching to name and username. To search for an email address, +you must provide the complete email address. + +### User impersonation + +An administrator can "impersonate" any other user, including other administrators. +This allows the administrator to "see what the user sees," and take actions on behalf of the user. +You can impersonate a user in the following ways: + +- Through the UI: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Overview > Users**. + 1. From the list of users, select a user. + 1. Select **Impersonate**. +- With the API, using [impersonation tokens](../api/rest/index.md#impersonation-tokens). + +All impersonation activities are [captured with audit events](audit_events.md#user-impersonation). + +By default, impersonation is enabled. GitLab can be configured to [disable impersonation](../api/rest/index.md#disable-impersonation). + + + +### User identities + +> The ability to see a user's SCIM identity was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294608) in GitLab 15.3. + +When using authentication providers, administrators can see the identities for a user: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. From the list of users, select a user. +1. Select **Identities**. + +This list shows the user's identities, including SCIM identities. Administrators can use this information to troubleshoot SCIM-related issues and confirm +the identities being used for an account. + +### User Permission Export **(PREMIUM SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1772) in GitLab 13.8. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292436) in GitLab 13.9. + +An administrator can export user permissions for all users in the GitLab instance from the Admin Area's Users page. +The export lists direct membership the users have in groups and projects. + +The following data is included in the export: + +- Username +- Email +- Type +- Path +- Access level ([Project](../user/permissions.md#project-members-permissions) and [Group](../user/permissions.md#group-members-permissions)) +- Date of last activity ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345388) in GitLab 14.6). For a list of activities that populate this column, see the [Users API documentation](../api/users.md#get-user-activities). + +Only the first 100,000 user accounts are exported. + + + +### Users statistics + +The **Users statistics** page provides an overview of user accounts by role. These statistics are +calculated daily, so user changes made since the last update are not reflected. + +The following totals are also included: + +- Billable users +- Blocked users +- Total users + +GitLab billing is based on the number of [**Billable users**](../subscriptions/self_managed/index.md#billable-users). + +### Add email to user + +You must be an administrator to manually add emails to users: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Locate the user and select them. +1. Select **Edit**. +1. In **Email**, enter the new email address. This adds the new email address to the + user and sets the previous email address to be a secondary. +1. Select **Save changes**. + +## User cohorts + +The [Cohorts](user_cohorts.md) tab displays the monthly cohorts of new users and their activities over time. + +## Prevent a user from creating groups + +By default, users can create groups. To prevent a user from creating a top level group: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Locate the user and select them. +1. Select **Edit**. +1. Clear the **Can create group** checkbox. +1. Select **Save changes**. + +It is also possible to [limit which roles can create a subgroup within a group](../user/group/subgroups/index.md#change-who-can-create-subgroups). + +## Administering groups + +You can administer all groups in the GitLab instance from the Admin Area's Groups page. + +To access the Groups page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Groups**. + +For each group, the page displays their name, description, size, number of projects in the group, +number of members, and whether the group is private, internal, or public. To edit a group, in the group's row, select **Edit**. To delete the group, in the group's row, select **Delete**. + +To change the sort order, select the sort dropdown list and select the desired order. The default +sort order is by **Last created**. + +To search for groups by name, enter your criteria in the search field. The group search is case +insensitive, and applies partial matching. + +To [Create a new group](../user/group/index.md#create-a-group) select **New group**. + +## Administering topics + +- > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340920) in GitLab 14.4. +- > Merging topics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/366884) in GitLab 15.5. + +[Topics](../user/project/working_with_projects.md#explore-topics) are used to categorize and find similar projects. + +### View all topics + +To view all topics in the GitLab instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. + +For each topic, the page displays its name and the number of projects labeled with the topic. + +### Search for topics + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. +1. In the search box, enter your search criteria. + The topic search is case-insensitive and applies partial matching. + +### Create a topic + +To create a topic: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. +1. Select **New topic**. +1. Enter the **Topic slug (name)** and **Topic title**. +1. Optional. Enter a **Description** and add a **Topic avatar**. +1. Select **Save changes**. + +The created topics are displayed on the **Explore topics** page. + +NOTE: +The assigned topics are visible only to everyone with access to the project, +but everyone can see which topics exist on the GitLab instance. +Do not include sensitive information in the name of a topic. + +### Edit a topic + +You can edit a topic's name, title, description, and avatar at any time. +To edit a topic: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. +1. Select **Edit** in that topic's row. +1. Edit the topic slug (name), title, description, or avatar. +1. Select **Save changes**. + +### Remove a topic + +If you no longer need a topic, you can permanently remove it. +To remove a topic: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. +1. To remove a topic, select **Remove** in that topic's row. + +### Merge topics + +You can move all projects assigned to a topic to another topic. +The source topic is then permanently deleted. +After a merged topic is deleted, you cannot restore it. + +To merge topics: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Topics**. +1. Select **Merge topics**. +1. From the **Source topic** dropdown list, select the topic you want to merge and remove. +1. From the **Target topic** dropdown list, select the topic you want to merge the source topic into. +1. Select **Merge**. + +## Administering Gitaly servers + +You can list all Gitaly servers in the GitLab instance from the Admin Area's **Gitaly Servers** +page. For more details, see [Gitaly](gitaly/index.md). + +To access the **Gitaly Servers** page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Gitaly Servers**. + +For each Gitaly server, the following details are listed: + +| Field | Description | +|----------------|-------------| +| Storage | Repository storage | +| Address | Network address on which the Gitaly server is listening | +| Server version | Gitaly version | +| Git version | Version of Git installed on the Gitaly server | +| Up to date | Indicates if the Gitaly server version is the latest version available. A green dot indicates the server is up to date. | + +## CI/CD section + +### Administering runners + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/340859) from **Overview > Runners** to **CI/CD > Runners** in GitLab 15.8. + +You can administer all runners in the GitLab instance from the Admin Area's **Runners** page. See +[GitLab Runner](https://docs.gitlab.com/runner/) for more information. + +To access the **Runners** page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Runners**. + +#### Search and filter runners + +To search runners' descriptions: + +1. In the **Search or filter results...** field, type the description of the runner you want to + find. +1. Press <kbd>Enter</kbd>. + +You can also filter runners by status, type, and tag. To filter: + +1. Select a tab or the **Search or filter results...** field. +1. Select any **Type**, or filter by **Status** or **Tags**. +1. Select or enter your search criteria. + + + +#### Bulk delete runners + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370241) in GitLab 15.4. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/353981) in GitLab 15.5. + +You can delete multiple runners at the same time. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Runners**. +1. To the left of the runners you want to delete, select the checkbox. + To select all of the runners on the page, select the checkbox above + the list. +1. Select **Delete selected**. + +#### Runner attributes + +For each runner, the following attributes are listed: + +| Attribute | Description | +|--------------|-------------| +| Status | The status of the runner. In [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/22224), for the **Ultimate** tier, the upgrade status is available. | +| Runner details | Information about the runner, including partial token and details about the computer the runner was registered from. | +| Version | GitLab Runner version. | +| Jobs | Total number of jobs run by the runner. | +| Tags | Tags associated with the runner. | +| Last contact | Timestamp indicating when the runner last contacted the GitLab instance. | + +You can also edit, pause, or remove each runner. + +### Administering Jobs + +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/386311) from **Overview > Jobs** to **CI/CD > Jobs** in GitLab 15.8. + +You can administer all jobs in the GitLab instance from the Admin Area's Jobs page. + +To access the Jobs page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **CI/CD > Jobs**. All jobs are listed, in descending order of job ID. +1. Select the **All** tab to list all jobs. Select the **Pending**, **Running**, or **Finished** + tab to list only jobs of that status. + +For each job, the following details are listed: + +| Field | Description | +|----------|-------------| +| Status | Job status, either **passed**, **skipped**, or **failed**. | +| Job | Includes links to the job, branch, and the commit that started the job. | +| Pipeline | Includes a link to the specific pipeline. | +| Project | Name of the project, and organization, to which the job belongs. | +| Runner | Name of the CI runner assigned to execute the job. | +| Stage | Stage that the job is declared in a `.gitlab-ci.yml` file. | +| Name | Name of the job specified in a `.gitlab-ci.yml` file. | +| Timing | Duration of the job, and how long ago the job completed. | +| Coverage | Percentage of tests coverage. | + +## Monitoring section + +The following topics document the **Monitoring** section of the Admin Area. + +### System Information + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341248) in GitLab 15.2, support for relative time. "Uptime" statistic was renamed to "System started". + +The **System Info** page provides the following statistics: + +| Field | Description | +|:---------------|:--------------------------------------------------| +| CPU | Number of CPU cores available | +| Memory Usage | Memory in use, and total memory available | +| Disk Usage | Disk space in use, and total disk space available | +| System started | When the system hosting GitLab was started. In GitLab 15.1 and earlier, this was an uptime statistic. | + +These statistics are updated only when you navigate to the **System Info** page, or you refresh the page in your browser. + +### Background Jobs + +The **Background Jobs** page displays the Sidekiq dashboard. Sidekiq is used by GitLab to +perform processing in the background. + +The Sidekiq dashboard consists of the following elements: + +- A tab per jobs' status. +- A breakdown of background job statistics. +- A live graph of **Processed** and **Failed** jobs, with a selectable polling interval. +- An historical graph of **Processed** and **Failed** jobs, with a selectable time span. +- Redis statistics, including: + - Version number + - Uptime, measured in days + - Number of connections + - Current memory usage, measured in MB + - Peak memory usage, measured in MB + +### Logs + +Since GitLab 13.0, **Log** view has been removed from the Admin Area dashboard since the logging does not work in multi-node setups and could cause confusion for administrators by displaying partial information. + +For multi-node systems we recommend ingesting the logs into services like Elasticsearch and Splunk. + +| Log file | Contents | +|:------------------------|:---------| +| `application_json.log` | GitLab user activity | +| `git_json.log` | Failed GitLab interaction with Git repositories | +| `production.log` | Requests received from Puma, and the actions taken to serve those requests | +| `sidekiq.log` | Background jobs | +| `repocheck.log` | Repository activity | +| `integrations_json.log` | Activity between GitLab and integrated systems | +| `kubernetes.log` | Kubernetes activity | + +The contents of these log files can be useful when troubleshooting a problem. + +For details of these log files and their contents, see [Log system](logs/index.md). + +The content of each log file is listed in chronological order. To minimize performance issues, a maximum 2000 lines of each log file are shown. + +### Audit Events **(PREMIUM SELF)** + +The **Audit Events** page lists changes made within the GitLab server. With this information you can control, analyze, and track every change. diff --git a/doc/administration/analytics/dev_ops_reports.md b/doc/administration/analytics/dev_ops_reports.md new file mode 100644 index 00000000000..1dcee5ec03d --- /dev/null +++ b/doc/administration/analytics/dev_ops_reports.md @@ -0,0 +1,74 @@ +--- +stage: Plan +group: Optimize +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# DevOps Reports **(FREE SELF)** + +DevOps Reports give you an overview of your entire instance's adoption of +[Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/) +from planning to monitoring. + +To see DevOps Reports: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Analytics > DevOps Reports**. + +## DevOps Score + +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/20976) from Conversational Development Index in GitLab 12.6. + +NOTE: +To see the DevOps score, you must activate your GitLab instance's [Service Ping](../settings/usage_statistics.md#service-ping). DevOps Score is a comparative tool, so your score data must be centrally processed by GitLab Inc. first. + +You can use the DevOps score to compare your DevOps status to other organizations. + +The DevOps Score tab displays usage of major GitLab features on your instance over +the last 30 days, averaged over the number of billable users in that time period. +You can also see the Leader usage score, calculated from top-performing instances based on +[Service Ping data](../settings/usage_statistics.md#service-ping) that GitLab has collected. +Your score is compared to the lead score of each feature and then expressed +as a percentage at the bottom of said feature. Your overall **DevOps Score** is an average of your +feature scores. + +Service Ping data is aggregated on GitLab servers for analysis. Your usage +information is **not sent** to any other GitLab instances. +If you have just started using GitLab, it might take a few weeks for data to be collected before this +feature is available. + +## DevOps Adoption **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../policy/experiment-beta-support.md#beta). +> - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. +> - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. +> - Fuzz Testing metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/330398) in GitLab 14.2. +> - Dependency Scanning metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328034) in GitLab 14.2. +> - Multi-select [added](https://gitlab.com/gitlab-org/gitlab/-/issues/333586) in GitLab 14.2. +> - Overview table [added](https://gitlab.com/gitlab-org/gitlab/-/issues/335638) in GitLab 14.3. + +DevOps Adoption shows feature adoption for development, security, and operations. + +| Category | Feature | +| --- | --- | +| Development | Approvals<br>Code owners<br>Issues<br>Merge requests | +| Security | DAST<br>Dependency Scanning<br>Fuzz Testing<br>SAST | +| Operations | Deployments<br>Pipelines<br>Runners | + +You can use Group DevOps Adoption to: + +- Identify specific subgroups that are lagging in their adoption of GitLab features, so you can guide them on +their DevOps journey. +- Find subgroups that have adopted certain features, and provide guidance to other subgroups on +how to use those features. +- Verify if you are getting the return on investment that you expected from GitLab. + +## Add or remove a group + +To add or remove a subgroup from the DevOps Adoption report: + +1. Select **Add or remove groups**. +1. Select the subgroup you want to add or remove and select **Save changes**. + + diff --git a/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_2.png b/doc/administration/analytics/img/admin_devops_adoption_v14_2.png Binary files differindex 666e03f1d9d..666e03f1d9d 100644 --- a/doc/user/admin_area/analytics/img/admin_devops_adoption_v14_2.png +++ b/doc/administration/analytics/img/admin_devops_adoption_v14_2.png diff --git a/doc/user/admin_area/analytics/img/instance_activity_pipelines_chart_v13_6_a.png b/doc/administration/analytics/img/instance_activity_pipelines_chart_v13_6_a.png Binary files differindex bd02065556c..bd02065556c 100644 --- a/doc/user/admin_area/analytics/img/instance_activity_pipelines_chart_v13_6_a.png +++ b/doc/administration/analytics/img/instance_activity_pipelines_chart_v13_6_a.png diff --git a/doc/administration/analytics/index.md b/doc/administration/analytics/index.md new file mode 100644 index 00000000000..6441cd866c8 --- /dev/null +++ b/doc/administration/analytics/index.md @@ -0,0 +1,26 @@ +--- +stage: Plan +group: Optimize +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Instance-level analytics **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41416) in GitLab 11.2. + +Instance-level analytics provide insights into the feature and data usage of your entire instance. + +## View instance-level analytics + +Prerequisite: + +- You must have administrator access to the instance. + +To view instance-level analytics: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Analytics**, then one of the available analytics: + + - [DevOps Reports](dev_ops_reports.md): Provides an overview of your entire instance's feature usage. + - [Usage Trends](usage_trends.md): Shows how much data your instance contains, and how the data is changing. diff --git a/doc/administration/analytics/usage_trends.md b/doc/administration/analytics/usage_trends.md new file mode 100644 index 00000000000..49e82f71a3a --- /dev/null +++ b/doc/administration/analytics/usage_trends.md @@ -0,0 +1,46 @@ +--- +stage: Plan +group: Optimize +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Usage Trends **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235754) in GitLab 13.5 behind a feature flag, disabled by default. +> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46962) in GitLab 13.6. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/285220) from Instance Statistics to Usage Trends in GitLab 13.6. +> - It's enabled on GitLab.com. +> - It's recommended for production use. + +Usage Trends gives you an overview of how much data your instance contains, and how quickly this volume is changing over time. +Usage Trends data refreshes daily. + +## View Usage Trends + +To view Usage Trends: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Analytics > Usage Trends**. + +## Total counts + +At the top of the page, Usage Trends shows total counts for: + +- Users +- Projects +- Groups +- Issues +- Merge requests +- Pipelines + +These figures can be useful for understanding how much data your instance contains in total. + +## Past year trend charts + +Usage Trends also displays line charts that show total counts per month, over the past 12 months, +in the categories shown in [Total counts](#total-counts). + +These charts help you visualize how rapidly these records are being created on your instance. + + diff --git a/doc/administration/appearance.md b/doc/administration/appearance.md new file mode 100644 index 00000000000..c5c50d95eb6 --- /dev/null +++ b/doc/administration/appearance.md @@ -0,0 +1,123 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GitLab Appearance **(FREE SELF)** + +Several options are available for customizing the appearance of a self-managed instance +of GitLab. To access these settings: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Appearance**. + +## Navigation bar + +By default, the navigation bar has the GitLab logo, but this can be customized with +any image desired. It is optimized for images 28px high (any width), but any image can be +used (less than 1 MB) and it is automatically resized. + +After you select and upload an image, select **Update appearance settings** at the bottom +of the page to activate it in the GitLab instance. + +NOTE: +GitLab pipeline emails also display the custom logo, unless the logo is in SVG format. If the +custom logo is in SVG format, the default logo is used instead because the SVG format is not +supported by many email clients. + +## Favicon + +By default, the favicon (used by the browser as the tab icon, as well as the CI status icon) +uses the GitLab logo. This can be customized with any icon desired. It must be a +32x32 `.png` or `.ico` image. + +After you select and upload an icon, select **Update appearance settings** at the bottom +of the page to activate it in the GitLab instance. + +## System header and footer messages + +> **Enable header and footer in emails** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344819) in GitLab 15.9. + +You can add a small header message, a small footer message, or both, to the interface +of your GitLab instance. These messages appear on all projects and pages of the +instance, including the sign in / sign up page. The default color is white text on +an orange background, but this can be customized by selecting **Customize colors**. + +Limited [Markdown](../user/markdown.md) is supported, such as bold, italics, and links, for +example. Other Markdown features, including lists, images, and quotes are not supported +as the header and footer messages can only be a single line. + +You can select **Enable header and footer in emails** to have the text of +the header and footer added to all emails sent by the GitLab instance. + +After you add a message, select **Update appearance settings** at the bottom of the page +to activate it in the GitLab instance. + +## Sign in / Sign up pages + +You can replace the default message on the sign in / sign up page with your own message +and logo. You can make full use of [Markdown](../user/markdown.md) in the description. + +The optimal size for the logo is 128 x 128 pixels, but any image can be used (below 1 MB) +and it is resized automatically. The logo image appears between the title and +the description, on the left of the sign-up page. + +After you add a message, select **Update appearance settings** at the bottom of the page +to activate it in the GitLab instance. You can also select **Sign-in page**, +to review the saved appearance settings: + +NOTE: +You can add also add a [customized hcelp message](settings/help_page.md) below the sign in message or add [a Sign in text message](settings/sign_in_restrictions.md#sign-in-information). + +## Progressive Web App + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375708) in GitLab 15.9. + +GitLab can be installed as a [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) (PWA). +Use the Progressive Web App settings to customize its appearance, including its name, +description, and icon. + +### Configure the PWA settings + +To configure the PWA settings: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Appearance**. +1. Scroll to the **Progressive Web App (PWA)** section. +1. Complete the fields. + - **Icon**: If you use the standard GitLab icon, it is available in sizes 192x192 pixels, + 512x512 pixels, also as a maskable icon. If you use a custom icon, it must be in either size + 192x192 pixels, or 512x512 pixels. +1. Select **Update appearance settings**. + +## New project pages + +You can add a new project guidelines message to the **New project page** in GitLab. +You can make full use of [Markdown](../user/markdown.md) in the description: + +The message is displayed below the **New Project** message, on the left side +of the **New project page**. + +After you add a message, select **Update appearance settings** at the bottom of the page +to activate it in the GitLab instance. You can also select **New project page**, +which brings you to the new project page so you can review the change. + +## Libravatar + +[Libravatar](https://www.libravatar.org) is supported by GitLab for avatar images, but you must +[manually enable Libravatar support on the GitLab instance](../administration/libravatar.md) to use the service. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/audit_event_streaming/graphql_api.md b/doc/administration/audit_event_streaming/graphql_api.md index f5a31f073dc..9f8fef0e3ca 100644 --- a/doc/administration/audit_event_streaming/graphql_api.md +++ b/doc/administration/audit_event_streaming/graphql_api.md @@ -14,6 +14,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Custom HTTP headers API [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366524) in GitLab 15.3. [Feature flag `streaming_audit_event_headers`](https://gitlab.com/gitlab-org/gitlab/-/issues/362941) removed. > - User-specified verification token API support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/360813) in GitLab 15.4. > - APIs for custom HTTP headers for instance level streaming destinations [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404560) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. +> - User-specified destination name API support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/413894) in GitLab 16.2. Audit event streaming destinations can be maintained using a GraphQL API. @@ -38,6 +40,7 @@ mutation { errors externalAuditEventDestination { id + name destinationUrl verificationToken group { @@ -59,6 +62,28 @@ mutation { errors externalAuditEventDestination { id + name + destinationUrl + verificationToken + group { + name + } + } + } +} +``` + +You can optionally specify your own destination name (instead of the default GitLab-generated one) using the GraphQL +`externalAuditEventDestinationCreate` +mutation. Name length must not exceed 72 characters and trailing whitespace are not trimmed. This value should be unique scoped to a group. For example: + +```graphql +mutation { + externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", name: "destination-name-here", groupPath: "my-group" }) { + errors + externalAuditEventDestination { + id + name destinationUrl verificationToken group { @@ -90,11 +115,12 @@ The header is created if the returned `errors` object is empty. ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`ff_external_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -110,6 +136,7 @@ mutation { instanceExternalAuditEventDestination { destinationUrl id + name verificationToken } } @@ -121,6 +148,24 @@ Event streaming is enabled if: - The returned `errors` object is empty. - The API responds with `200 OK`. +You can optionally specify your own destination name (instead of the default GitLab-generated one) using the GraphQL +`instanceExternalAuditEventDestinationCreate` +mutation. Name length must not exceed 72 characters and trailing whitespace are not trimmed. This value should be unique. For example: + +```graphql +mutation { + instanceExternalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", name: "destination-name-here"}) { + errors + instanceExternalAuditEventDestination { + destinationUrl + id + name + verificationToken + } + } +} +``` + Instance administrators can add an HTTP header using the GraphQL `auditEventsStreamingInstanceHeadersCreate` mutation. You can retrieve the destination ID by [listing all the streaming destinations](#list-streaming-destinations) for the instance or from the mutation above. @@ -144,6 +189,39 @@ mutation { The header is created if the returned `errors` object is empty. +### Google Cloud Logging streaming + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409422) in GitLab 16.1. + +Prerequisites: + +- Owner role for a top-level group. +- A Google Cloud project with the necessary permissions to create service accounts and enable Google Cloud Logging. + +To enable streaming and add a configuration, use the +`googleCloudLoggingConfigurationCreate` mutation in the GraphQL API. + +```graphql +mutation { + googleCloudLoggingConfigurationCreate(input: { groupPath: "my-group", googleProjectIdName: "my-google-project", clientEmail: "my-email@my-google-project.iam.gservice.account.com", privateKey: "YOUR_PRIVATE_KEY", logIdName: "audit-events" } ) { + errors + googleCloudLoggingConfiguration { + id + googleProjectIdName + logIdName + privateKey + clientEmail + } + errors + } +} +``` + +Event streaming is enabled if: + +- The returned `errors` object is empty. +- The API responds with `200 OK`. + ## List streaming destinations List new streaming destinations for top-level groups or an entire instance. @@ -166,6 +244,7 @@ query { destinationUrl verificationToken id + name headers { nodes { key @@ -184,11 +263,12 @@ If the resulting list is empty, then audit streaming is not enabled for that gro ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`ff_external_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -202,6 +282,7 @@ query { instanceExternalAuditEventDestinations { nodes { id + name destinationUrl verificationToken headers { @@ -220,6 +301,38 @@ If the resulting list is empty, then audit streaming is not enabled for the inst You need the ID values returned by this query for the update and delete mutations. +### Google Cloud Logging configurations + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409422) in GitLab 16.1. + +Prerequisite: + +- Owner role for a top-level group. + +You can view a list of streaming configurations for a top-level group using the `googleCloudLoggingConfigurations` query +type. + +```graphql +query { + group(fullPath: "my-group") { + id + googleCloudLoggingConfigurations { + nodes { + id + logIdName + googleProjectIdName + privateKey + clientEmail + } + } + } +} +``` + +If the resulting list is empty, then audit streaming is not enabled for the group. + +You need the ID values returned by this query for the update and delete mutations. + ## Update streaming destinations Update streaming destinations for a top-level group or an entire instance. @@ -236,8 +349,20 @@ by [listing all the custom HTTP headers](#list-streaming-destinations) for the g ```graphql mutation { - externalAuditEventDestinationDestroy(input: { id: destination }) { + externalAuditEventDestinationUpdate(input: { + id:"gid://gitlab/AuditEvents::ExternalAuditEventDestination/1", + destinationUrl: "https://www.new-domain.com/webhook", + name: "destination-name"} ) { errors + externalAuditEventDestination { + id + name + destinationUrl + verificationToken + group { + name + } + } } } ``` @@ -262,11 +387,12 @@ The header is deleted if the returned `errors` object is empty. ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`ff_external_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -278,11 +404,15 @@ by [listing all the external destinations](#list-streaming-destinations) for the ```graphql mutation { - instanceExternalAuditEventDestinationUpdate(input: { id: "gid://gitlab/AuditEvents::InstanceExternalAuditEventDestination/1", destinationUrl: "https://www.new-domain.com/webhook"}) { + instanceExternalAuditEventDestinationUpdate(input: { + id: "gid://gitlab/AuditEvents::InstanceExternalAuditEventDestination/1", + destinationUrl: "https://www.new-domain.com/webhook", + name: "destination-name"}) { errors instanceExternalAuditEventDestination { destinationUrl id + name verificationToken } } @@ -313,6 +443,40 @@ mutation { The header is updated if the returned `errors` object is empty. +### Google Cloud Logging configurations + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409422) in GitLab 16.1. + +Prerequisite: + +- Owner role for a top-level group. + +To update streaming configuration for a top-level group, use the +`googleCloudLoggingConfigurationUpdate` mutation type. You can retrieve the configuration ID +by [listing all the external destinations](#list-streaming-destinations). + +```graphql +mutation { + googleCloudLoggingConfigurationUpdate( + input: {id: "gid://gitlab/AuditEvents::GoogleCloudLoggingConfiguration/1", googleProjectIdName: "my-google-project", clientEmail: "my-email@my-google-project.iam.gservice.account.com", privateKey: "YOUR_PRIVATE_KEY", logIdName: "audit-events"} + ) { + errors + googleCloudLoggingConfiguration { + id + logIdName + privateKey + googleProjectIdName + clientEmail + } + } +} +``` + +Streaming configuration is updated if: + +- The returned `errors` object is empty. +- The API responds with `200 OK`. + ## Delete streaming destinations Delete streaming destinations for a top-level group or an entire instance. @@ -357,11 +521,12 @@ The header is deleted if the returned `errors` object is empty. ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335175) in GitLab 16.0 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`ff_external_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -384,6 +549,45 @@ Streaming destination is deleted if: - The returned `errors` object is empty. - The API responds with `200 OK`. +To remove an HTTP header, use the GraphQL `auditEventsStreamingInstanceHeadersDestroy` mutation. +To retrieve the header ID, +[list all the custom HTTP headers](#list-streaming-destinations) for the instance. + +```graphql +mutation { + auditEventsStreamingInstanceHeadersDestroy(input: { headerId: "gid://gitlab/AuditEvents::Streaming::InstanceHeader/<id>" }) { + errors + } +} +``` + +The header is deleted if the returned `errors` object is empty. + +### Google Cloud Logging configurations + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409422) in GitLab 16.1. + +Prerequisite: + +- Owner role for a top-level group. + +Users with the Owner role for a group can delete streaming configurations using the +`googleCloudLoggingConfigurationDestroy` mutation type. You can retrieve the configurations ID +by [listing all the streaming destinations](#list-streaming-destinations) for the group. + +```graphql +mutation { + googleCloudLoggingConfigurationDestroy(input: { id: "gid://gitlab/AuditEvents::GoogleCloudLoggingConfiguration/1" }) { + errors + } +} +``` + +Streaming configuration is deleted if: + +- The returned `errors` object is empty. +- The API responds with `200 OK`. + ## Event type filters > Event type filters API [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344845) in GitLab 15.7. diff --git a/doc/administration/audit_event_streaming/index.md b/doc/administration/audit_event_streaming/index.md index 44c6cff7455..622d29fa9a7 100644 --- a/doc/administration/audit_event_streaming/index.md +++ b/doc/administration/audit_event_streaming/index.md @@ -41,7 +41,7 @@ To add streaming destinations to a top-level group: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select **Streams** tab. -1. Select **Add streaming destination** to show the section for adding destinations. +1. Select **Add streaming destination** and select **HTTP endpoint** to show the section for adding destinations. 1. Enter the destination URL to add. 1. Optional. Locate the **Custom HTTP headers** table. 1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the @@ -50,13 +50,14 @@ To add streaming destinations to a top-level group: 20 headers per streaming destination. 1. After all headers have been filled out, select **Add** to add the new streaming destination. -### Instance streaming destinations +### Instance streaming destinations **(ULTIMATE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `instance_streaming_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`instance_streaming_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -68,8 +69,30 @@ To add a streaming destination for an instance: 1. Select **Admin Area**. 1. On the left sidebar, select **Monitoring > Audit Events**. 1. On the main area, select **Streams** tab. -1. Select **Add streaming destination** to show the section for adding destinations. +1. Select **Add streaming destination** and select **HTTP endpoint** to show the section for adding destinations. 1. Enter the destination URL to add. +1. Optional. To add custom HTTP headers, select **Add header** to create a new name and value pair, and input their values. Repeat this step for as many name and value pairs are required. You can add up to 20 headers per streaming destination. +1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the + **Active** checkbox, see [issue 367509](https://gitlab.com/gitlab-org/gitlab/-/issues/367509). +1. Select **Add header** to create a new name and value pair. Repeat this step for as many name and value pairs are required. You can add up to + 20 headers per streaming destination. +1. After all headers have been filled out, select **Add** to add the new streaming destination. + +### Google Cloud Logging streaming + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124384) in GitLab 16.2. + +Prerequisites: + +- Owner role for a top-level group. + +To add Google Cloud Logging streaming destinations to a top-level group: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Secure > Audit events**. +1. On the main area, select **Streams** tab. +1. Select **Add streaming destination** and select **Google Cloud Logging** to show the section for adding destinations. +1. Enter the Google Project ID, Google Client Email, Log ID, and Google Private Key to add. 1. Select **Add** to add the new streaming destination. ## List streaming destinations @@ -87,15 +110,16 @@ To list the streaming destinations for a top-level group: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select **Streams** tab. -1. To the right of the item, select **Edit** (**{pencil}**) to see all the custom HTTP headers. +1. Select the stream URL to expand it and see all the custom HTTP headers. ### For instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `instance_streaming_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`instance_streaming_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -108,6 +132,21 @@ To list the streaming destinations for an instance: 1. On the left sidebar, select **Monitoring > Audit Events**. 1. On the main area, select **Streams** tab. +### Google Cloud Logging streaming + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124384) in GitLab 16.2. + +Prerequisites: + +- Owner role for a top-level group. + +To list Google Cloud Logging streaming destinations for a top-level group: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Secure > Audit events**. +1. On the main area, select **Streams** tab. +1. Select the Google Cloud Logging stream to expand and see all the fields. + ## Update streaming destinations Update streaming destinations for a top-level group or an entire instance. @@ -123,6 +162,33 @@ To update a streaming destination's custom HTTP headers: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select **Streams** tab. +1. Select the stream URL to expand. +1. Locate the **Custom HTTP headers** table. +1. Locate the header that you wish to update. +1. Ignore the **Active** checkbox because it isn't functional. To track progress on adding functionality to the + **Active** checkbox, see [issue 367509](https://gitlab.com/gitlab-org/gitlab/-/issues/367509). +1. Select **Add header** to create a new name and value pair. Enter as many name and value pairs as required. You can add up to + 20 headers per streaming destination. +1. Select **Save** to update the streaming destination. + +### Instance streaming destinations **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. + +FLAG: +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. + +Prerequisites: + +- Administrator access on the instance. + +To update the streaming destinations for an instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Secure > Audit events**. +1. On the main area, select the **Streams** tab. 1. To the right of the item, select **Edit** (**{pencil}**). 1. Locate the **Custom HTTP headers** table. 1. Locate the header that you wish to update. @@ -132,6 +198,23 @@ To update a streaming destination's custom HTTP headers: 20 headers per streaming destination. 1. Select **Save** to update the streaming destination. +### Google Cloud Logging streaming + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124384) in GitLab 16.2. + +Prerequisites: + +- Owner role for a top-level group. + +To update Google Cloud Logging streaming destinations to a top-level group: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Secure > Audit events**. +1. On the main area, select **Streams** tab. +1. Select the Google Cloud Logging stream to expand. +1. Enter the Google Project ID, Google Client Email, Log ID, and Google Private Key to update. +1. Select **Save** to update the streaming destination. + ## Delete streaming destinations Delete streaming destinations for a top-level group or an entire instance. When the last destination is successfully @@ -148,14 +231,16 @@ To delete a streaming destination: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select the **Streams** tab. -1. To the right of the item, select **Delete** (**{remove}**). +1. Select the stream URL to expand. +1. Select **Delete destination**. +1. Confirm by selecting **Delete destination** in the modal. To delete only the custom HTTP headers for a streaming destination: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select the **Streams** tab. -1. To the right of the item, **Edit** (**{pencil}**). +1. Select the stream URL to expand. 1. Locate the **Custom HTTP headers** table. 1. Locate the header that you wish to remove. 1. To the right of the header, select **Delete** (**{remove}**). @@ -163,11 +248,12 @@ To delete only the custom HTTP headers for a streaming destination: ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `instance_streaming_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`instance_streaming_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -179,7 +265,38 @@ To delete the streaming destinations for an instance: 1. Select **Admin Area**. 1. On the left sidebar, select **Monitoring > Audit Events**. 1. On the main area, select the **Streams** tab. -1. To the right of the item, select **Delete** (**{remove}**). +1. Select the stream URL to expand. +1. Select **Delete destination**. +1. Confirm by selecting **Delete destination** in the modal. + +To delete only the custom HTTP headers for a streaming destination: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **Monitoring > Audit Events**. +1. On the main area, select the **Streams** tab. +1. To the right of the item, **Edit** (**{pencil}**). +1. Locate the **Custom HTTP headers** table. +1. Locate the header that you wish to remove. +1. To the right of the header, select **Delete** (**{remove}**). +1. Select **Save** to update the streaming destination. + +### Google Cloud Logging streaming + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124384) in GitLab 16.2. + +Prerequisites: + +- Owner role for a top-level group. + +To delete Google Cloud Logging streaming destinations to a top-level group: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Secure > Audit events**. +1. On the main area, select the **Streams** tab. +1. Select the Google Cloud Logging stream to expand. +1. Select **Delete destination**. +1. Confirm by selecting **Delete destination** in the modal. ## Verify event authenticity @@ -204,15 +321,17 @@ To list streaming destinations and see the verification tokens: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Secure > Audit events**. 1. On the main area, select the **Streams**. -1. View the verification token on the right side of each item. +1. Select the stream URL to expand. +1. Locate the **Verification token** input. ### Instance streaming destinations -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `instance_streaming_audit_events`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/398107) in GitLab 16.1 [with a flag](../feature_flags.md) named `ff_external_audit_events`. Disabled by default. +> - [Feature flag `ff_external_audit_events`](https://gitlab.com/gitlab-org/gitlab/-/issues/393772) enabled by default in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../feature_flags.md) named -`instance_streaming_audit_events`. On GitLab.com, this feature is not available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is enabled. To disable it, an administrator can [disable the feature flag](../feature_flags.md) named +`ff_external_audit_events`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is ready for production use. Prerequisites: @@ -226,6 +345,25 @@ To list streaming destinations for an instance and see the verification tokens: 1. On the main area, select the **Streams**. 1. View the verification token on the right side of each item. +## Event type filters + +> Event type filtering in the UI with a defined list of audit event types [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/413581) in GitLab 16.1. + +When this feature is enabled for a group, you can permit users to filter streamed audit events per destination. +If the feature is enabled with no filters, the destination receives all audit events. + +A streaming destination that has an event type filter set has a **filtered** (**{filter}**) label. + +To update a streaming destination's event filters: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Secure > Audit events**. +1. On the main area, select the **Streams** tab. +1. Select the stream URL to expand. +1. Locate the **Filter by audit event type** dropdown. +1. Select the dropdown list and select or clear the required event types. +1. Select **Save** to update the event filters. + ## Override default content type header By default, streaming destinations use a `content-type` header of `application/x-www-form-urlencoded`. However, you diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index e924da39145..69c97982562 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -49,7 +49,7 @@ You can view audit events scoped to a group or project. To view a group's audit events: 1. Go to the group. -1. On the left sidebar, select **Security and Compliance > Audit Events**. +1. On the left sidebar, select **Secure > Audit events**. Group events do not include project audit events. Group events can also be accessed using the [Group Audit Events API](../api/audit_events.md#group-audit-events). Group event queries are limited to a maximum of 30 @@ -58,7 +58,7 @@ days. To view a project's audit events: 1. Go to the project. -1. On the left sidebar, select **Security & Compliance > Audit Events**. +1. On the left sidebar, select **Secure > Audit events**. Project events can also be accessed using the [Project Audit Events API](../api/audit_events.md#project-audit-events). Project event queries are limited to a maximum of 30 days. @@ -119,7 +119,7 @@ The first row contains the headers, which are listed in the following table alon Successful sign-in events are the only audit events available at all tiers. To see successful sign-in events: -1. Select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile > Authentication log**. After upgrading to a paid tier, you can also see successful sign-in events on audit event pages. @@ -140,7 +140,7 @@ From audit events pages, different filters are available depending on the page y > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/536) in GitLab 13.0. > - Impersonation session events included in group audit events in GitLab 14.8. -When a user is [impersonated](../user/admin_area/index.md#user-impersonation), their actions are logged as audit events +When a user is [impersonated](../administration/admin_area.md#user-impersonation), their actions are logged as audit events with additional details: - Audit events include information about the impersonating administrator. These audit events are visible in audit event @@ -362,6 +362,10 @@ GitLab generates audit events when a cluster agent token is created or revoked. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276250) in GitLab 13.6, audit events for when a user is approved using the Admin Area. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276921) in GitLab 13.6, audit events for when a user's personal access token is successfully or unsuccessfully created or revoked. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298783) in GitLab 13.9, audit events for when a user requests access to an instance or is rejected using the Admin Area. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238177) in + GitLab 15.1, audit events when a user's two-factor authentication is disabled. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124169) in GitLab 16.2, audit events when a user's access is locked. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124973) in GitLab 16.2, audit events when a user's access is unlocked. The following user actions on a GitLab instance generate instance audit events: @@ -373,8 +377,9 @@ The following user actions on a GitLab instance generate instance audit events: - Grant OAuth access. - Failed second-factor authentication attempt. - A user's personal access token was successfully or unsuccessfully created or revoked. -- A user's two-factor authentication was disabled. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/238177) in - GitLab 15.1. +- A user's two-factor authentication was disabled. +- A user's access is locked. +- A user's access is unlocked. #### User management @@ -400,6 +405,13 @@ The following user actions on a GitLab instance generate instance audit events: Instance events can also be accessed using the [Instance Audit Events API](../api/audit_events.md#instance-audit-events). +#### Application settings **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282428) in GitLab 16.2. + +When a user changes an application setting in an instance, project, or group, +that change and the user that made the change are recorded in the audit log. + ### GitLab Runner events > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335509) in GitLab 14.8, audit events for when a runner is registered. diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index 3b6992c92e0..e8ed0eb4313 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Users with auditor access have read-only access to all groups, projects, and other resources except: -- The [Admin Area](../user/admin_area/index.md). +- The [Admin Area](../administration/admin_area.md). - Project and group settings. For more information, see [Auditor user permissions and restrictions](#auditor-user-permissions-and-restrictions) @@ -50,9 +50,9 @@ users with auditor access have the same [permissions](../user/permissions.md) as If you are signed in with auditor access, you: -- Have full access to projects you own. -- Have read-only access to projects you aren't a member of. -- Have [permissions](../user/permissions.md) based on your role to projects you are a member of. For example, if you have the Developer role, +- Have full access to the projects and groups you own. +- Have read-only access to the projects and groups you are not a member of. +- Have [permissions](../user/permissions.md) based on your role to projects and groups you are a member of. For example, if you have the Developer role, you can push commits or comment on issues. - Can access the same resources using the GitLab UI or API. - Can't view the Admin Area, or perform any administration actions. diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index a4484da5940..1905a009eb6 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -25,7 +25,7 @@ Users added through LDAP: - Usually use a [licensed seat](../../../subscriptions/self_managed/index.md#billable-users). - Can authenticate with Git using either their GitLab username or their email and LDAP password, even if password authentication for Git - [is disabled](../../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). + [is disabled](../../settings/sign_in_restrictions.md#password-authentication-enabled). The LDAP DN is associated with existing GitLab users when: @@ -1000,13 +1000,13 @@ authenticated with the TLS protocol. Users deleted from the LDAP server: - Are immediately blocked from signing in to GitLab. -- [No longer consume a license](../../../user/admin_area/moderate_users.md). +- [No longer consume a license](../../../administration/moderate_users.md). However, these users can continue to use Git with SSH until the next time the [LDAP check cache runs](ldap_synchronization.md#adjust-ldap-user-sync-schedule). To delete the account immediately, you can manually -[block the user](../../../user/admin_area/moderate_users.md#block-a-user). +[block the user](../../../administration/moderate_users.md#block-a-user). ## Update user email addresses diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md index 9fb3888b995..8cffff7b725 100644 --- a/doc/administration/auth/ldap/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap/ldap-troubleshooting.md @@ -378,7 +378,7 @@ things to debug the situation. an LDAP DN as the `Identifier`. If not, this user hasn't signed in with LDAP yet and must do so first. - You've waited an hour or [the configured interval](ldap_synchronization.md#adjust-ldap-group-sync-schedule) for - the group to sync. To speed up the process, either go to the GitLab group **Group information > Members** + the group to sync. To speed up the process, either go to the GitLab group **Manage > Members** and press **Sync now** (sync one group) or [run the group sync Rake task](../../raketasks/ldap.md#run-a-group-sync) (sync all groups). @@ -447,7 +447,7 @@ A displayed error can identify the problem and point to a solution. For example, ```ruby irb(main):018:0> group.members.map(&:errors).map(&:full_messages) -=> [["The member's email address is not allowed for this group. Go to the group’s 'Settings > General' page, and check 'Restrict membership by email domain'."]] +=> [["The member's email address is not allowed for this group. Go to the group's 'Settings > General' page, and check 'Restrict membership by email domain'."]] ``` This error showed that an Administrator chose to [restrict group membership by email domain](../../../user/group/access_and_permissions.md#restrict-group-access-by-domain), @@ -737,7 +737,7 @@ To resolve this error, you must apply a new license to the GitLab instance witho 1. Remove or comment out the GitLab configuration lines for all non-primary LDAP servers. 1. [Reconfigure GitLab](../../restart_gitlab.md#reconfigure-a-linux-package-installation) so that it temporarily uses only one LDAP server. -1. Enter the [Rails console and add the license key](../../../user/admin_area/license_file.md#add-a-license-through-the-console). +1. Enter the [Rails console and add the license key](../../../administration/license_file.md#add-a-license-through-the-console). 1. Re-enable the additional LDAP servers in the GitLab configuration and reconfigure GitLab again. ## Users are being removed from group and re-added again diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md index e4b43feacc2..9d9ed563fe5 100644 --- a/doc/administration/auth/ldap/ldap_synchronization.md +++ b/doc/administration/auth/ldap/ldap_synchronization.md @@ -40,7 +40,7 @@ For more information, see [Bitmask Searches in LDAP](https://ctovswild.com/2009/ The process also updates the following user information: - Name. Because of a [sync issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342598), `name` is not synchronized if - [**Prevent users from changing their profile name**](../../../user/admin_area/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled or `sync_name` is set to `false`. + [**Prevent users from changing their profile name**](../../../administration/settings/account_and_limit_settings.md#disable-user-profile-name-changes) is enabled or `sync_name` is set to `false`. - Email address. - SSH public keys if `sync_ssh_keys` is set. - Kerberos identity if Kerberos is enabled. @@ -627,7 +627,7 @@ sync to run once every two hours at the top of the hour. ### External groups Using the `external_groups` setting allows you to mark all users belonging -to these groups as [external users](../../../user/admin_area/external_users.md). +to these groups as [external users](../../../administration/external_users.md). Group membership is checked periodically through the `LdapGroupSync` background task. diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index 23d2ab512db..d48de109bd0 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -757,8 +757,8 @@ For more information, see the [GitLab API user method documentation](https://pyt You can configure OIDC group membership to: - Require users to be members of a certain group. -- Assign users [external roles](../../user/admin_area/external_users.md), or as - administrators based on group membership. +- Assign users [external](../external_users.md), administrator or + [auditor](../auditor_users.md) roles based on group membership. GitLab checks these groups on each sign in and updates user attributes as necessary. This feature **does not** allow you to automatically add users to GitLab @@ -845,12 +845,12 @@ For self-compiled installations: ### External groups Your IdP must pass group information to GitLab in the OIDC response. To use this -response to identify users as [external users](../../user/admin_area/external_users.md) +response to identify users as [external users](../external_users.md) based on group membership, configure GitLab to identify: - Where to look for the groups in the OIDC response, using the `groups_attribute` setting. - Which group memberships should identify a user as an - [external user](../../user/admin_area/external_users.md), using the + [external user](../external_users.md), using the `external_groups` setting. For Linux package installations: @@ -921,6 +921,83 @@ For self-compiled installations: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +### Auditor groups **(PREMIUM SELF)** + +Your IdP must pass group information to GitLab in the OIDC response. To use this +response to assign users as auditors based on group membership, configure GitLab to identify: + +- Where to look for the groups in the OIDC response, using the `groups_attribute` setting. +- Which group memberships grant the user auditor access, using the `auditor_groups` + setting. + +For Linux package installations: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['omniauth_providers'] = [ + { + name: "openid_connect", + label: "Provider name", + args: { + name: "openid_connect", + scope: ["openid","profile","email","groups"], + response_type: "code", + issuer: "<your_oidc_url>", + discovery: true, + client_auth_method: "query", + uid_field: "<uid_field>", + client_options: { + identifier: "<your_oidc_client_id>", + secret: "<your_oidc_client_secret>", + redirect_uri: "<your_gitlab_url>/users/auth/openid_connect/callback", + gitlab: { + groups_attribute: "groups", + auditor_groups: ["Auditor"] + } + } + } + } + ] + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect. + +For self-compiled installations: + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: &base + omniauth: + providers: + - { name: 'openid_connect', + label: 'Provider name', + args: { + name: 'openid_connect', + scope: ['openid','profile','email','groups'], + response_type: 'code', + issuer: '<your_oidc_url>', + discovery: true, + client_auth_method: 'query', + uid_field: '<uid_field>', + client_options: { + identifier: '<your_oidc_client_id>', + secret: '<your_oidc_client_secret>', + redirect_uri: '<your_gitlab_url>/users/auth/openid_connect/callback', + gitlab: { + groups_attribute: "groups", + auditor_groups: ["Auditor"] + } + } + } + } + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect. + ### Administrator groups Your IdP must pass group information to GitLab in the OIDC response. To use this diff --git a/doc/administration/auth/smartcard.md b/doc/administration/auth/smartcard.md index 9432a627ed7..5802db78dd6 100644 --- a/doc/administration/auth/smartcard.md +++ b/doc/administration/auth/smartcard.md @@ -17,7 +17,7 @@ By default, existing users can continue to sign in with a username and password authentication is enabled. To force existing users to use only smartcard authentication, -[disable username and password authentication](../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). +[disable username and password authentication](../settings/sign_in_restrictions.md#password-authentication-enabled). ## Authentication methods diff --git a/doc/administration/backup_restore/backup_gitlab.md b/doc/administration/backup_restore/backup_gitlab.md new file mode 100644 index 00000000000..24b7b453517 --- /dev/null +++ b/doc/administration/backup_restore/backup_gitlab.md @@ -0,0 +1,1925 @@ +--- +stage: Systems +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Back up GitLab **(FREE SELF)** + +The exact procedure for backing up GitLab depends on many factors. Your particular deployment's usage and configuration determine what kind of data exists, where it is located, and how much there is. These factors influence your options for how to perform a back up, how to store it, and how to restore it. + +## Simple back up procedure + +As a rough guideline, if you are using a [1k reference architecture](../reference_architectures/1k_users.md) with less than 100 GB of data, then follow these steps: + +1. Run the [backup command](#backup-command). +1. Back up [object storage](#object-storage), if applicable. +1. Manually back up [configuration files](#storing-configuration-files). + +## Scaling backups + +As the volume of GitLab data grows, the [backup command](#backup-command) takes longer to execute. At some point, the execution time becomes impractical. For example, it can take 24 hours or more. + +For more information, see [alternative backup strategies](#alternative-backup-strategies). + +## What data needs to be backed up? + +- [PostgreSQL databases](#postgresql-databases) +- [Git repositories](#git-repositories) +- [Blobs](#blobs) +- [Configuration files](#storing-configuration-files) +- [Other data](#other-data) + +### PostgreSQL databases + +In the simplest case, GitLab has one PostgreSQL database in one PostgreSQL server on the same VM as all other GitLab services. But depending on configuration, GitLab may use multiple PostgreSQL databases in multiple PostgreSQL servers. + +In general, this data is the single source of truth for most user-generated content in the Web interface, such as issue and merge request content, comments, permissions, and credentials. + +PostgreSQL also holds some cached data like HTML-rendered Markdown, and by default, merge request diffs. +However, merge request diffs can also be configured to be offloaded to the file system or object storage, see [Blobs](#blobs). + +Gitaly Cluster's Praefect service uses a PostgreSQL database as a single source of truth to manage its Gitaly nodes. + +A common PostgreSQL utility, [`pg_dump`](https://www.postgresql.org/docs/current/app-pgdump.html), produces a backup file which can be used to restore a PostgreSQL database. The [backup command](#backup-command) uses this utility under the hood. + +Unfortunately, the larger the database, the longer it takes `pg_dump` to execute. Depending on your situation, the duration becomes impractical at some point (days, for example). If your database is over 100 GB, `pg_dump`, and by extension the [backup command](#backup-command), is likely not usable. For more information, see [alternative backup strategies](#alternative-backup-strategies). + +### Git repositories + +A GitLab instance can have one or more repository shards. Each shard is a Gitaly instance or Gitaly Cluster that +is responsible for allowing access and operations on the locally stored Git repositories. Gitaly can run +on a machine: + +- With a single disk. +- With multiple disks mounted as a single mount-point (like with a RAID array). +- Using LVM. + +Each project can have up to 3 different repositories: + +- A project repository, where the source code is stored. +- A wiki repository, where the wiki content is stored. +- A design repository, where design artifacts are indexed (assets are actually in LFS). + +They all live in the same shard and share the same base name with a `-wiki` and `-design` suffix +for Wiki and Design Repository cases. + +Personal and project snippets, and group wiki content, are stored in Git repositories. + +Project forks are deduplicated in live a GitLab site using pool repositories. + +The [backup command](#backup-command) produces a Git bundle for each repository and tars them all up. This duplicates pool repository data into every fork. In [our testing](https://gitlab.com/gitlab-org/gitlab/-/issues/396343), 100 GB of Git repositories took a little over 2 hours to back up and upload to S3. At around 400 GB of Git data, the backup command is likely not viable for regular backups. For more information, see [alternative backup strategies](#alternative-backup-strategies). + +### Blobs + +GitLab stores blobs (or files) such as issue attachments or LFS objects into either: + +- The file system in a specific location. +- An [Object Storage](../object_storage.md) solution. Object Storage solutions can be: + - Cloud based like Amazon S3 and Google Cloud Storage. + - Hosted by you (like MinIO). + - A Storage Appliance that exposes an Object Storage-compatible API. + +#### Object storage + +The [backup command](#backup-command) doesn't back up blobs that aren't stored on the file system. If you're using [object storage](../object_storage.md), be sure to enable backups with your object storage provider. For example, see: + +- [Amazon S3 backups](https://docs.aws.amazon.com/aws-backup/latest/devguide/s3-backups.html) +- [Google Cloud Storage Transfer Service](https://cloud.google.com/storage-transfer-service) and [Google Cloud Storage Object Versioning](https://cloud.google.com/storage/docs/object-versioning) + +### Storing configuration files + +WARNING: +The [backup Rake task](#back-up-gitlab) GitLab provides does _not_ store your configuration files. The primary reason for this is that your database contains items including encrypted information for two-factor authentication and the CI/CD _secure variables_. Storing encrypted information in the same location as its key defeats the purpose of using encryption in the first place. For example, the secrets file contains your database encryption key. If you lose it, then the GitLab application will not be able to decrypt any encrypted values in the database. + +WARNING: +The secrets file may change after upgrades. + +You should back up the configuration directory. At the very **minimum**, you must back up: + +::Tabs + +:::TabTitle Linux package + +- `/etc/gitlab/gitlab-secrets.json` +- `/etc/gitlab/gitlab.rb` + +For more information, see [Backup and restore Linux package (Omnibus) configuration](https://docs.gitlab.com/omnibus/settings/backups.html#backup-and-restore-omnibus-gitlab-configuration). + +:::TabTitle Self-compiled + +- `/home/git/gitlab/config/secrets.yml` +- `/home/git/gitlab/config/gitlab.yml` + +:::TabTitle Docker + +- Back up the volume where the configuration files are stored. If you created +the GitLab container according to the documentation, it should be in the +`/srv/gitlab/config` directory. + +:::TabTitle GitLab Helm chart + +- Follow the [Back up the secrets](https://docs.gitlab.com/charts/backup-restore/backup.html#back-up-the-secrets) +instructions. + +::EndTabs + +You may also want to back up any TLS keys and certificates (`/etc/gitlab/ssl`, `/etc/gitlab/trusted-certs`), and your +[SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079) +to avoid man-in-the-middle attack warnings if you have to perform a full machine restore. + +In the unlikely event that the secrets file is lost, see the +[troubleshooting section](#when-the-secrets-file-is-lost). + +### Other data + +GitLab uses Redis both as a cache store and to hold persistent data for our background jobs system, Sidekiq. The provided [backup command](#backup-command) does _not_ back up Redis data. This means that in order to take a consistent backup with the [backup command](#backup-command), there must be no pending or running background jobs. It is possible to [manually back up Redis](https://redis.io/docs/management/persistence/#backing-up-redis-data). + +Elasticsearch is an optional database for advanced search. It can improve search +in both source-code level, and user generated content in issues, merge requests, and discussions. The [backup command](#backup-command) does _not_ back up Elasticsearch data. Elasticsearch data can be regenerated from PostgreSQL data after a restore. It is possible to [manually back up Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshot-restore.html). + +## Command line interface + +GitLab provides a command line interface to back up your entire instance, +including: + +- Database +- Attachments +- Git repositories data +- CI/CD job output logs +- CI/CD job artifacts +- LFS objects +- Terraform states ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331806) in GitLab 14.7) +- Container Registry images +- GitLab Pages content +- Packages ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332006) in GitLab 14.7) +- Snippets +- [Group wikis](../../user/project/wiki/group.md) +- Project-level Secure Files ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121142) in GitLab 16.1) + +Backups do not include: + +- [Mattermost data](https://docs.mattermost.com/administration/config-settings.html#file-storage) +- Redis (and thus Sidekiq jobs) +- [Object storage](#object-storage) + +WARNING: +GitLab does not back up any configuration files (`/etc/gitlab`), TLS keys and certificates, or system +files. You are highly advised to read about [storing configuration files](#storing-configuration-files). + +### Requirements + +To be able to back up and restore, ensure that Rsync is installed on your +system. If you installed GitLab: + +- Using the Linux package, Rsync is already installed. +- Using self-compiled, check if `rsync` is installed. If Rsync is not installed, install it. For example: + + ```shell + # Debian/Ubuntu + sudo apt-get install rsync + + # RHEL/CentOS + sudo yum install rsync + ``` + +### Backup command + +WARNING: +The backup command does not back up items in [object storage](#object-storage). + +WARNING: +The backup command requires [additional parameters](#back-up-and-restore-for-installations-using-pgbouncer) when +your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. + +WARNING: +Before GitLab 15.5.0, the backup command doesn't verify if another backup is already running, as described in +[issue 362593](https://gitlab.com/gitlab-org/gitlab/-/issues/362593). We strongly recommend +you make sure that all backups are complete before starting a new one. + +NOTE: +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create +``` + +:::TabTitle Helm chart (Kubernetes) + +Run the backup task by using `kubectl` to run the `backup-utility` script on the GitLab toolbox pod. For more details, see the [charts backup documentation](https://docs.gitlab.com/charts/backup-restore/backup.html). + +:::TabTitle Docker + +Run the backup from the host. + +- GitLab 12.2 or later: + +```shell +docker exec -t <container name> gitlab-backup create +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +::EndTabs + +If your GitLab deployment has multiple nodes, you need to pick a node for running the backup command. You must ensure that the designated node: + +- is persistent, and not subject to auto-scaling. +- has the GitLab Rails application already installed. If Puma or Sidekiq is running, then Rails is installed. +- has sufficient storage and memory to produce the backup file. + +Example output: + +```plaintext +Dumping database tables: +- Dumping table events... [DONE] +- Dumping table issues... [DONE] +- Dumping table keys... [DONE] +- Dumping table merge_requests... [DONE] +- Dumping table milestones... [DONE] +- Dumping table namespaces... [DONE] +- Dumping table notes... [DONE] +- Dumping table projects... [DONE] +- Dumping table protected_branches... [DONE] +- Dumping table schema_migrations... [DONE] +- Dumping table services... [DONE] +- Dumping table snippets... [DONE] +- Dumping table taggings... [DONE] +- Dumping table tags... [DONE] +- Dumping table users... [DONE] +- Dumping table users_projects... [DONE] +- Dumping table web_hooks... [DONE] +- Dumping table wikis... [DONE] +Dumping repositories: +- Dumping repository abcd... [DONE] +Creating backup archive: $TIMESTAMP_gitlab_backup.tar [DONE] +Deleting tmp directories...[DONE] +Deleting old backups... [SKIPPING] +``` + +### Backup timestamp + +The backup archive is saved in `backup_path`, which is specified in the +`config/gitlab.yml` file. The default path is `/var/opt/gitlab/backups`. The filename is `[TIMESTAMP]_gitlab_backup.tar`, +where `TIMESTAMP` identifies the time at which each backup was created, plus +the GitLab version. The timestamp is needed if you need to restore GitLab and +multiple backups are available. + +For example, if the backup name is `1493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar`, +the timestamp is `1493107454_2018_04_25_10.6.4-ce`. + +### Backup options + +The command line tool GitLab provides to back up your instance can accept more +options. + +#### Backup strategy option + +The default backup strategy is to essentially stream data from the respective +data locations to the backup using the Linux command `tar` and `gzip`. This works +fine in most cases, but can cause problems when data is rapidly changing. + +When data changes while `tar` is reading it, the error `file changed as we read +it` may occur, and causes the backup process to fail. In that case, you can use +the backup strategy called `copy`. The strategy copies data files +to a temporary location before calling `tar` and `gzip`, avoiding the error. + +A side-effect is that the backup process takes up to an additional 1X disk +space. The process does its best to clean up the temporary files at each stage +so the problem doesn't compound, but it could be a considerable change for large +installations. + +To use the `copy` strategy instead of the default streaming strategy, specify +`STRATEGY=copy` in the Rake task command. For example: + +```shell +sudo gitlab-backup create STRATEGY=copy +``` + +#### Backup filename + +WARNING: +If you use a custom backup filename, you can't +[limit the lifetime of the backups](#limit-backup-lifetime-for-local-files-prune-old-backups). + +By default, a backup file is created according to the specification in the +previous [Backup timestamp](#backup-timestamp) section. You can, however, +override the `[TIMESTAMP]` portion of the filename by setting the `BACKUP` +environment variable. For example: + +```shell +sudo gitlab-backup create BACKUP=dump +``` + +The resulting file is named `dump_gitlab_backup.tar`. This is useful for +systems that make use of rsync and incremental backups, and results in +considerably faster transfer speeds. + +#### Confirm archive can be transferred + +To ensure the generated archive is transferable by rsync, you can set the `GZIP_RSYNCABLE=yes` +option. This sets the `--rsyncable` option to `gzip`, which is useful only in +combination with setting [the Backup filename option](#backup-filename). + +The `--rsyncable` option in `gzip` isn't guaranteed to be available +on all distributions. To verify that it's available in your distribution, run +`gzip --help` or consult the man pages. + +```shell +sudo gitlab-backup create BACKUP=dump GZIP_RSYNCABLE=yes +``` + +#### Excluding specific directories from the backup + +You can exclude specific directories from the backup by adding the environment variable `SKIP`, whose values are a comma-separated list of the following options: + +- `db` (database) +- `uploads` (attachments) +- `builds` (CI job output logs) +- `artifacts` (CI job artifacts) +- `lfs` (LFS objects) +- `terraform_state` (Terraform states) +- `registry` (Container Registry images) +- `pages` (Pages content) +- `repositories` (Git repositories data) +- `packages` (Packages) +- `ci_secure_files` (Project-level Secure Files) + +NOTE: +When [backing up and restoring Helm Charts](https://docs.gitlab.com/charts/architecture/backup-restore.html), there is an additional option `packages`, which refers to any packages managed by the GitLab [package registry](../../user/packages/package_registry/index.md). +For more information see [command line arguments](https://docs.gitlab.com/charts/architecture/backup-restore.html#command-line-arguments). + +All wikis are backed up as part of the `repositories` group. Non-existent +wikis are skipped during a backup. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create SKIP=db,uploads +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=production +``` + +::EndTabs + +`SKIP=` is also used to: + +- [Skip creation of the tar file](#skipping-tar-creation) (`SKIP=tar`). +- [Skip uploading the backup to remote storage](#skip-uploading-backups-to-remote-storage) (`SKIP=remote`). + +#### Skipping tar creation + +NOTE: +It is not possible to skip the tar creation when using [object storage](#upload-backups-to-a-remote-cloud-storage) for backups. + +The last part of creating a backup is generation of a `.tar` file containing all the parts. In some cases, creating a `.tar` file might be wasted effort or even directly harmful, so you can skip this step by adding `tar` to the `SKIP` environment variable. Example use-cases: + +- When the backup is picked up by other backup software. +- To speed up incremental backups by avoiding having to extract the backup every time. (In this case, `PREVIOUS_BACKUP` and `BACKUP` must not be specified, otherwise the specified backup is extracted, but no `.tar` file is generated at the end.) + +Adding `tar` to the `SKIP` variable leaves the files and directories containing the +backup in the directory used for the intermediate files. These files are +overwritten when a new backup is created, so you should make sure they are copied +elsewhere, because you can only have one backup on the system. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create SKIP=tar +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create SKIP=tar RAILS_ENV=production +``` + +::EndTabs + +#### Back up Git repositories concurrently + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3. +> - [Concurrent restore introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69330) in GitLab 14.3 + +When using [multiple repository storages](../repository_storage_paths.md), +repositories can be backed up or restored concurrently to help fully use CPU time. The +following variables are available to modify the default behavior of the Rake +task: + +- `GITLAB_BACKUP_MAX_CONCURRENCY`: The maximum number of projects to back up at + the same time. Defaults to the number of logical CPUs (in GitLab 14.1 and + earlier, defaults to `1`). +- `GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY`: The maximum number of projects to + back up at the same time on each storage. This allows the repository backups + to be spread across storages. Defaults to `2` (in GitLab 14.1 and earlier, + defaults to `1`). + +For example, with 4 repository storages: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create GITLAB_BACKUP_MAX_CONCURRENCY=4 GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY=1 +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create GITLAB_BACKUP_MAX_CONCURRENCY=4 GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY=1 +``` + +::EndTabs + +#### Incremental repository backups + +> - Introduced in GitLab 14.9 [with a flag](../feature_flags.md) named `incremental_repository_backup`. Disabled by default. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/355945) in GitLab 14.10. +> - `PREVIOUS_BACKUP` option [introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4184) in GitLab 15.0. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../feature_flags.md) named `incremental_repository_backup`. +On GitLab.com, this feature is not available. + +NOTE: +Only repositories support incremental backups. Therefore, if you use `INCREMENTAL=yes`, the task +creates a self-contained backup tar archive. This is because all subtasks except repositories are +still creating full backups (they overwrite the existing full backup). +See [issue 19256](https://gitlab.com/gitlab-org/gitlab/-/issues/19256) for a feature request to +support incremental backups for all subtasks. + +Incremental repository backups can be faster than full repository backups because they only pack changes since the last backup into the backup bundle for each repository. +The incremental backup archives are not linked to each other: each archive is a self-contained backup of the instance. There must be an existing backup +to create an incremental backup from: + +- In GitLab 14.9 and 14.10, use the `BACKUP=<timestamp_of_backup>` option to choose the backup to use. The chosen previous backup is overwritten. +- In GitLab 15.0 and later, use the `PREVIOUS_BACKUP=<timestamp_of_backup>` option to choose the backup to use. By default, a backup file is created + as documented in the [Backup timestamp](#backup-timestamp) section. You can override the `[TIMESTAMP]` portion of the filename by setting the + [`BACKUP` environment variable](#backup-filename). + +To create an incremental backup, run: + +- In GitLab 15.0 or later: + + ```shell + sudo gitlab-backup create INCREMENTAL=yes PREVIOUS_BACKUP=<timestamp_of_backup> + ``` + +- In GitLab 14.9 and 14.10: + + ```shell + sudo gitlab-backup create INCREMENTAL=yes BACKUP=<timestamp_of_backup> + ``` + +To create an [untarred](#skipping-tar-creation) incremental backup from a tarred backup, use `SKIP=tar`: + +```shell +sudo gitlab-backup create INCREMENTAL=yes SKIP=tar +``` + +#### Back up specific repository storages + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86896) in GitLab 15.0. + +When using [multiple repository storages](../repository_storage_paths.md), +repositories from specific repository storages can be backed up separately +using the `REPOSITORIES_STORAGES` option. The option accepts a comma-separated list of +storage names. + +For example: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create REPOSITORIES_STORAGES=storage1,storage2 +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_STORAGES=storage1,storage2 +``` + +::EndTabs + +#### Back up specific repositories + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. + +You can back up specific repositories using the `REPOSITORIES_PATHS` option. +Similarly, you can use `SKIP_REPOSITORIES_PATHS` to skip certain repositories. +Both options accept a comma-separated list of project or group paths. If you +specify a group path, all repositories in all projects in the group and +descendent groups are included or skipped, depending on which option you used. + +For example, to back up all repositories for all projects in **Group A** (`group-a`), the repository for **Project C** in **Group B** (`group-b/project-c`), +and skip the **Project D** in **Group A** (`group-a/project-d`): + +::Tabs + +:::TabTitle Linux package (Omnibus) + + ```shell + sudo gitlab-backup create REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d + ``` + +:::TabTitle Self-compiled + + ```shell + sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d + ``` + +::EndTabs + +#### Upload backups to a remote (cloud) storage + +NOTE: +It is not possible to [skip the tar creation](#skipping-tar-creation) when using object storage for backups. + +You can let the backup script upload (using the [Fog library](https://fog.io/)) +the `.tar` file it creates. In the following example, we use Amazon S3 for +storage, but Fog also lets you use [other storage providers](https://fog.io/storage/). +GitLab also [imports cloud drivers](https://gitlab.com/gitlab-org/gitlab/-/blob/da46c9655962df7d49caef0e2b9f6bbe88462a02/Gemfile#L113) +for AWS, Google, and Aliyun. A local driver is +[also available](#upload-to-locally-mounted-shares). + +[Read more about using object storage with GitLab](../object_storage.md). + +##### Using Amazon S3 + +For Linux package (Omnibus): + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-west-1', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123' + # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key + # 'use_iam_profile' => true + } + gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' + # Consider using multipart uploads when file size reaches 100MB. Enter a number in bytes. + # gitlab_rails['backup_multipart_chunk_size'] = 104857600 + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect + +##### S3 Encrypted Buckets + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64765) in GitLab 14.3. + +AWS supports these [modes for server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html): + +- Amazon S3-Managed Keys (SSE-S3) +- Customer Master Keys (CMKs) Stored in AWS Key Management Service (SSE-KMS) +- Customer-Provided Keys (SSE-C) + +Use your mode of choice with GitLab. Each mode has similar, but slightly +different, configuration methods. + +###### SSE-S3 + +To enable SSE-S3, in the backup storage options set the `server_side_encryption` +field to `AES256`. For example, in the Linux package (Omnibus): + +```ruby +gitlab_rails['backup_upload_storage_options'] = { + 'server_side_encryption' => 'AES256' +} +``` + +###### SSE-KMS + +To enable SSE-KMS, you need the +[KMS key via its Amazon Resource Name (ARN) in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). +Under the `backup_upload_storage_options` configuration setting, set: + +- `server_side_encryption` to `aws:kms`. +- `server_side_encryption_kms_key_id` to the ARN of the key. + +For example, in the Linux package (Omnibus): + +```ruby +gitlab_rails['backup_upload_storage_options'] = { + 'server_side_encryption' => 'aws:kms', + 'server_side_encryption_kms_key_id' => 'arn:aws:<YOUR KMS KEY ID>:' +} +``` + +###### SSE-C + +SSE-C requires you to set these encryption options: + +- `backup_encryption`: AES256. +- `backup_encryption_key`: Unencoded, 32-byte (256 bits) key. The upload fails if this isn't exactly 32 bytes. + +For example, in the Linux package (Omnibus): + +```ruby +gitlab_rails['backup_encryption'] = 'AES256' +gitlab_rails['backup_encryption_key'] = '<YOUR 32-BYTE KEY HERE>' +``` + +If the key contains binary characters and cannot be encoded in UTF-8, +instead, specify the key with the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable. +For example: + +```ruby +gitlab_rails['env'] = { 'GITLAB_BACKUP_ENCRYPTION_KEY' => "\xDE\xAD\xBE\xEF" * 8 } +``` + +##### Digital Ocean Spaces + +This example can be used for a bucket in Amsterdam (AMS3): + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'ams3', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123', + 'endpoint' => 'https://ams3.digitaloceanspaces.com' + } + gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect + +If you see a `400 Bad Request` error message when using Digital Ocean Spaces, +the cause may be the use of backup encryption. Because Digital Ocean Spaces +doesn't support encryption, remove or comment the line that contains +`gitlab_rails['backup_encryption']`. + +##### Other S3 Providers + +Not all S3 providers are fully compatible with the Fog library. For example, +if you see a `411 Length Required` error message after attempting to upload, +you may need to downgrade the `aws_signature_version` value from the default +value to `2`, [due to this issue](https://github.com/fog/fog-aws/issues/428). + +For installations from source: + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + # snip + upload: + # Fog storage connection settings, see https://fog.io/storage/ . + connection: + provider: AWS + region: eu-west-1 + aws_access_key_id: AKIAKIAKI + aws_secret_access_key: 'secret123' + # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty + # ie. aws_access_key_id: '' + # use_iam_profile: 'true' + # The remote 'directory' to store your backups. For S3, this would be the bucket name. + remote_directory: 'my.s3.bucket' + # Specifies Amazon S3 storage class to use for backups, this is optional + # storage_class: 'STANDARD' + # + # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional + # 'encryption' must be set in order for this to have any effect. + # 'encryption_key' should be set to the 256-bit encryption key for Amazon S3 to use to encrypt or decrypt. + # To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` your data. + # encryption: 'AES256' + # encryption_key: '<key>' + # + # + # Turns on AWS Server-Side Encryption with Amazon S3-Managed keys (optional) + # https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html + # For SSE-S3, set 'server_side_encryption' to 'AES256'. + # For SS3-KMS, set 'server_side_encryption' to 'aws:kms'. Set + # 'server_side_encryption_kms_key_id' to the ARN of customer master key. + # storage_options: + # server_side_encryption: 'aws:kms' + # server_side_encryption_kms_key_id: 'arn:aws:kms:YOUR-KEY-ID-HERE' + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect + +If you're uploading your backups to S3, you should create a new +IAM user with restricted access rights. To give the upload user access only for +uploading backups create the following IAM profile, replacing `my.s3.bucket` +with the name of your bucket: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Stmt1412062044000", + "Effect": "Allow", + "Action": [ + "s3:AbortMultipartUpload", + "s3:GetBucketAcl", + "s3:GetBucketLocation", + "s3:GetObject", + "s3:GetObjectAcl", + "s3:ListBucketMultipartUploads", + "s3:PutObject", + "s3:PutObjectAcl" + ], + "Resource": [ + "arn:aws:s3:::my.s3.bucket/*" + ] + }, + { + "Sid": "Stmt1412062097000", + "Effect": "Allow", + "Action": [ + "s3:GetBucketLocation", + "s3:ListAllMyBuckets" + ], + "Resource": [ + "*" + ] + }, + { + "Sid": "Stmt1412062128000", + "Effect": "Allow", + "Action": [ + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::my.s3.bucket" + ] + } + ] +} +``` + +##### Using Google Cloud Storage + +To use Google Cloud Storage to save backups, you must first create an +access key from the Google console: + +1. Go to the [Google storage settings page](https://console.cloud.google.com/storage/settings). +1. Select **Interoperability**, and then create an access key. +1. Make note of the **Access Key** and **Secret** and replace them in the + following configurations. +1. In the buckets advanced settings ensure the Access Control option + **Set object-level and bucket-level permissions** is selected. +1. Ensure you have already created a bucket. + +For the Linux package (Omnibus): + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'Google', + 'google_storage_access_key_id' => 'Access Key', + 'google_storage_secret_access_key' => 'Secret', + + ## If you have CNAME buckets (foo.example.com), you might run into SSL issues + ## when uploading backups ("hostname foo.example.com.storage.googleapis.com + ## does not match the server certificate"). In that case, uncomnent the following + ## setting. See: https://github.com/fog/fog/issues/2834 + #'path_style' => true + } + gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket' + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect + +For installations from source: + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + upload: + connection: + provider: 'Google' + google_storage_access_key_id: 'Access Key' + google_storage_secret_access_key: 'Secret' + remote_directory: 'my.google.bucket' + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect + +##### Using Azure Blob storage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25877) in GitLab 13.4. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AzureRM', + 'azure_storage_account_name' => '<AZURE STORAGE ACCOUNT NAME>', + 'azure_storage_access_key' => '<AZURE STORAGE ACCESS KEY>', + 'azure_storage_domain' => 'blob.core.windows.net', # Optional + } + gitlab_rails['backup_upload_remote_directory'] = '<AZURE BLOB CONTAINER>' + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect + +:::TabTitle Self-compiled + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + upload: + connection: + provider: 'AzureRM' + azure_storage_account_name: '<AZURE STORAGE ACCOUNT NAME>' + azure_storage_access_key: '<AZURE STORAGE ACCESS KEY>' + remote_directory: '<AZURE BLOB CONTAINER>' + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect + +::EndTabs + +For more details, see the [table of Azure parameters](../object_storage.md#azure-blob-storage). + +##### Specifying a custom directory for backups + +This option works only for remote storage. If you want to group your backups, +you can pass a `DIRECTORY` environment variable: + +```shell +sudo gitlab-backup create DIRECTORY=daily +sudo gitlab-backup create DIRECTORY=weekly +``` + +#### Skip uploading backups to remote storage + +If you have configured GitLab to [upload backups in a remote storage](#upload-backups-to-a-remote-cloud-storage), +you can use the `SKIP=remote` option to skip uploading your backups to the remote storage. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create SKIP=remote +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create SKIP=remote RAILS_ENV=production +``` + +::EndTabs + +#### Upload to locally-mounted shares + +You can send backups to a locally-mounted share (for example, `NFS`,`CIFS`, or `SMB`) using the Fog +[`Local`](https://github.com/fog/fog-local#usage) storage provider. + +To do this, you must set the following configuration keys: + +- `backup_upload_connection.local_root`: mounted directory that backups are copied to. +- `backup_upload_remote_directory`: subdirectory of the `backup_upload_connection.local_root` directory. It is created if it doesn't exist. + If you want to copy the tarballs to the root of your mounted directory, use `.`. + +When mounted, the directory set in the `local_root` key must be owned by either: + +- The `git` user. So, mounting with the `uid=` of the `git` user for `CIFS` and `SMB`. +- The user that you are executing the backup tasks as. For the Linux package (Omnibus), this is the `git` user. + +Because file system performance may affect overall GitLab performance, +[we don't recommend using cloud-based file systems for storage](../nfs.md#avoid-using-cloud-based-file-systems). + +##### Avoid conflicting configuration + +Don't set the following configuration keys to the same path: + +- `gitlab_rails['backup_path']` (`backup.path` for source installations). +- `gitlab_rails['backup_upload_connection'].local_root` (`backup.upload.connection.local_root` for source installations). + +The `backup_path` configuration key sets the local location of the backup file. The `upload` configuration key is +intended for use when the backup file is uploaded to a separate server, perhaps for archival purposes. + +If these configuration keys are set to the same location, the upload feature fails because a backup already exists at +the upload location. This failure causes the upload feature to delete the backup because it assumes it's a residual file +remaining after the failed upload attempt. + +##### Configure uploads to locally-mounted shares + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_upload_connection'] = { + :provider => 'Local', + :local_root => '/mnt/backups' + } + + # The directory inside the mounted folder to copy backups to + # Use '.' to store them in the root directory + gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups' + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect. + +:::TabTitle Self-compiled + +1. Edit `home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + upload: + # Fog storage connection settings, see https://fog.io/storage/ . + connection: + provider: Local + local_root: '/mnt/backups' + # The directory inside the mounted folder to copy backups to + # Use '.' to store them in the root directory + remote_directory: 'gitlab_backups' + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect. + +::EndTabs + +#### Backup archive permissions + +The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`) +have the owner/group `git`/`git` and 0600 permissions by default. This is +meant to avoid other system users reading GitLab data. If you need the backup +archives to have different permissions, you can use the `archive_permissions` +setting. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect. + +:::TabTitle Self-compiled + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + archive_permissions: 0644 # Makes the backup archives world-readable + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect. + +::EndTabs + +#### Configuring cron to make daily backups + +WARNING: +The following cron jobs do not [back up your GitLab configuration files](#storing-configuration-files) +or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). + +You can schedule a cron job that backs up your repositories and GitLab metadata. + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit the crontab for the `root` user: + + ```shell + sudo su - + crontab -e + ``` + +1. There, add the following line to schedule the backup for everyday at 2 AM: + + ```plaintext + 0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1 + ``` + +:::TabTitle Self-compiled + +1. Edit the crontab for the `git` user: + + ```shell + sudo -u git crontab -e + ``` + +1. Add the following lines at the bottom: + + ```plaintext + # Create a full backup of the GitLab repositories and SQL database every day at 2am + 0 2 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 + ``` + +::EndTabs + +The `CRON=1` environment setting directs the backup script to hide all progress +output if there aren't any errors. This is recommended to reduce cron spam. +When troubleshooting backup problems, however, replace `CRON=1` with `--trace` to log verbosely. + +#### Limit backup lifetime for local files (prune old backups) + +WARNING: +The process described in this section doesn't work if you used a [custom filename](#backup-filename) +for your backups. + +To prevent regular backups from using all your disk space, you may want to set a limited lifetime +for backups. The next time the backup task runs, backups older than the `backup_keep_time` are +pruned. + +This configuration option manages only local files. GitLab doesn't prune old +files stored in a third-party [object storage](#upload-backups-to-a-remote-cloud-storage) +because the user may not have permission to list and delete files. It's +recommended that you configure the appropriate retention policy for your object +storage (for example, [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html)). + +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + ## Limit backup lifetime to 7 days - 604800 seconds + gitlab_rails['backup_keep_time'] = 604800 + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect. + +:::TabTitle Self-compiled + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + backup: + ## Limit backup lifetime to 7 days - 604800 seconds + keep_time: 604800 + ``` + +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect. + +::EndTabs + +#### Back up and restore for installations using PgBouncer + +Do not back up or restore GitLab through a PgBouncer connection. These +tasks must [bypass PgBouncer and connect directly to the PostgreSQL primary database node](#bypassing-pgbouncer), +or they cause a GitLab outage. + +When the GitLab backup or restore task is used with PgBouncer, the +following error message is shown: + +```ruby +ActiveRecord::StatementInvalid: PG::UndefinedTable +``` + +Each time the GitLab backup runs, GitLab starts generating 500 errors and errors about missing +tables will [be logged by PostgreSQL](../logs/index.md#postgresql-logs): + +```plaintext +ERROR: relation "tablename" does not exist at character 123 +``` + +This happens because the task uses `pg_dump`, which +[sets a null search path and explicitly includes the schema in every SQL query](https://gitlab.com/gitlab-org/gitlab/-/issues/23211) +to address [CVE-2018-1058](https://www.postgresql.org/about/news/postgresql-103-968-9512-9417-and-9322-released-1834/). + +Because connections are reused with PgBouncer in transaction pooling mode, +PostgreSQL fails to search the default `public` schema. As a result, +this clearing of the search path causes tables and columns to appear +missing. + +##### Bypassing PgBouncer + +There are two ways to fix this: + +1. [Use environment variables to override the database settings](#environment-variable-overrides) for the backup task. +1. Reconfigure a node to [connect directly to the PostgreSQL primary database node](../postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). + +###### Environment variable overrides + +By default, GitLab uses the database configuration stored in a +configuration file (`database.yml`). However, you can override the database settings +for the backup and restore task by setting environment +variables that are prefixed with `GITLAB_BACKUP_`: + +- `GITLAB_BACKUP_PGHOST` +- `GITLAB_BACKUP_PGUSER` +- `GITLAB_BACKUP_PGPORT` +- `GITLAB_BACKUP_PGPASSWORD` +- `GITLAB_BACKUP_PGSSLMODE` +- `GITLAB_BACKUP_PGSSLKEY` +- `GITLAB_BACKUP_PGSSLCERT` +- `GITLAB_BACKUP_PGSSLROOTCERT` +- `GITLAB_BACKUP_PGSSLCRL` +- `GITLAB_BACKUP_PGSSLCOMPRESSION` + +For example, to override the database host and port to use 192.168.1.10 +and port 5432 with the Linux package (Omnibus): + +```shell +sudo GITLAB_BACKUP_PGHOST=192.168.1.10 GITLAB_BACKUP_PGPORT=5432 /opt/gitlab/bin/gitlab-backup create +``` + +See the [PostgreSQL documentation](https://www.postgresql.org/docs/12/libpq-envars.html) +for more details on what these parameters do. + +#### `gitaly-backup` for repository backup and restore + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333034) in GitLab 14.2. +> - [Deployed behind a feature flag](../../user/feature_flags.md), enabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/333034) in GitLab 14.10. [Feature flag `gitaly_backup`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83254) removed. + +The `gitaly-backup` binary is used by the backup Rake task to create and restore repository backups from Gitaly. +`gitaly-backup` replaces the previous backup method that directly calls RPCs on Gitaly from GitLab. + +The backup Rake task must be able to find this executable. In most cases, you don't need to change +the path to the binary as it should work fine with the default path `/opt/gitlab/embedded/bin/gitaly-backup`. +If you have a specific reason to change the path, it can be configured in the Linux package (Omnibus): + +1. Add the following to `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['backup_gitaly_backup_path'] = '/path/to/gitaly-backup' + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) + for the changes to take effect. + +## Alternative backup strategies + +Because every deployment may have different capabilities, you should first review [what data needs to be backed up](#what-data-needs-to-be-backed-up) to better understand if, and how, you can leverage them. + +For example, if you use Amazon RDS, you might choose to use its built-in backup and restore features to handle your GitLab [PostgreSQL data](#postgresql-databases), and [exclude PostgreSQL data](#excluding-specific-directories-from-the-backup) when using the [backup command](#backup-command). + +In the following cases, consider using file system data transfer or snapshots as part of your backup strategy: + +- Your GitLab instance contains a lot of Git repository data and the GitLab backup script is too slow. +- Your GitLab instance has a lot of forked projects and the regular backup task duplicates the Git data for all of them. +- Your GitLab instance has a problem and using the regular backup and import Rake tasks isn't possible. + +WARNING: +Gitaly Cluster [does not support snapshot backups](../gitaly/index.md#snapshot-backup-and-recovery-limitations). + +When considering using file system data transfer or snapshots: + +- Don't use these methods to migrate from one operating system to another. The operating systems of the source and destination should be as similar as possible. For example, + don't use these methods to migrate from Ubuntu to Fedora. +- Data consistency is very important. You should stop GitLab with `sudo gitlab-ctl stop` before taking doing a file system transfer (with `rsync`, for example) or taking a + snapshot. + +Example: Amazon Elastic Block Store (EBS) + +- A GitLab server using the Linux package (Omnibus) hosted on Amazon AWS. +- An EBS drive containing an ext4 file system is mounted at `/var/opt/gitlab`. +- In this case you could make an application backup by taking an EBS snapshot. +- The backup includes all repositories, uploads and PostgreSQL data. + +Example: Logical Volume Manager (LVM) snapshots + rsync + +- A GitLab server using the Linux package (Omnibus), with an LVM logical volume mounted at `/var/opt/gitlab`. +- Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. +- Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only file system at `/mnt/gitlab_backup`. +- Now we can have a longer running rsync job which creates a consistent replica on the remote server. +- The replica includes all repositories, uploads and PostgreSQL data. + +If you're running GitLab on a virtualized server, you can possibly also create +VM snapshots of the entire GitLab server. It's not uncommon however for a VM +snapshot to require you to power down the server, which limits this solution's +practical use. + +### Back up repository data separately + +First, ensure you back up existing GitLab data while [skipping repositories](#excluding-specific-directories-from-the-backup): + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-backup create SKIP=repositories +``` + +:::TabTitle Self-compiled + +```shell +sudo -u git -H bundle exec rake gitlab:backup:create SKIP=repositories RAILS_ENV=production +``` + +::EndTabs + +For manually backing up the Git repository data on disk, there are multiple possible strategies: + +- Use snapshots, such as the previous examples of Amazon EBS drive snapshots, or LVM snapshots + rsync. +- Use [GitLab Geo](../geo/index.md) and rely on the repository data on a Geo secondary site. +- [Prevent writes and copy the Git repository data](#prevent-writes-and-copy-the-git-repository-data). +- [Create an online backup by marking repositories as read-only (experimental)](#online-backup-through-marking-repositories-as-read-only-experimental). + +#### Prevent writes and copy the Git repository data + +Git repositories must be copied in a consistent way. They should not be copied during concurrent write +operations, as this can lead to inconsistencies or corruption issues. For more details, +[issue #270422](https://gitlab.com/gitlab-org/gitlab/-/issues/270422 "Provide documentation on preferred method of migrating Gitaly servers") +has a longer discussion explaining the potential problems. + +To prevent writes to the Git repository data, there are two possible approaches: + +- Use [maintenance mode](../maintenance_mode/index.md) to place GitLab in a read-only state. +- Create explicit downtime by stopping all Gitaly services before backing up the repositories: + + ```shell + sudo gitlab-ctl stop gitaly + # execute git data copy step + sudo gitlab-ctl start gitaly + ``` + +You can copy Git repository data using any method, as long as writes are prevented on the data being copied +(to prevent inconsistencies and corruption issues). In order of preference and safety, the recommended methods are: + +1. Use `rsync` with archive-mode, delete, and checksum options, for example: + + ```shell + rsync -aR --delete --checksum source destination # be extra safe with the order as it will delete existing data if inverted + ``` + +1. Use a [`tar` pipe to copy the entire repository's directory to another server or location](../operations/moving_repositories.md#tar-pipe-to-another-server). + +1. Use `sftp`, `scp`, `cp`, or any other copying method. + +#### Online backup through marking repositories as read-only (experimental) + +One way of backing up repositories without requiring instance-wide downtime +is to programmatically mark projects as read-only while copying the underlying data. + +There are a few possible downsides to this: + +- Repositories are read-only for a period of time that scales with the size of the repository. +- Backups take a longer time to complete due to marking each project as read-only, potentially leading to inconsistencies. For example, + a possible date discrepancy between the last data available for the first project that gets backed up compared to + the last project that gets backed up. +- Fork networks should be entirely read-only while the projects inside get backed up to prevent potential changes to the pool repository. + +There is an **experimental** script that attempts to automate this process in +[the Geo team Runbooks project](https://gitlab.com/gitlab-org/geo-team/runbooks/-/tree/main/experimental-online-backup-through-rsync). + +## Troubleshooting + +The following are possible problems you might encounter, along with potential +solutions. + +### When the secrets file is lost + +If you didn't [back up the secrets file](#storing-configuration-files), you +must complete several steps to get GitLab working properly again. + +The secrets file is responsible for storing the encryption key for the columns +that contain required, sensitive information. If the key is lost, GitLab can't +decrypt those columns, preventing access to the following items: + +- [CI/CD variables](../../ci/variables/index.md) +- [Kubernetes / GCP integration](../../user/infrastructure/clusters/index.md) +- [Custom Pages domains](../../user/project/pages/custom_domains_ssl_tls_certification/index.md) +- [Project error tracking](../../operations/error_tracking.md) +- [Runner authentication](../../ci/runners/index.md) +- [Project mirroring](../../user/project/repository/mirror/index.md) +- [Integrations](../../user/project/integrations/index.md) +- [Web hooks](../../user/project/integrations/webhooks.md) + +In cases like CI/CD variables and runner authentication, you can experience +unexpected behaviors, such as: + +- Stuck jobs. +- 500 errors. + +In this case, you must reset all the tokens for CI/CD variables and +runner authentication, which is described in more detail in the following +sections. After resetting the tokens, you should be able to visit your project +and the jobs begin running again. + +WARNING: +The steps in this section can potentially lead to **data loss** on the above listed items. +Consider opening a [Support Request](https://support.gitlab.com/hc/en-us/requests/new) if you're a Premium or Ultimate customer. + +#### Verify that all values can be decrypted + +You can determine if your database contains values that can't be decrypted by using a +[Rake task](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). + +#### Take a backup + +You must directly modify GitLab data to work around your lost secrets file. + +WARNING: +Be sure to create a full database backup before attempting any changes. + +#### Disable user two-factor authentication (2FA) + +Users with 2FA enabled can't sign in to GitLab. In that case, you must +[disable 2FA for everyone](../../security/two_factor_authentication.md#for-all-users), +after which users must reactivate 2FA. + +#### Reset CI/CD variables + +1. Enter the database console: + + For the Linux package (Omnibus) GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + + For the Linux package (Omnibus) GitLab 14.2 and later: + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + For installations from source, GitLab 14.1 and earlier: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production + ``` + + For installations from source, GitLab 14.2 and later: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production --database main + ``` + +1. Examine the `ci_group_variables` and `ci_variables` tables: + + ```sql + SELECT * FROM public."ci_group_variables"; + SELECT * FROM public."ci_variables"; + ``` + + These are the variables that you need to delete. + +1. Delete all variables: + + ```sql + DELETE FROM ci_group_variables; + DELETE FROM ci_variables; + ``` + +1. If you know the specific group or project from which you wish to delete variables, you can include a `WHERE` statement to specify that in your `DELETE`: + + ```sql + DELETE FROM ci_group_variables WHERE group_id = <GROUPID>; + DELETE FROM ci_variables WHERE project_id = <PROJECTID>; + ``` + +You may need to reconfigure or restart GitLab for the changes to take effect. + +#### Reset runner registration tokens + +1. Enter the database console: + + For the Linux package (Omnibus) GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + + For the Linux package (Omnibus) GitLab 14.2 and later: + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + For installations from source, GitLab 14.1 and earlier: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production + ``` + + For installations from source, GitLab 14.2 and later: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production --database main + ``` + +1. Clear all tokens for projects, groups, and the entire instance: + + WARNING: + The final `UPDATE` operation stops the runners from being able to pick + up new jobs. You must register new runners. + + ```sql + -- Clear project tokens + UPDATE projects SET runners_token = null, runners_token_encrypted = null; + -- Clear group tokens + UPDATE namespaces SET runners_token = null, runners_token_encrypted = null; + -- Clear instance tokens + UPDATE application_settings SET runners_registration_token_encrypted = null; + -- Clear key used for JWT authentication + -- This may break the $CI_JWT_TOKEN job variable: + -- https://gitlab.com/gitlab-org/gitlab/-/issues/325965 + UPDATE application_settings SET encrypted_ci_jwt_signing_key = null; + -- Clear runner tokens + UPDATE ci_runners SET token = null, token_encrypted = null; + ``` + +#### Reset pending pipeline jobs + +1. Enter the database console: + + For the Linux package (Omnibus) GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + + For the Linux package (Omnibus) GitLab 14.2 and later: + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + For installations from source, GitLab 14.1 and earlier: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production + ``` + + For installations from source, GitLab 14.2 and later: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production --database main + ``` + +1. Clear all the tokens for pending jobs: + + For GitLab 15.3 and earlier: + + ```sql + -- Clear build tokens + UPDATE ci_builds SET token = null, token_encrypted = null; + ``` + + For GitLab 15.4 and later: + + ```sql + -- Clear build tokens + UPDATE ci_builds SET token_encrypted = null; + ``` + +A similar strategy can be employed for the remaining features. By removing the +data that can't be decrypted, GitLab can be returned to operation, and the +lost data can be manually replaced. + +#### Fix integrations and webhooks + +If you've lost your secrets, the [integrations settings](../../user/project/integrations/index.md) +and [webhooks settings](../../user/project/integrations/webhooks.md) pages might display `500` error messages. Lost secrets might also produce `500` errors when you try to access a repository in a project with a previously configured integration or webhook. + +The fix is to truncate the affected tables (those containing encrypted columns). +This deletes all your configured integrations, webhooks, and related metadata. +You should verify that the secrets are the root cause before deleting any data. + +1. Enter the database console: + + For the Linux package (Omnibus) GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + + For the Linux package (Omnibus) GitLab 14.2 and later: + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + For installations from source, GitLab 14.1 and earlier: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production + ``` + + For installations from source, GitLab 14.2 and later: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production --database main + ``` + +1. Truncate the following tables: + + ```sql + -- truncate web_hooks table + TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs CASCADE; + ``` + +### Container Registry push failures after restoring from a backup + +If you use the [Container Registry](../../user/packages/container_registry/index.md), +pushes to the registry may fail after restoring your backup on a Linux package (Omnibus) +instance after restoring the registry data. + +These failures mention permission issues in the registry logs, similar to: + +```plaintext +level=error +msg="response completed with error" +err.code=unknown +err.detail="filesystem: mkdir /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/...: permission denied" +err.message="unknown error" +``` + +This issue is caused by the restore running as the unprivileged user `git`, +which is unable to assign the correct ownership to the registry files during +the restore process ([issue #62759](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62759 "Incorrect permissions on registry filesystem after restore")). + +To get your registry working again: + +```shell +sudo chown -R registry:registry /var/opt/gitlab/gitlab-rails/shared/registry/docker +``` + +If you changed the default file system location for the registry, run `chown` +against your custom location, instead of `/var/opt/gitlab/gitlab-rails/shared/registry/docker`. + +### Backup fails to complete with Gzip error + +When running the backup, you may receive a Gzip error message: + +```shell +sudo /opt/gitlab/bin/gitlab-backup create +... +Dumping ... +... +gzip: stdout: Input/output error + +Backup failed +``` + +If this happens, examine the following: + +- Confirm there is sufficient disk space for the Gzip operation. It's not uncommon for backups that + use the [default strategy](#backup-strategy-option) to require half the instance size + in free disk space during backup creation. +- If NFS is being used, check if the mount option `timeout` is set. The + default is `600`, and changing this to smaller values results in this error. + +### Backup fails with `File name too long` error + +During backup, you can get the `File name too long` error ([issue #354984](https://gitlab.com/gitlab-org/gitlab/-/issues/354984)). For example: + +```plaintext +Problem: <class 'OSError: [Errno 36] File name too long: +``` + +This problem stops the backup script from completing. To fix this problem, you must truncate the filenames causing the problem. A maximum of 246 characters, including the file extension, is permitted. + +WARNING: +The steps in this section can potentially lead to **data loss**. All steps must be followed strictly in the order given. +Consider opening a [Support Request](https://support.gitlab.com/hc/en-us/requests/new) if you're a Premium or Ultimate customer. + +Truncating filenames to resolve the error involves: + +- Cleaning up remote uploaded files that aren't tracked in the database. +- Truncating the filenames in the database. +- Rerunning the backup task. + +#### Clean up remote uploaded files + +A [known issue](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45425) caused object store uploads to remain after a parent resource was deleted. This issue was [resolved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18698). + +To fix these files, you must clean up all remote uploaded files that are in the storage but not tracked in the `uploads` database table. + +1. List all the object store upload files that can be moved to a lost and found directory if they don't exist in the GitLab database: + + ```shell + bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production + ``` + +1. If you are sure you want to delete these files and remove all non-referenced uploaded files, run: + + WARNING: + The following action is **irreversible**. + + ```shell + bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production DRY_RUN=false + ``` + +#### Truncate the filenames referenced by the database + +You must truncate the files referenced by the database that are causing the problem. The filenames referenced by the database are stored: + +- In the `uploads` table. +- In the references found. Any reference found from other database tables and columns. +- On the file system. + +Truncate the filenames in the `uploads` table: + +1. Enter the database console: + + For the Linux package (Omnibus) GitLab 14.2 and later: + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + For the Linux package (Omnibus) GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + + For installations from source, GitLab 14.2 and later: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production --database main + ``` + + For installations from source, GitLab 14.1 and earlier: + + ```shell + sudo -u git -H bundle exec rails dbconsole -e production + ``` + +1. Search the `uploads` table for filenames longer than 246 characters: + + The following query selects the `uploads` records with filenames longer than 246 characters in batches of 0 to 10000. This improves the performance on large GitLab instances with tables having thousand of records. + + ```sql + CREATE TEMP TABLE uploads_with_long_filenames AS + SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, id, path + FROM uploads AS u + WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; + + CREATE INDEX ON uploads_with_long_filenames(row_id); + + SELECT + u.id, + u.path, + -- Current filename + (regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] AS current_filename, + -- New filename + CONCAT( + LEFT(SPLIT_PART((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), + COALESCE(SUBSTRING((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) + ) AS new_filename, + -- New path + CONCAT( + COALESCE((regexp_match(u.path, '(.*\/).*'))[1], ''), + CONCAT( + LEFT(SPLIT_PART((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), + COALESCE(SUBSTRING((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) + ) + ) AS new_path + FROM uploads_with_long_filenames AS u + WHERE u.row_id > 0 AND u.row_id <= 10000; + ``` + + Output example: + + ```postgresql + -[ RECORD 1 ]----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + id | 34 + path | public/@hashed/loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisit.txt + current_filename | loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisit.txt + new_filename | loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelits.txt + new_path | public/@hashed/loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelits.txt + ``` + + Where: + + - `current_filename`: a filename that is currently more than 246 characters long. + - `new_filename`: a filename that has been truncated to 246 characters maximum. + - `new_path`: new path considering the `new_filename` (truncated). + + After you validate the batch results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. + +1. Rename the files found in the `uploads` table from long filenames to new truncated filenames. The following query rolls back the update so you can check the results safely in a transaction wrapper: + + ```sql + CREATE TEMP TABLE uploads_with_long_filenames AS + SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, path, id + FROM uploads AS u + WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; + + CREATE INDEX ON uploads_with_long_filenames(row_id); + + BEGIN; + WITH updated_uploads AS ( + UPDATE uploads + SET + path = + CONCAT( + COALESCE((regexp_match(updatable_uploads.path, '(.*\/).*'))[1], ''), + CONCAT( + LEFT(SPLIT_PART((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), + COALESCE(SUBSTRING((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) + ) + ) + FROM + uploads_with_long_filenames AS updatable_uploads + WHERE + uploads.id = updatable_uploads.id + AND updatable_uploads.row_id > 0 AND updatable_uploads.row_id <= 10000 + RETURNING uploads.* + ) + SELECT id, path FROM updated_uploads; + ROLLBACK; + ``` + + After you validate the batch update results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. + +1. Validate that the new filenames from the previous query are the expected ones. If you are sure you want to truncate the records found in the previous step to 246 characters, run the following: + + WARNING: + The following action is **irreversible**. + + ```sql + CREATE TEMP TABLE uploads_with_long_filenames AS + SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, path, id + FROM uploads AS u + WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; + + CREATE INDEX ON uploads_with_long_filenames(row_id); + + UPDATE uploads + SET + path = + CONCAT( + COALESCE((regexp_match(updatable_uploads.path, '(.*\/).*'))[1], ''), + CONCAT( + LEFT(SPLIT_PART((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), + COALESCE(SUBSTRING((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) + ) + ) + FROM + uploads_with_long_filenames AS updatable_uploads + WHERE + uploads.id = updatable_uploads.id + AND updatable_uploads.row_id > 0 AND updatable_uploads.row_id <= 10000; + ``` + + After you finish the batch update, you must change the batch size (`updatable_uploads.row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. + +Truncate the filenames in the references found: + +1. Check if those records are referenced somewhere. One way to do this is to dump the database and search for the parent directory name and filename: + + 1. To dump your database, you can use the following command as an example: + + ```shell + pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.tmp + ``` + + 1. Then you can search for the references using the `grep` command. Combining the parent directory and the filename can be a good idea. For example: + + ```shell + grep public/alongfilenamehere.txt gitlab-dump.tmp + ``` + +1. Replace those long filenames using the new filenames obtained from querying the `uploads` table. + +Truncate the filenames on the file system. You must manually rename the files in your file system to the new filenames obtained from querying the `uploads` table. + +#### Re-run the backup task + +After following all the previous steps, re-run the backup task. + +### Restoring database backup fails when `pg_stat_statements` was previously enabled + +The GitLab backup of the PostgreSQL database includes all SQL statements required to enable extensions that were +previously enabled in the database. + +The `pg_stat_statements` extension can only be enabled or disabled by a PostgreSQL user with `superuser` role. +As the restore process uses a database user with limited permissions, it can't execute the following SQL statements: + +```sql +DROP EXTENSION IF EXISTS pg_stat_statements; +CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; +``` + +When trying to restore the backup in a PostgreSQL instance that doesn't have the `pg_stats_statements` extension, +the following error message is displayed: + +```plaintext +ERROR: permission denied to create extension "pg_stat_statements" +HINT: Must be superuser to create this extension. +ERROR: extension "pg_stat_statements" does not exist +``` + +When trying to restore in an instance that has the `pg_stats_statements` extension enabled, the cleaning up step +fails with an error message similar to the following: + +```plaintext +rake aborted! +ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/bin/bundle:23:in `load' +/opt/gitlab/embedded/bin/bundle:23:in `<main>' +Caused by: +PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/bin/bundle:23:in `load' +/opt/gitlab/embedded/bin/bundle:23:in `<main>' +Tasks: TOP => gitlab:db:drop_tables +(See full trace by running task with --trace) +``` + +#### Prevent the dump file to include `pg_stat_statements` + +To prevent the inclusion of the extension in the PostgreSQL dump file that is part of the backup bundle, +enable the extension in any schema except the `public` schema: + +```sql +CREATE SCHEMA adm; +CREATE EXTENSION pg_stat_statements SCHEMA adm; +``` + +If the extension was previously enabled in the `public` schema, move it to a new one: + +```sql +CREATE SCHEMA adm; +ALTER EXTENSION pg_stat_statements SET SCHEMA adm; +``` + +To query the `pg_stat_statements` data after changing the schema, prefix the view name with the new schema: + +```sql +SELECT * FROM adm.pg_stat_statements limit 0; +``` + +To make it compatible with third-party monitoring solutions that expect it to be enabled in the `public` schema, +you need to include it in the `search_path`: + +```sql +set search_path to public,adm; +``` + +#### Fix an existing dump file to remove references to `pg_stat_statements` + +To fix an existing backup file, do the following changes: + +1. Extract from the backup the following file: `db/database.sql.gz`. +1. Decompress the file or use an editor that is capable of handling it compressed. +1. Remove the following lines, or similar ones: + + ```sql + CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; + ``` + + ```sql + COMMENT ON EXTENSION pg_stat_statements IS 'track planning and execution statistics of all SQL statements executed'; + ``` + +1. Save the changes and recompress the file. +1. Update the backup file with the modified `db/database.sql.gz`. diff --git a/doc/administration/backup_restore/index.md b/doc/administration/backup_restore/index.md new file mode 100644 index 00000000000..72a0176adc1 --- /dev/null +++ b/doc/administration/backup_restore/index.md @@ -0,0 +1,232 @@ +--- +stage: Systems +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Back up and restore GitLab **(FREE SELF)** + +Your software or organization depends on the data in your GitLab instance. You need to ensure this data is protected from adverse events such as: + +- Corrupted data +- Accidental deletion of data +- Ransomware attacks +- Unexpected cloud provider downtime + +You can mitigate all of these risks with a disaster recovery plan that includes backups. + +## Back up GitLab + +For detailed information on backing up GitLab, see [Backup GitLab](backup_gitlab.md). + +## Restore GitLab + +For detailed information on restoring GitLab, see [Restore GitLab](restore_gitlab.md). + +## Migrate to a new server + +<!-- some details borrowed from GitLab.com move from Azure to GCP detailed at https://gitlab.com/gitlab-com/migration/-/blob/master/.gitlab/issue_templates/failover.md --> + +You can use GitLab backup and restore to migrate your instance to a new server. This section outlines a typical procedure for a GitLab deployment running on a single server. +If you're running GitLab Geo, an alternative option is [Geo disaster recovery for planned failover](../geo/disaster_recovery/planned_failover.md). + +WARNING: +Avoid uncoordinated data processing by both the new and old servers, where multiple +servers could connect concurrently and process the same data. For example, when using +[incoming email](../incoming_email.md), if both GitLab instances are +processing email at the same time, then both instances miss some data. +This type of problem can occur with other services as well, such as a +[non-packaged database](https://docs.gitlab.com/omnibus/settings/database.html#using-a-non-packaged-postgresql-database-management-server), +a non-packaged Redis instance, or non-packaged Sidekiq. + +Prerequisites: + +- Some time before your migration, consider notifying your users of upcoming + scheduled maintenance with a [broadcast message banner](../broadcast_messages.md). +- Ensure your backups are complete and current. Create a complete system-level backup, or + take a snapshot of all servers involved in the migration, in case destructive commands + (like `rm`) are run incorrectly. + +### Prepare the new server + +To prepare the new server: + +1. Copy the + [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079) + from the old server to avoid man-in-the-middle attack warnings. + See [Manually replicate the primary site's SSH host keys](../geo/replication/configuration.md#step-2-manually-replicate-the-primary-sites-ssh-host-keys) for example steps. +1. [Install and configure GitLab](https://about.gitlab.com/install/) except + [incoming email](../incoming_email.md): + 1. Install GitLab. + 1. Configure by copying `/etc/gitlab` files from the old server to the new server, and update as necessary. + Read the + [Omnibus configuration backup and restore instructions](https://docs.gitlab.com/omnibus/settings/backups.html) for more detail. + 1. If applicable, disable [incoming email](../incoming_email.md). + 1. Block new CI/CD jobs from starting upon initial startup after the backup and restore. + Edit `/etc/gitlab/gitlab.rb` and set the following: + + ```ruby + nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" + ``` + + 1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Stop GitLab to avoid any potential unnecessary and unintentional data processing: + + ```shell + sudo gitlab-ctl stop + ``` + +1. Configure the new server to allow receiving the Redis database and GitLab backup files: + + ```shell + sudo rm -f /var/opt/gitlab/redis/dump.rdb + sudo chown <your-linux-username> /var/opt/gitlab/redis /var/opt/gitlab/backups + ``` + +### Prepare and transfer content from the old server + +1. Ensure you have an up-to-date system-level backup or snapshot of the old server. +1. Enable [maintenance mode](../maintenance_mode/index.md), + if supported by your GitLab edition. +1. Block new CI/CD jobs from starting: + 1. Edit `/etc/gitlab/gitlab.rb`, and set the following: + + ```ruby + nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" + ``` + + 1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Disable periodic background jobs: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Monitoring > Background Jobs**. + 1. Under the Sidekiq dashboard, select **Cron** tab and then + **Disable All**. +1. Wait for the currently running CI/CD jobs to finish, or accept that jobs that have not completed may be lost. + To view jobs currently running, on the left sidebar, select **Overviews > Jobs**, + and then select **Running**. +1. Wait for Sidekiq jobs to finish: + 1. On the left sidebar, select **Monitoring > Background Jobs**. + 1. Under the Sidekiq dashboard, select **Queues** and then **Live Poll**. + Wait for **Busy** and **Enqueued** to drop to 0. + These queues contain work that has been submitted by your users; + shutting down before these jobs complete may cause the work to be lost. + Make note of the numbers shown in the Sidekiq dashboard for post-migration verification. +1. Flush the Redis database to disk, and stop GitLab other than the services needed for migration: + + ```shell + sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket save && sudo gitlab-ctl stop && sudo gitlab-ctl start postgresql && sudo gitlab-ctl start gitaly + ``` + +1. Create a GitLab backup: + + ```shell + sudo gitlab-backup create + ``` + +1. Disable the following GitLab services and prevent unintentional restarts by adding the following to the bottom of `/etc/gitlab/gitlab.rb`: + + ```ruby + alertmanager['enable'] = false + gitlab_exporter['enable'] = false + gitlab_pages['enable'] = false + gitlab_workhorse['enable'] = false + grafana['enable'] = false + logrotate['enable'] = false + gitlab_rails['incoming_email_enabled'] = false + nginx['enable'] = false + node_exporter['enable'] = false + postgres_exporter['enable'] = false + postgresql['enable'] = false + prometheus['enable'] = false + puma['enable'] = false + redis['enable'] = false + redis_exporter['enable'] = false + registry['enable'] = false + sidekiq['enable'] = false + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Verify everything is stopped, and confirm no services are running: + + ```shell + sudo gitlab-ctl status + ``` + +1. Transfer the Redis database and GitLab backups to the new server: + + ```shell + sudo scp /var/opt/gitlab/redis/dump.rdb <your-linux-username>@new-server:/var/opt/gitlab/redis + sudo scp /var/opt/gitlab/backups/your-backup.tar <your-linux-username>@new-server:/var/opt/gitlab/backups + ``` + +### Restore data on the new server + +1. Restore appropriate file system permissions: + + ```shell + sudo chown gitlab-redis /var/opt/gitlab/redis + sudo chown gitlab-redis:gitlab-redis /var/opt/gitlab/redis/dump.rdb + sudo chown git:root /var/opt/gitlab/backups + sudo chown git:git /var/opt/gitlab/backups/your-backup.tar + ``` + +1. [Restore the GitLab backup](#restore-gitlab). +1. Verify that the Redis database restored correctly: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Monitoring > Background Jobs**. + 1. Under the Sidekiq dashboard, verify that the numbers + match with what was shown on the old server. + 1. While still under the Sidekiq dashboard, select **Cron** and then **Enable All** + to re-enable periodic background jobs. +1. Test that read-only operations on the GitLab instance work as expected. For example, browse through project repository files, merge requests, and issues. +1. Disable [Maintenance Mode](../maintenance_mode/index.md), if previously enabled. +1. Test that the GitLab instance is working as expected. +1. If applicable, re-enable [incoming email](../incoming_email.md) and test it is working as expected. +1. Update your DNS or load balancer to point at the new server. +1. Unblock new CI/CD jobs from starting by removing the custom NGINX configuration + you added previously: + + ```ruby + # The following line must be removed + nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Remove the scheduled maintenance [broadcast message banner](../broadcast_messages.md). + +## Additional notes + +This documentation is for GitLab Community and Enterprise Edition. We back up +GitLab.com and ensure your data is secure. You can't, however, use these +methods to export or back up your data yourself from GitLab.com. + +Issues are stored in the database, and can't be stored in Git itself. + +## Related features + +- [Geo](../geo/index.md) +- [Disaster Recovery (Geo)](../geo/disaster_recovery/index.md) +- [Migrating GitLab groups](../../user/group/import/index.md) +- [Import and migrate projects](../../user/project/import/index.md) diff --git a/doc/administration/backup_restore/restore_gitlab.md b/doc/administration/backup_restore/restore_gitlab.md new file mode 100644 index 00000000000..2cc0c68c66b --- /dev/null +++ b/doc/administration/backup_restore/restore_gitlab.md @@ -0,0 +1,432 @@ +--- +stage: Systems +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Restore GitLab **(FREE SELF)** + +GitLab provides a command line interface to restore your entire installation, +and is flexible enough to fit your needs. + +The [restore prerequisites section](#restore-prerequisites) includes crucial +information. Be sure to read and test the complete restore process at least +once before attempting to perform it in a production environment. + +NOTE: +You can only restore a backup to **exactly the same version and type (CE/EE)** +of GitLab on which it was created (for example CE 15.1.4). + +If your backup is a different version than the current installation, you must +[downgrade](../../update/package/downgrade.md) or [upgrade](../../update/package/index.md#upgrade-to-a-specific-version-using-the-official-repositories) your GitLab installation +before restoring the backup. + +Each backup archive contains a full self-contained backup, including those created through the [incremental repository backup procedure](backup_gitlab.md#incremental-repository-backups). To restore an incremental repository backup, use the same instructions as restoring any other regular backup archive. + +## Restore prerequisites + +You need to have a working GitLab installation before you can perform a +restore. This is because the system user performing the restore actions (`git`) +is usually not allowed to create or delete the SQL database needed to import +data into (`gitlabhq_production`). All existing data is either erased +(SQL) or moved to a separate directory (such as repositories and uploads). +Restoring SQL data skips views owned by PostgreSQL extensions. + +To restore a backup, **you must also restore the GitLab secrets**. +These include the database encryption key, [CI/CD variables](../../ci/variables/index.md), and +variables used for [two-factor authentication](../../user/profile/account/two_factor_authentication.md). +Without the keys, [multiple issues occur](backup_gitlab.md#when-the-secrets-file-is-lost), +including loss of access by users with [two-factor authentication enabled](../../user/profile/account/two_factor_authentication.md), +and GitLab Runners cannot log in. + +Restore: + +- `/etc/gitlab/gitlab-secrets.json` (Linux package) +- `/home/git/gitlab/.secret` (self-compiled installations) +- Rails secret (cloud-native GitLab) + - [This can be converted to the Linux package format](https://docs.gitlab.com/charts/installation/migration/helm_to_package.html), if required. + +You may also want to restore your previous `/etc/gitlab/gitlab.rb` (for Omnibus packages) +or `/home/git/gitlab/config/gitlab.yml` (for installations from source) and +any TLS keys, certificates (`/etc/gitlab/ssl`, `/etc/gitlab/trusted-certs`), or +[SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). + +Depending on your case, you might want to run the restore command with one or +more of the following options: + +- `BACKUP=timestamp_of_backup`: Required if more than one backup exists. + Read what the [backup timestamp is about](backup_gitlab.md#backup-timestamp). +- `force=yes`: Doesn't ask if the `authorized_keys` file should get regenerated, + and assumes 'yes' for warning about database tables being removed, + enabling the `Write to authorized_keys file` setting, and updating LDAP + providers. + +If you're restoring into directories that are mount points, you must ensure these directories are +empty before attempting a restore. Otherwise, GitLab attempts to move these directories before +restoring the new data, which causes an error. + +Read more about [configuring NFS mounts](../nfs.md) + +Restoring a backup from an instance using local storage restores to local storage even if the target instance uses object storage. +Migrations to object storage must be done before or after restoration. + +## Restore for Omnibus GitLab installations + +This procedure assumes that: + +- You have installed the **exact same version and type (CE/EE)** of GitLab + Omnibus with which the backup was created. +- You have run `sudo gitlab-ctl reconfigure` at least once. +- GitLab is running. If not, start it using `sudo gitlab-ctl start`. + +First ensure your backup tar file is in the backup directory described in the +`gitlab.rb` configuration `gitlab_rails['backup_path']`. The default is +`/var/opt/gitlab/backups`. The backup file needs to be owned by the `git` user. + +```shell +sudo cp 11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar /var/opt/gitlab/backups/ +sudo chown git:git /var/opt/gitlab/backups/11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar +``` + +Stop the processes that are connected to the database. Leave the rest of GitLab +running: + +```shell +sudo gitlab-ctl stop puma +sudo gitlab-ctl stop sidekiq +# Verify +sudo gitlab-ctl status +``` + +Next, ensure you have completed the [restore prerequisites](#restore-prerequisites) steps and have run `gitlab-ctl reconfigure` +after copying over the GitLab secrets file from the original installation. + +Next, restore the backup, specifying the timestamp of the backup you wish to +restore: + +```shell +# This command will overwrite the contents of your GitLab database! +# NOTE: "_gitlab_backup.tar" is omitted from the name +sudo gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce +``` + +If there's a GitLab version mismatch between your backup tar file and the +installed version of GitLab, the restore command aborts with an error +message. Install the [correct GitLab version](https://packages.gitlab.com/gitlab/), +and then try again. + +WARNING: +The restore command requires [additional parameters](backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer) when +your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. + +Next, restart and [check](../raketasks/maintenance.md#check-gitlab-configuration) GitLab: + +```shell +sudo gitlab-ctl restart +sudo gitlab-rake gitlab:check SANITIZE=true +``` + +In GitLab 13.1 and later, check [database values can be decrypted](../raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets) +especially if `/etc/gitlab/gitlab-secrets.json` was restored, or if a different server is +the target for the restore. + +```shell +sudo gitlab-rake gitlab:doctor:secrets +``` + +For added assurance, you can perform [an integrity check on the uploaded files](../raketasks/check.md#uploaded-files-integrity): + +```shell +sudo gitlab-rake gitlab:artifacts:check +sudo gitlab-rake gitlab:lfs:check +sudo gitlab-rake gitlab:uploads:check +``` + +## Restore for Docker image and GitLab Helm chart installations + +For GitLab installations using the Docker image or the GitLab Helm chart on a +Kubernetes cluster, the restore task expects the restore directories to be +empty. However, with Docker and Kubernetes volume mounts, some system level +directories may be created at the volume roots, such as the `lost+found` +directory found in Linux operating systems. These directories are usually owned +by `root`, which can cause access permission errors since the restore Rake task +runs as the `git` user. To restore a GitLab installation, users have to confirm +the restore target directories are empty. + +For both these installation types, the backup tarball has to be available in +the backup location (default location is `/var/opt/gitlab/backups`). + +### Restore for Helm chart installations + +The GitLab Helm chart uses the process documented in +[restoring a GitLab Helm chart installation](https://docs.gitlab.com/charts/backup-restore/restore.html#restoring-a-gitlab-installation) + +### Restore for Docker image installations + +If you're using [Docker Swarm](../../install/docker.md#install-gitlab-using-docker-swarm-mode), +the container might restart during the restore process because Puma is shut down, +and so the container health check fails. To work around this problem, +temporarily disable the health check mechanism. + +1. Edit `docker-compose.yml`: + + ```yaml + healthcheck: + disable: true + ``` + +1. Deploy the stack: + + ```shell + docker stack deploy --compose-file docker-compose.yml mystack + ``` + +For more information, see [issue 6846](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6846 "GitLab restore can fail owing to `gitlab-healthcheck`"). + +The restore task can be run from the host: + +```shell +# Stop the processes that are connected to the database +docker exec -it <name of container> gitlab-ctl stop puma +docker exec -it <name of container> gitlab-ctl stop sidekiq + +# Verify that the processes are all down before continuing +docker exec -it <name of container> gitlab-ctl status + +# Run the restore. NOTE: "_gitlab_backup.tar" is omitted from the name +docker exec -it <name of container> gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce + +# Restart the GitLab container +docker restart <name of container> + +# Check GitLab +docker exec -it <name of container> gitlab-rake gitlab:check SANITIZE=true +``` + +## Restore for installation from source + +First, ensure your backup tar file is in the backup directory described in the +`gitlab.yml` configuration: + +```yaml +## Backup settings +backup: + path: "tmp/backups" # Relative paths are relative to Rails.root (default: tmp/backups/) +``` + +The default is `/home/git/gitlab/tmp/backups`, and it needs to be owned by the `git` user. Now, you can begin the backup procedure: + +```shell +# Stop processes that are connected to the database +sudo service gitlab stop + +sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +Example output: + +```plaintext +Unpacking backup... [DONE] +Restoring database tables: +-- create_table("events", {:force=>true}) + -> 0.2231s +[...] +- Loading fixture events...[DONE] +- Loading fixture issues...[DONE] +- Loading fixture keys...[SKIPPING] +- Loading fixture merge_requests...[DONE] +- Loading fixture milestones...[DONE] +- Loading fixture namespaces...[DONE] +- Loading fixture notes...[DONE] +- Loading fixture projects...[DONE] +- Loading fixture protected_branches...[SKIPPING] +- Loading fixture schema_migrations...[DONE] +- Loading fixture services...[SKIPPING] +- Loading fixture snippets...[SKIPPING] +- Loading fixture taggings...[SKIPPING] +- Loading fixture tags...[SKIPPING] +- Loading fixture users...[DONE] +- Loading fixture users_projects...[DONE] +- Loading fixture web_hooks...[SKIPPING] +- Loading fixture wikis...[SKIPPING] +Restoring repositories: +- Restoring repository abcd... [DONE] +- Object pool 1 ... +Deleting tmp directories...[DONE] +``` + +Next, restore `/home/git/gitlab/.secret` if necessary, [as previously mentioned](#restore-prerequisites). + +Restart GitLab: + +```shell +sudo service gitlab restart +``` + +## Restoring only one or a few projects or groups from a backup + +Although the Rake task used to restore a GitLab instance doesn't support +restoring a single project or group, you can use a workaround by restoring +your backup to a separate, temporary GitLab instance, and then export your +project or group from there: + +1. [Install a new GitLab](../../install/index.md) instance at the same version as + the backed-up instance from which you want to restore. +1. [Restore the backup](#restore-gitlab) into this new instance, then + export your [project](../../user/project/settings/import_export.md) + or [group](../../user/group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). For more information about what is and isn't exported, see the export feature's documentation. +1. After the export is complete, go to the old instance and then import it. +1. After importing the projects or groups that you wanted is complete, you may + delete the new, temporary GitLab instance. + +A feature request to provide direct restore of individual projects or groups +is being discussed in [issue #17517](https://gitlab.com/gitlab-org/gitlab/-/issues/17517). + +## Restore options + +The command line tool GitLab provides to restore from backup can accept more +options. + +### Disabling prompts during restore + +During a restore from backup, the restore script may ask for confirmation before +proceeding. If you wish to disable these prompts, you can set the `GITLAB_ASSUME_YES` +environment variable to `1`. + +For Omnibus GitLab packages: + +```shell +sudo GITLAB_ASSUME_YES=1 gitlab-backup restore +``` + +For installations from source: + +```shell +sudo -u git -H GITLAB_ASSUME_YES=1 bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +### Excluding tasks on restore + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19347) in GitLab 14.10. + +You can exclude specific tasks on restore by adding the environment variable `SKIP`, whose values are a comma-separated list of the following options: + +- `db` (database) +- `uploads` (attachments) +- `builds` (CI job output logs) +- `artifacts` (CI job artifacts) +- `lfs` (LFS objects) +- `terraform_state` (Terraform states) +- `registry` (Container Registry images) +- `pages` (Pages content) +- `repositories` (Git repositories data) +- `packages` (Packages) + +For Omnibus GitLab packages: + +```shell +sudo gitlab-backup restore BACKUP=timestamp_of_backup SKIP=db,uploads +``` + +For installations from source: + +```shell +sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup SKIP=db,uploads RAILS_ENV=production +``` + +### Restore specific repository storages + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86896) in GitLab 15.0. + +When using [multiple repository storages](../repository_storage_paths.md), +repositories from specific repository storages can be restored separately +using the `REPOSITORIES_STORAGES` option. The option accepts a comma-separated list of +storage names. + +For example, for Omnibus GitLab installations: + +```shell +sudo gitlab-backup restore BACKUP=timestamp_of_backup REPOSITORIES_STORAGES=storage1,storage2 +``` + +For example, for installations from source: + +```shell +sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup REPOSITORIES_STORAGES=storage1,storage2 +``` + +### Restore specific repositories + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. + +You can restore specific repositories using the `REPOSITORIES_PATHS` and the `SKIP_REPOSITORIES_PATHS` options. +Both options accept a comma-separated list of project and group paths. If you +specify a group path, all repositories in all projects in the group and +descendent groups are included or skipped, depending on which option you used. The project and group repositories must exist within the specified backup. + +For example, to restore all repositories for all projects in **Group A** (`group-a`), the repository for **Project C** in **Group B** (`group-b/project-c`), +and skip the **Project D** in **Group A** (`group-a/project-d`): + +- Omnibus GitLab installations: + + ```shell + sudo gitlab-backup restore BACKUP=timestamp_of_backup REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d + ``` + +- Installations from source: + + ```shell + sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d + ``` + +### Restore untarred backups + +If an [untarred backup](backup_gitlab.md#skipping-tar-creation) (made with `SKIP=tar`) is found, +and no backup is chosen with `BACKUP=<timestamp>`, the untarred backup is used. + +For example, for Omnibus GitLab installations: + +```shell +sudo gitlab-backup restore +``` + +For example, for installations from source: + +```shell +sudo -u git -H bundle exec rake gitlab:backup:restore +``` + +## Troubleshooting + +The following are possible problems you might encounter, along with potential +solutions. + +### Restoring database backup using Omnibus packages outputs warnings + +If you're using backup restore procedures, you may encounter the following +warning messages: + +```plaintext +ERROR: must be owner of extension pg_trgm +ERROR: must be owner of extension btree_gist +ERROR: must be owner of extension plpgsql +WARNING: no privileges could be revoked for "public" (two occurrences) +WARNING: no privileges were granted for "public" (two occurrences) +``` + +Be advised that the backup is successfully restored in spite of these warning +messages. + +The Rake task runs this as the `gitlab` user, which doesn't have superuser +access to the database. When restore is initiated, it also runs as the `gitlab` +user, but it also tries to alter the objects it doesn't have access to. +Those objects have no influence on the database backup or restore, but display +a warning message. + +For more information, see: + +- PostgreSQL issue tracker: + - [Not being a superuser](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com). + - [Having different owners](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us). + +- Stack Overflow: [Resulting errors](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). diff --git a/doc/administration/broadcast_messages.md b/doc/administration/broadcast_messages.md new file mode 100644 index 00000000000..556edbd3e5e --- /dev/null +++ b/doc/administration/broadcast_messages.md @@ -0,0 +1,120 @@ +--- +stage: Growth +group: Acquisition +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference, howto +--- + +# Broadcast messages **(FREE SELF)** + +> - Target roles [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default. +> - Theme [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83251) and background color removed in GitLab 14.10. + +GitLab can display broadcast messages to users of a GitLab instance. There are two types of broadcast messages: + +- Banners +- Notifications + +Broadcast messages can be managed using the [broadcast messages API](../api/broadcast_messages.md). + +## Banners + +Banners are shown on the top of a page and optionally in the command line as a Git remote response. + + + +```shell +$ git push +... +remote: +remote: **Welcome** to GitLab :wave: +remote: +... +``` + +If more than one banner is active at one time, they are displayed at the top of the page in order of creation. In the command line, only the latest banner is shown. + +## Notifications + +Notifications are shown on the bottom right of a page and can contain placeholders. A placeholder is replaced with an attribute of the active user. Placeholders must be surrounded by curly braces, for example `{{name}}`. +The available placeholders are: + +- `{{email}}` +- `{{name}}` +- `{{user_id}}` +- `{{username}}` +- `{{instance_id}}` + +If the user is not signed in, user related values are empty. + + + +If more than one notification is active at one time, only the newest is shown. + +## Add a broadcast message + +To display messages to users on your GitLab instance, add a broadcast message. + +To add a broadcast message: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Messages**. +1. Add the text for the message to the **Message** field. You can style a message's content using Markdown, emoji, and the `a` and `br` HTML tags. + The `br` tag inserts a line break. The `a` HTML tag accepts `class` and `style` attributes with the following CSS properties: + - `color` + - `border` + - `background` + - `padding` + - `margin` + - `text-decoration` +1. Select a **Theme**. The default theme is `indigo`. +1. Select the **Dismissable** checkbox to enable users to dismiss the broadcast message. +1. Optional. Clear the **Git remote responses** checkbox to prevent broadcast messages from being displayed in the command line as Git remote responses. +1. Optional. Select **Target roles** to only show the broadcast message to users with the selected roles. The message displays on group, subgroup, and project pages, and does not display in Git remote responses. +1. If required, add a **Target Path** to only show the broadcast message on URLs matching that path. You can use the wildcard character `*` to match multiple URLs, for example `mygroup/myproject*`. +1. Select a date and time (UTC) for the message to start and end. +1. Select **Add broadcast message**. + +When a broadcast message expires, it no longer displays in the user interface but is still listed in the +list of broadcast messages. + +## Edit a broadcast message + +If you must make changes to a broadcast message, you can edit it. + +To edit a broadcast message: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Messages**. +1. From the list of broadcast messages, select the edit button for the message. +1. After making the required changes, select **Update broadcast message**. + +Expired messages can be made active again by changing their end date. + +## Delete a broadcast message + +If you no longer require a broadcast message, you can delete it. +You can delete a broadcast message while it's active. + +To delete a broadcast message: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Messages**. +1. From the list of broadcast messages, select the delete button for the message. + +When a broadcast message is deleted, it's removed from the list of broadcast messages. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/clusters/kas.md b/doc/administration/clusters/kas.md index b44c9571715..583a6401c05 100644 --- a/doc/administration/clusters/kas.md +++ b/doc/administration/clusters/kas.md @@ -142,10 +142,9 @@ For details, see [how to use the GitLab-KAS chart](https://docs.gitlab.com/chart ## Kubernetes API proxy cookie -> Introduced in GitLab 15.10 [with feature flags](../feature_flags.md) named `kas_user_access` and `kas_user_access_project`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flags](../feature_flags.md) named `kas_user_access` and `kas_user_access_project`. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104504) in GitLab 15.10 [with feature flags](../feature_flags.md) named `kas_user_access` and `kas_user_access_project`. Disabled by default. +> - Feature flags `kas_user_access` and `kas_user_access_project` [enabled](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123479) in GitLab 16.1. +> - Feature flags `kas_user_access` and `kas_user_access_project` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125835) in GitLab 16.2. KAS proxies Kubernetes API requests to the GitLab agent with either: diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 79133d5a6c7..978e43b2e2c 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -20,7 +20,7 @@ and secure supply chain best practices: | Feature | Instances | Groups | Projects | Description | |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Credentials inventory](../user/admin_area/credentials_inventory.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Keep track of the credentials used by all of the users in a GitLab instance. | +| [Credentials inventory](credentials_inventory.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Keep track of the credentials used by all of the users in a GitLab instance. | | [Granular user roles<br/>and flexible permissions](../user/permissions.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Manage access and permissions with five different user roles and settings for external users. Set permissions according to people's role, rather than either read or write access to a repository. Don't share the source code with people that only need access to the issue tracker. | | [Merge request approvals](../user/project/merge_requests/approvals/index.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Configure approvals required for merge requests. | | [Push rules](../user/project/repository/push_rules.md) | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Control pushes to your repositories. | @@ -65,10 +65,10 @@ These features can also help with compliance requirements: | Feature | Instances | Groups | Projects | Description | |:------------------------------------------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Email all users of a project,<br/>group, or entire server](../user/admin_area/email_from_gitlab.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Email groups of users based on project or group membership, or email everyone using the GitLab instance. These emails are great for scheduled maintenance or upgrades. | -| [Enforce ToS acceptance](../user/admin_area/settings/terms.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Enforce your users accepting new terms of service by blocking GitLab traffic. | +| [Email all users of a project,<br/>group, or entire server](email_from_gitlab.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Email groups of users based on project or group membership, or email everyone using the GitLab instance. These emails are great for scheduled maintenance or upgrades. | +| [Enforce ToS acceptance](settings/terms.md) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Enforce your users accepting new terms of service by blocking GitLab traffic. | | [External Status Checks](../user/project/merge_requests/status_checks.md) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Interface with third-party systems you already use during development to ensure you remain compliant. | -| [Generate reports on permission<br/>levels of users](../user/admin_area/index.md#user-permission-export) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Generate a report listing all users' access permissions for groups and projects in the instance. | +| [Generate reports on permission<br/>levels of users](../administration/admin_area.md#user-permission-export) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Generate a report listing all users' access permissions for groups and projects in the instance. | | [License compliance](../user/compliance/license_compliance/index.md) | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Search dependencies for their licenses. This lets you determine if the licenses of your project's dependencies are compatible with your project's license. | | [Lock project membership to group](../user/group/access_and_permissions.md#prevent-members-from-being-added-to-projects-in-a-group) | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Group owners can prevent new members from being added to projects in a group. | | [LDAP group sync](auth/ldap/ldap_synchronization.md#group-sync) | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Automatically synchronize groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools. | diff --git a/doc/administration/configure.md b/doc/administration/configure.md index d68e81ebf69..8ff2ae5aa9d 100644 --- a/doc/administration/configure.md +++ b/doc/administration/configure.md @@ -11,7 +11,7 @@ Customize and configure your self-managed GitLab installation. - [Authentication](auth/index.md) - [CI/CD](../administration/cicd.md) -- [Configuration](../user/admin_area/index.md) +- [Configuration](../administration/admin_area.md) - [Consul](../administration/consul.md) - [Environment variables](../administration/environment_variables.md) - [File hooks](../administration/file_hooks.md) @@ -35,7 +35,7 @@ Customize and configure your self-managed GitLab installation. - [Agent server for Kubernetes](../administration/clusters/kas.md) - [Server hooks](../administration/server_hooks.md) - [Terraform state](../administration/terraform_state.md) -- [Terraform limits](../user/admin_area/settings/terraform_limits.md) +- [Terraform limits](settings/terraform_limits.md) - [Packages](../administration/packages/index.md) - [Web terminals](../administration/integration/terminal.md) - [Wikis](../administration/wikis/index.md) diff --git a/doc/administration/credentials_inventory.md b/doc/administration/credentials_inventory.md new file mode 100644 index 00000000000..482d46498e3 --- /dev/null +++ b/doc/administration/credentials_inventory.md @@ -0,0 +1,86 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Credentials inventory **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20912) in GitLab 12.6. +> - [Bot-created access tokens not displayed in personal access token list](https://gitlab.com/gitlab-org/gitlab/-/issues/351759) in GitLab 14.9. + +GitLab administrators are responsible for the overall security of their instance. To assist, GitLab +provides a Credentials inventory to keep track of all the credentials that can be used to access +their self-managed instance. + +Use Credentials inventory to see for your GitLab instance all: + +- Personal access tokens (PAT). +- Project access tokens (GitLab 14.8 and later). +- SSH keys. +- GPG keys. + +You can also [revoke](#revoke-a-users-personal-access-token) and [delete](#delete-a-users-ssh-key) and see: + +- Who they belong to. +- Their access scope. +- Their usage pattern. +- When they expire. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. +- When they were revoked. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. + +To access the Credentials inventory: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Credentials**. + +## Revoke a user's personal access token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214811) in GitLab 13.4. + +If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: + +| Token state | Show Revoke button? | Comments | +|-------------|---------------------|----------------------------------------------------------------------------| +| Active | Yes | Allows administrators to revoke the PAT, such as for a compromised account | +| Expired | No | Not applicable; token is already expired | +| Revoked | No | Not applicable; token is already revoked | + +When a PAT is revoked from the credentials inventory, the instance notifies the user by email. + + + +## Revoke a user's project access token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243833) in GitLab 14.8. + +The **Revoke** button next to a project access token can be selected to revoke that particular project access token. This both: + +- Revokes the token project access token. +- Enqueues a background worker to delete the project bot user. + + + +## Delete a user's SSH key + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225248) in GitLab 13.5. + +You can **Delete** a user's SSH key by navigating to the credentials inventory's SSH Keys tab. +The instance then notifies the user. + + + +## Review existing GPG keys + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282429) in GitLab 13.10. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292961) in GitLab 13.12. + +You can view all existing GPG in your GitLab instance by navigating to the +credentials inventory GPG Keys tab, as well as the following properties: + +- Who the GPG key belongs to. +- The ID of the GPG key. +- Whether the GPG key is [verified or unverified](../user/project/repository/gpg_signed_commits/index.md) + + diff --git a/doc/administration/custom_project_templates.md b/doc/administration/custom_project_templates.md new file mode 100644 index 00000000000..2bbbb5649e6 --- /dev/null +++ b/doc/administration/custom_project_templates.md @@ -0,0 +1,73 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Custom instance-level project templates **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6860) in GitLab 11.2. + +GitLab administrators can set a group to be the source of project templates that are +selectable when a new project is created on the instance. These templates can be selected +when you go to **New project > Create from template** and select the **Instance** tab. + +Every project in the group, but not its subgroups, can be selected when a new project +is created, based on the user's access permissions: + +- Public projects can be selected by any authenticated user as a template for a new project, + if all enabled [project features](../user/project/settings/index.md#configure-project-visibility-features-and-permissions) + except for **GitLab Pages** and **Security and Compliance** are set to **Everyone With Access**. + The same applies to internal projects. +- Private projects can be selected only by users who are members of the projects. + +The **Metrics Dashboard** is set to **Only Project Members** when you create a new project. Make +sure you change it to **Everyone With Access** before making it a project template. + +Repository and database information that are copied over to each new project are +identical to the data exported with the [GitLab Project Import/Export](../user/project/settings/import_export.md). + +To set project templates at the group level, see [Custom group-level project templates](../user/group/custom_project_templates.md). + +## Select instance-level project template group + +To select the group to use as the source for the project templates: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Templates**. +1. Expand **Custom project templates**. +1. Select a group to use. +1. Select **Save changes**. + +Projects in subgroups of the template group are **not** included in the template list. + +## What is copied from the templates + +The entire custom instance-level project templates repository is copied, including: + +- Branches +- Commits +- Tags + +If the user: + +- Has the Owner role on the custom instance-level project templates project or is a GitLab administrator, all project settings are copied over to the new + project. +- Doesn't have the Owner role or is not a GitLab administrator, project [deploy keys](../user/project/deploy_keys/index.md#view-deploy-keys) and project + [webhooks](../user/project/integrations/webhooks.md) aren't copied over because they contain sensitive data. + +To learn more about what is migrated, see +[Items that are exported](../user/project/settings/import_export.md#items-that-are-exported). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/dedicated/index.md b/doc/administration/dedicated/index.md index f2caa5fa368..db30684ab1c 100644 --- a/doc/administration/dedicated/index.md +++ b/doc/administration/dedicated/index.md @@ -14,7 +14,11 @@ The instructions on this page guide you through: 1. Onboarding and initial setup of your GitLab Dedicated instance. 1. Configuring your GitLab Dedicated instance including enabling and updating the settings for [available functionality](../../subscriptions/gitlab_dedicated/index.md#available-features). -Any functionality in the GitLab application that is not controlled by the SaaS environment can be configured by using the [Admin Panel](../../user/admin_area/index.md). +Any functionality in the GitLab application that is not controlled by the SaaS environment can be configured by using the [Admin Panel](../../administration/admin_area.md). + +Examples of SaaS environment settings include `gitlab.rb` configurations and access to shell, Rails console, and PostgreSQL console. +These environment settings cannot be changed by tenants. +GitLab Dedicated Engineers also don't have direct access to tenant environments, except for [break glass situations](../../subscriptions/gitlab_dedicated/index.md#access-controls). ## Onboarding @@ -68,8 +72,9 @@ To create a KMS key using the AWS Console: 1. Key material origin: **KMS** 1. Regionality: **Multi-Region key** 1. Enter your values for key alias, description, and tags. -1. Select Key administrators (optionally allow or deny key administrators to delete the key). -1. For Key usage permissions, add the GitLab AWS account using the **Other AWS accounts** dialog. +1. Select key administrators. +1. Optional. Allow or prevent key administrators from deleting the key. +1. On the **Define key usage permissions** page, under **Other AWS accounts**, add the GitLab AWS account. The last page asks you to confirm the KMS key policy. It should look similar to the following example, populated with your account IDs and usernames: diff --git a/doc/administration/diff_limits.md b/doc/administration/diff_limits.md new file mode 100644 index 00000000000..48b52950ee3 --- /dev/null +++ b/doc/administration/diff_limits.md @@ -0,0 +1,53 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Diff limits administration **(FREE SELF)** + +You can set a maximum size for display of diff files (patches). + +For details about diff files, [view changes between files](../user/project/merge_requests/changes.md). +Read more about the [built-in limits for merge requests and diffs](../administration/instance_limits.md#merge-requests). + +## Configure diff limits + +WARNING: +These settings are experimental. An increased maximum increases resource +consumption of your instance. Keep this in mind when adjusting the maximum. + +To speed the loading time of merge request views and branch comparison views +on your instance, you can configure three instance-level maximum values for diffs: + +| Value | Definition | Default value | Maximum value | +| ----- | ---------- | :-----------: | :-----------: | +| **Maximum diff patch size** | The total size, in bytes, of the entire diff. | 200 KB | 500 KB | +| **Maximum diff files** | The total number of files changed in a diff. | 1000 | 3000 | +| **Maximum diff lines** | The total number of lines changed in a diff. | 50,000 | 100,000 | + +When a diff reaches 10% of any of these values, the files are shown in a +collapsed view, with a link to expand the diff. Diffs that exceed any of the +set values are presented as **Too large** are cannot be expanded in the UI. + +To configure these values: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Diff limits**. +1. Enter a value for the diff limit. +1. Select **Save changes**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/docs_self_host.md b/doc/administration/docs_self_host.md index 54f8c15a922..1eecc139d1f 100644 --- a/doc/administration/docs_self_host.md +++ b/doc/administration/docs_self_host.md @@ -163,7 +163,7 @@ To extract the HTML files of the documentation site: ## Redirect the `/help` links to the new Docs site After your local product documentation site is running, -[redirect the help links](../user/admin_area/settings/help_page.md#redirect-help-pages) +[redirect the help links](../administration/settings/help_page.md#redirect-help-pages) in the GitLab application to your local site, by using the fully qualified domain name as the documentation URL. For example, if you used the [Docker method](#self-host-the-product-documentation-with-docker), enter `http://0.0.0.0:4000`. @@ -173,9 +173,9 @@ documentation URL requests as needed. For example, if your GitLab version is 14.5: - The GitLab documentation URL becomes `http://0.0.0.0:4000/14.5/`. -- The link in GitLab displays as `<instance_url>/help/user/admin_area/settings/help_page#destination-requirements`. +- The link in GitLab displays as `<instance_url>/help/administration/settings/help_page#destination-requirements`. - When you select the link, you are redirected to -`http://0.0.0.0:4000/14.5/ee/user/admin_area/settings/help_page/#destination-requirements`. +`http://0.0.0.0:4000/14.5/ee/administration/settings/help_page/#destination-requirements`. To test the setting, in GitLab, select a **Learn more** link. For example: diff --git a/doc/administration/email_from_gitlab.md b/doc/administration/email_from_gitlab.md new file mode 100644 index 00000000000..9272f58d2b9 --- /dev/null +++ b/doc/administration/email_from_gitlab.md @@ -0,0 +1,61 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto, reference +--- + +# Email from GitLab **(PREMIUM SELF)** + +GitLab provides a tool to administrators for emailing all users, or users of +a chosen group or project, right from the Admin Area. Users receive the email +at their primary email address. + +For information about email notifications originating from GitLab, read +[GitLab notification emails](../user/profile/notifications.md). + +## Use-cases + +- Notify your users about a new project, a new feature, or a new product launch. +- Notify your users about a new deployment, or that downtime is expected + for a particular reason. + +## Sending emails to users from GitLab + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select **Send email to users**. + +  + +1. Compose an email and choose where to send it (all users or users of a + chosen group or project). The email body only supports plain text messages. + HTML, Markdown, and other rich text formats are not supported, and is + sent as plain text to users. + +  + +NOTE: +[Starting with GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/31509), email notifications can be sent only once every 10 minutes. This helps minimize performance issues. + +## Unsubscribing from emails + +Users can choose to unsubscribe from receiving emails from GitLab by following +the unsubscribe link in the email. Unsubscribing is unauthenticated in order +to keep this feature simple. + +On unsubscribe, users receive an email notification that unsubscribe happened. +The endpoint that provides the unsubscribe option is rate-limited. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/external_users.md b/doc/administration/external_users.md new file mode 100644 index 00000000000..f8ca379d10c --- /dev/null +++ b/doc/administration/external_users.md @@ -0,0 +1,80 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# External users **(FREE SELF)** + +In cases where it is desired that a user has access only to some internal or +private projects, there is the option of creating **External Users**. This +feature may be useful when for example a contractor is working on a given +project and should only have access to that project. + +External users: + +- Cannot create project, groups, and snippets in their personal namespaces. +- Can only create projects (including forks), subgroups, and snippets within top-level groups to which they are explicitly granted access. +- Can only access public projects and projects to which they are explicitly granted access, + thus hiding all other internal or private ones from them (like being + logged out). +- Can only access public groups and groups to which they are explicitly granted access, + thus hiding all other internal or private ones from them (like being + logged out). +- Can only access public snippets. + +Access can be granted by adding the user as member to the project or group. +Like usual users, they receive a role in the project or group with all +the abilities that are mentioned in the [permissions table](../user/permissions.md#project-members-permissions). +For example, if an external user is added as Guest, and your project is internal or +private, they do not have access to the code; you need to grant the external +user access at the Reporter level or above if you want them to have access to the code. You should +always take into account the +[project's visibility and permissions settings](../user/project/settings/index.md#configure-project-visibility-features-and-permissions) +as well as the permission level of the user. + +NOTE: +External users still count towards a license seat. + +An administrator can flag a user as external by either of the following methods: + +- [Through the API](../api/users.md#user-modification). +- Using the GitLab UI: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Overview > Users** to create a new user or edit an existing one. + There, you can find the option to flag the user as external. + +Additionally, users can be set as external users using: + +- [SAML groups](../integration/saml.md#external-groups). +- [LDAP groups](../administration/auth/ldap/ldap_synchronization.md#external-groups). +- the [External providers list](../integration/omniauth.md#create-an-external-providers-list). + +## Set a new user to external + +By default, new users are not set as external users. This behavior can be changed +by an administrator: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. + +If you change the default behavior of creating new users as external, you +have the option to narrow it down by defining a set of internal users. +The **Internal users** field allows specifying an email address regex pattern to +identify default internal users. New users whose email address matches the regex +pattern are set to internal by default rather than an external collaborator. + +The regex pattern format is in Ruby, but it needs to be convertible to JavaScript, +and the ignore case flag is set (`/regex pattern/i`). Here are some examples: + +- Use `\.internal@domain\.com$` to mark email addresses ending with + `.internal@domain.com` as internal. +- Use `^(?:(?!\.ext@domain\.com).)*$\r?` to mark users with email addresses + not including `.ext@domain.com` as internal. + +WARNING: +Be aware that this regex could lead to a +[regular expression denial of service (ReDoS) attack](https://en.wikipedia.org/wiki/ReDoS). diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 13e0938fa59..6ac67c3d21e 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -56,12 +56,12 @@ site you are about to failover to: rsync --archive --perms --delete root@<geo-primary>:/var/opt/gitlab/gitlab-rails/shared/registry/. /var/opt/gitlab/gitlab-rails/shared/registry ``` -Alternatively, you can [back up](../../../raketasks/backup_restore.md#back-up-gitlab) +Alternatively, you can [back up](../../../administration/backup_restore/index.md#back-up-gitlab) the container registry on the primary site and restore it onto the secondary site: 1. On your primary site, back up only the registry and - [exclude specific directories from the backup](../../../raketasks/backup_gitlab.md#excluding-specific-directories-from-the-backup): + [exclude specific directories from the backup](../../../administration/backup_restore/backup_gitlab.md#excluding-specific-directories-from-the-backup): ```shell # Create a backup in the /var/opt/gitlab/backups folder @@ -71,7 +71,7 @@ site: 1. Copy the backup tarball generated from your primary site to the `/var/opt/gitlab/backups` folder on your secondary site. -1. On your secondary site, restore the registry following the [Restore GitLab](../../../raketasks/backup_restore.md#restore-gitlab) +1. On your secondary site, restore the registry following the [Restore GitLab](../../../administration/backup_restore/index.md#restore-gitlab) documentation. ## Preflight checks diff --git a/doc/administration/geo/glossary.md b/doc/administration/geo/glossary.md index 9abd7ea9347..2e9a637eb5c 100644 --- a/doc/administration/geo/glossary.md +++ b/doc/administration/geo/glossary.md @@ -19,19 +19,20 @@ these definitions yet. We provide example diagrams and statements to demonstrate correct usage of terms. -| Term | Definition | Scope | Discouraged synonyms | -|---------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|-------------------------------------------------| -| Node | An individual server that runs GitLab either with a specific role or as a whole (for example a Rails application node). In a cloud context this can be a specific machine type. | GitLab | instance, server | -| Site | One or a collection of nodes running a single GitLab application. A site can be single-node or multi-node. | GitLab | deployment, installation instance | -| Single-node site | A specific configuration of GitLab that uses exactly one node. | GitLab | single-server, single-instance -| Multi-node site | A specific configuration of GitLab that uses more than one node. | GitLab | multi-server, multi-instance, high availability | -| Primary site | A GitLab site whose data is being replicated by at least one secondary site. There can only be a single primary site. | Geo-specific | Geo deployment, Primary node | -| Secondary site | A GitLab site that is configured to replicate the data of a primary site. There can be one or more secondary sites. | Geo-specific | Geo deployment, Secondary node | -| Geo deployment | A collection of two or more GitLab sites with exactly one primary site being replicated by one or more secondary sites. | Geo-specific | | -| Reference architecture | A [specified configuration of GitLab for a number of users](../reference_architectures/index.md), possibly including multiple nodes and multiple sites. | GitLab | | -| Promoting | Changing the role of a site from secondary to primary. | Geo-specific | | -| Demoting | Changing the role of a site from primary to secondary. | Geo-specific | | -| Failover | The entire process that shifts users from a primary Site to a secondary site. This includes promoting a secondary, but contains other parts as well. For example, scheduling maintenance. | Geo-specific | | +| Term | Definition | Scope | Discouraged synonyms | +|------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------|-------------------------------------------------| +| Node | An individual server that runs GitLab either with a specific role or as a whole (for example a Rails application node). In a cloud context this can be a specific machine type. | GitLab | instance, server | +| Site | One or a collection of nodes running a single GitLab application. A site can be single-node or multi-node. | GitLab | deployment, installation instance | +| Single-node site | A specific configuration of GitLab that uses exactly one node. | GitLab | single-server, single-instance | +| Multi-node site | A specific configuration of GitLab that uses more than one node. | GitLab | multi-server, multi-instance, high availability | +| Primary site | A GitLab site whose data is being replicated by at least one secondary site. There can only be a single primary site. | Geo-specific | Geo deployment, Primary node | +| Secondary site | A GitLab site that is configured to replicate the data of a primary site. There can be one or more secondary sites. | Geo-specific | Geo deployment, Secondary node | +| Geo deployment | A collection of two or more GitLab sites with exactly one primary site being replicated by one or more secondary sites. | Geo-specific | | +| Reference architecture | A [specified configuration of GitLab for a number of users](../reference_architectures/index.md), possibly including multiple nodes and multiple sites. | GitLab | | +| Promoting | Changing the role of a site from secondary to primary. | Geo-specific | | +| Demoting | Changing the role of a site from primary to secondary. | Geo-specific | | +| Failover | The entire process that shifts users from a primary Site to a secondary site. This includes promoting a secondary, but contains other parts as well. For example, scheduling maintenance. | Geo-specific | | +| Replication | Also called "synchronization". The uni-directional process that updates a resource on a secondary site to match the resource on the primary site. | Geo-specific | | ## Examples diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index ae2cc262160..0ab24cc4fb8 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -203,6 +203,7 @@ This list of limitations only reflects the latest version of GitLab. If you are - [Disaster recovery](disaster_recovery/index.md) for multi-secondary sites causes downtime due to the complete re-synchronization and re-configuration of all non-promoted secondaries. - For Git over SSH, to make the project clone URL display correctly regardless of which site you are browsing, secondary sites must use the same port as the primary. [GitLab issue #339262](https://gitlab.com/gitlab-org/gitlab/-/issues/339262) proposes to remove this limitation. - Git push over SSH against a secondary site does not work for pushes over 1.86 GB. [GitLab issue #413109](https://gitlab.com/gitlab-org/gitlab/-/issues/413109) tracks this bug. +- Backups [cannot be run on secondaries](replication/troubleshooting.md#message-error-canceling-statement-due-to-conflict-with-recovery). ### Limitations on replication/verification @@ -217,7 +218,7 @@ If you try to view replication data on the primary site, you receive a warning t The only way to view projects replication data for a particular secondary site is to visit that secondary site directly. For example, `https://<IP of your secondary site>/admin/geo/replication/projects`. An [epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4623) to fix this limitation. -Keep in mind that mentioned URLs don't work when [Admin Mode](../../user/admin_area/settings/sign_in_restrictions.md#admin-mode) is enabled. +Keep in mind that mentioned URLs don't work when [Admin Mode](../settings/sign_in_restrictions.md#admin-mode) is enabled. When using Unified URL, visiting the secondary site directly means you must route your requests to the secondary site. Exactly how this might be done depends on your networking configuration. If using DNS to route requests to the appropriate site, then you can, for example, edit your local machine's `/etc/hosts` file to route your requests to the desired secondary site. If the Geo sites are all behind a load balancer, then depending on the load balancer, you might be able to configure all requests from your IP to go to a particular secondary site. diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 18d0440965e..a5d85187812 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -190,7 +190,7 @@ keys must be manually replicated to the **secondary** site. ```ruby ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' ``` @@ -332,7 +332,7 @@ the **primary** site. After you sign in: 1. Verify that it's correctly identified as a **secondary** Geo site, and that Geo is enabled. -The initial replication may take some time. The status of the site or the ‘backfill’ may still in progress. You +The initial replication may take some time. The status of the site or the 'backfill' may still in progress. You can monitor the synchronization process on each Geo site from the **primary** site's **Geo Sites** dashboard in your browser. diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index f038bfd705c..b25700ccd29 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -80,10 +80,8 @@ on a machine: - With multiple disks mounted as a single mount-point (like with a RAID array). - Using LVM. -GitLab does not require a special file system and can work with: - -- NFS. -- A mounted Storage Appliance (there may be performance limitations when using a remote file system). +GitLab does not require a special file system and can work with a mounted Storage Appliance. However, there can be +performance limitations and consistency issues when using a remote file system. Geo triggers garbage collection in Gitaly to [deduplicate forked repositories](../../../development/git_object_deduplication.md#git-object-deduplication-and-gitlab-geo) on Geo secondary sites. @@ -111,7 +109,7 @@ GitLab stores files and blobs such as Issue attachments or LFS objects into eith - The file system in a specific location. - An [Object Storage](../../object_storage.md) solution. Object Storage solutions can be: - - Cloud based like Amazon S3 Google Cloud Storage. + - Cloud based like Amazon S3 and Google Cloud Storage. - Hosted by you (like MinIO). - A Storage Appliance that exposes an Object Storage-compatible API. @@ -192,7 +190,7 @@ successfully, you must replicate their data using some other means. |:--------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------|:---------------------------------------------------------------------------|:--------------------------------------------------------------------|:----------------------------------------------------------------|:------| |[Application data in PostgreSQL](../../postgresql/index.md) | **Yes** (10.2) | **Yes** (10.2) | N/A | N/A | | |[Project repository](../../../user/project/repository/index.md) | **Yes** (10.2) | **Yes** (10.7) | N/A | N/A | | -|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2)<sup>2</sup> | **Yes** (10.7)<sup>2</sup> | N/A | N/A | Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag geo_project_wiki_repository_replication, enabled by default in (15.11). | +|[Project wiki repository](../../../user/project/wiki/index.md) | **Yes** (10.2)<sup>2</sup> | **Yes** (10.7)<sup>2</sup> | N/A | N/A | Migrated to [self-service framework](../../../development/geo/framework.md) in 15.11. See GitLab issue [#367925](https://gitlab.com/gitlab-org/gitlab/-/issues/367925) for more details.<br /><br />Behind feature flag `geo_project_wiki_repository_replication`, enabled by default in (15.11). | |[Group wiki repository](../../../user/project/wiki/group.md) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | No | N/A | N/A | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | |[Uploads](../../uploads.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Replication is behind the feature flag `geo_upload_replication`, enabled by default. Verification was behind the feature flag `geo_upload_verification`, removed in 14.8. | |[LFS objects](../../lfs/index.md) | **Yes** (10.2) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).<br /><br />Replication is behind the feature flag `geo_lfs_object_replication`, enabled by default. Verification was behind the feature flag `geo_lfs_object_verification`, removed in 14.7. | @@ -203,7 +201,7 @@ successfully, you must replicate their data using some other means. |[CI Secure Files](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/secure_file.rb) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [**Yes** (15.3)](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91430) | [No](object_storage.md#verification-of-files-in-object-storage) | Verification is behind the feature flag `geo_ci_secure_file_replication`, enabled by default in 15.3. | |[Container Registry](../../packages/container_registry.md) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | **Yes** (12.3)<sup>1</sup> | **Yes** (15.10) | See [instructions](container_registry.md) to set up the Container Registry replication. | |[Terraform Module Registry](../../../user/packages/terraform_module_registry/index.md) | **Yes** (14.0) | **Yes** (14.0) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | **Yes** (16.1) | N/A | N/A | Designs also require replication of LFS objects and Uploads. Replication is behind the feature flag geo_design_management_repository_replication, enabled by default.| +|[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | **Yes** (16.1) | N/A | N/A | Designs also require replication of LFS objects and Uploads. Replication is behind the feature flag `geo_design_management_repository_replication`, enabled by default.| |[Package Registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | **Yes** (13.10) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Behind feature flag `geo_package_file_replication`, enabled by default. | |[Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | **Yes** (13.12) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Replication is behind the feature flag `geo_terraform_state_version_replication`, enabled by default. Verification was behind the feature flag `geo_terraform_state_version_verification`, which was removed in 14.0. | |[External merge request diffs](../../merge_request_diffs.md) | **Yes** (13.5) | **Yes** (14.6) | [**Yes** (15.1)](https://gitlab.com/groups/gitlab-org/-/epics/5551) | [No](object_storage.md#verification-of-files-in-object-storage) | Replication is behind the feature flag `geo_merge_request_diff_replication`, enabled by default. Verification was behind the feature flag `geo_merge_request_diff_verification`, removed in 14.7.| diff --git a/doc/administration/geo/replication/location_aware_git_url.md b/doc/administration/geo/replication/location_aware_git_url.md index 4a3f9c86041..a3abc945288 100644 --- a/doc/administration/geo/replication/location_aware_git_url.md +++ b/doc/administration/geo/replication/location_aware_git_url.md @@ -107,7 +107,7 @@ You can customize the: - SSH remote URL to use the location-aware `git.example.com`. To do so, change the SSH remote URL host by setting `gitlab_rails['gitlab_ssh_host']` in `gitlab.rb` of web nodes. - HTTP remote URL as shown in - [Custom Git clone URL for HTTP(S)](../../../user/admin_area/settings/visibility_and_access_controls.md#customize-git-clone-url-for-https). + [Custom Git clone URL for HTTP(S)](../../settings/visibility_and_access_controls.md#customize-git-clone-url-for-https). ## Example Git request handling behavior diff --git a/doc/administration/geo/replication/multiple_servers.md b/doc/administration/geo/replication/multiple_servers.md index 4e597a21922..29edac1be83 100644 --- a/doc/administration/geo/replication/multiple_servers.md +++ b/doc/administration/geo/replication/multiple_servers.md @@ -67,7 +67,7 @@ The following steps enable a GitLab site to serve as the Geo **primary** site. ```ruby ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' @@ -217,7 +217,7 @@ then make the following modifications: ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' @@ -318,7 +318,7 @@ application nodes above, with some changes to run only the `sidekiq` service: ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' diff --git a/doc/administration/geo/replication/single_sign_on.md b/doc/administration/geo/replication/single_sign_on.md index 55e77d5657c..15e24cdcefb 100644 --- a/doc/administration/geo/replication/single_sign_on.md +++ b/doc/administration/geo/replication/single_sign_on.md @@ -31,6 +31,10 @@ If you have configured SAML on the primary site correctly, then it should work o ### SAML with separate URL with proxying enabled +NOTE: +When proxying is enabled, SAML can only be used to sign in the secondary site if your SAML Identity Provider (IdP) allows an +application to have multiple callback URLs configured. Check with your IdP provider support team to confirm if this is the case. + If a secondary site uses a different `external_url` to the primary site, then configure your SAML Identity Provider (IdP) to allow the secondary site's SAML callback URL. For example, to configure Okta: 1. [Sign in to Okta](https://www.okta.com/login/). diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 4047167e4af..c63480db389 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -51,7 +51,7 @@ Geo::MetricsUpdateWorker.new.perform If it raises an error, then the error is probably also preventing the jobs from completing. If it takes longer than 10 minutes, then there may be a performance issue, and the UI may always show "Unhealthy" even if the status eventually does get updated. -If it successfully updates the status, then something may be wrong with Sidekiq. Is it running? Do the logs show errors? This job is supposed to be enqueued every minute. It takes an exclusive lease in Redis to ensure that only one of these jobs can run at a time. The primary site updates its status directly in the PostgreSQL database. Secondary sites send an HTTP Post request to the primary site with their status data. +If it successfully updates the status, then something may be wrong with Sidekiq. Is it running? Do the logs show errors? This job is supposed to be enqueued every minute and might not run if a [job deduplication idempotency](../../sidekiq/sidekiq_troubleshooting.md#clearing-a-sidekiq-job-deduplication-idempotency-key) key was not cleared properly. It takes an exclusive lease in Redis to ensure that only one of these jobs can run at a time. The primary site updates its status directly in the PostgreSQL database. Secondary sites send an HTTP Post request to the primary site with their status data. A site also shows as "Unhealthy" if certain health checks fail. You can reveal the failure by running the following in the [Rails console](../../operations/rails_console.md) on the affected secondary site: @@ -240,7 +240,7 @@ This machine's Geo node name matches a database record ... no ``` For more information about recommended site names in the description of the Name field, see -[Geo Admin Area Common Settings](../../../user/admin_area/geo_sites.md#common-settings). +[Geo Admin Area Common Settings](../../../administration/geo_sites.md#common-settings). ### Reverify all uploads (or any SSF data type which is verified) @@ -622,7 +622,7 @@ This happens on wrongly-formatted addresses in `postgresql['md5_auth_cidr_addres ``` To fix this, update the IP addresses in `/etc/gitlab/gitlab.rb` under `postgresql['md5_auth_cidr_addresses']` -to respect the CIDR format (that is, `1.2.3.4/32`). +to respect the CIDR format (for example, `10.0.0.1/32`). ### Message: `LOG: invalid IP mask "md5": Name or service not known` @@ -634,7 +634,7 @@ This happens when you have added IP addresses without a subnet mask in `postgres ``` To fix this, add the subnet mask in `/etc/gitlab/gitlab.rb` under `postgresql['md5_auth_cidr_addresses']` -to respect the CIDR format (that is, `1.2.3.4/32`). +to respect the CIDR format (for example, `10.0.0.1/32`). ### Message: `Found data in the gitlabhq_production database!` when running `gitlab-ctl replicate-geo-database` @@ -1295,7 +1295,7 @@ When [Geo proxying for secondary sites](../secondary_proxy/index.md) is enabled, Check the NGINX logs for errors similar to this example: ```plaintext -2022/01/26 00:02:13 [error] 26641#0: *829148 upstream sent too big header while reading response header from upstream, client: 1.2.3.4, server: geo.staging.gitlab.com, request: "POST /users/sign_in HTTP/2.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/users/sign_in", host: "geo.staging.gitlab.com", referrer: "https://geo.staging.gitlab.com/users/sign_in" +2022/01/26 00:02:13 [error] 26641#0: *829148 upstream sent too big header while reading response header from upstream, client: 10.0.2.2, server: geo.staging.gitlab.com, request: "POST /users/sign_in HTTP/2.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/users/sign_in", host: "geo.staging.gitlab.com", referrer: "https://geo.staging.gitlab.com/users/sign_in" ``` To resolve this issue: @@ -1345,15 +1345,8 @@ To fix this issue, set the primary site's internal URL to a URL that is: - Unique to the primary site. - Accessible from all secondary sites. -1. Enter the [Rails console](../../operations/rails_console.md) on the primary site. - -1. Run the following, replacing `https://unique.url.for.primary.site` with your specific internal URL. - For example, depending on your network configuration, you could use an IP address, like - `http://1.2.3.4`. - - ```ruby - GeoNode.where(primary: true).first.update!(internal_url: "https://unique.url.for.primary.site") - ``` +1. Visit the primary site. +1. [Set up the internal URLs](../../../administration/geo_sites.md#set-up-the-internal-urls). ### Secondary site returns `Received HTTP code 403 from proxy after CONNECT` @@ -1404,6 +1397,27 @@ In this case, make sure to update the changed URL on all your sites: 1. On the left sidebar, select **Geo > Sites**. 1. Change the URL and save the change. +### Message: `ERROR: canceling statement due to conflict with recovery` during backup + +Running a backup on a Geo **secondary** [is not supported](https://gitlab.com/gitlab-org/gitlab/-/issues/211668). + +When running a backup on a **secondary** you might encounter the following error message: + +```plaintext +Dumping PostgreSQL database gitlabhq_production ... +pg_dump: error: Dumping the contents of table "notes" failed: PQgetResult() failed. +pg_dump: error: Error message from server: ERROR: canceling statement due to conflict with recovery +DETAIL: User query might have needed to see row versions that must be removed. +pg_dump: error: The command was: COPY public.notes (id, note, [...], last_edited_at) TO stdout; +``` + +To prevent a database backup being made automatically during GitLab upgrades on your Geo **secondaries**, +create the following empty file: + +```shell +sudo touch /etc/gitlab/skip-auto-backup +``` + ## Fixing non-PostgreSQL replication failures If you notice replication failures in `Admin > Geo > Sites` or the [Sync status Rake task](#sync-status-rake-task), you can try to resolve the failures with the following general steps: @@ -1665,7 +1679,7 @@ Repository check failures on a Geo secondary site do not necessarily imply a rep 1. Find affected repositories as mentioned below, as well as their [logged errors](../../repository_checks.md#what-to-do-if-a-check-failed). 1. Try to diagnose specific `git fsck` errors. The range of possible errors is wide, try putting them into search engines. -1. Test normal functions of the affected repositories. Pull from the secondary, view the files. +1. Test typical functions of the affected repositories. Pull from the secondary, view the files. 1. Check if the primary site's copy of the repository has an identical `git fsck` error. If you are planning a failover, then consider prioritizing that the secondary site has the same information that the primary site has. Ensure you have a backup of the primary, and follow [planned failover guidelines](../disaster_recovery/planned_failover.md). 1. Push to the primary and check if the change gets replicated to the secondary site. 1. If replication is not automatically working, try to manually sync the repository. @@ -1806,3 +1820,30 @@ If the output differs on some hosts, PostgreSQL replication does not work proper A full index rebuild is required if the on-disk data is transferred 'at rest' to an operating system with an incompatible locale, or through replication. This check is also required when using a mixture of GitLab deployments. The locale might be different between an Linux package install, a GitLab Docker container, a Helm chart deployment, or external database services. + +## Investigate causes of database replication lag + +If the output of `sudo gitlab-rake geo:status` shows that `Database replication lag` remains significantly high over time, the primary node in database replication can be checked to determine the status of lag for +different parts of the database replication process. These values are known as `write_lag`, `flush_lag`, and `replay_lag`. For more information, see +[the official PostgreSQL documentation](https://www.postgresql.org/docs/current/monitoring-stats.html#MONITORING-PG-STAT-REPLICATION-VIEW). + +Run the following command from the primary Geo node's database to provide relevant output: + +```shell +gitlab-psql -xc 'SELECT write_lag,flush_lag,replay_lag FROM pg_stat_replication;' + +-[ RECORD 1 ]--------------- +write_lag | 00:00:00.072392 +flush_lag | 00:00:00.108168 +replay_lag | 00:00:00.108283 +``` + +If one or more of these values is significantly high, this could indicate a problem and should be investigated further. When determining the cause, consider that: + +- `write_lag` indicates the time since when WAL bytes have been sent by the primary, then received to the secondary, but not yet flushed or applied. +- A high `write_lag` value may indicate degraded network performance or insufficient network speed between the primary and secondary nodes. +- A high `flush_lag` value may indicate degraded or sub-optimal disk I/O performance with the secondary node's storage device. +- A high `replay_lag` value may indicate long running transactions in PostgreSQL, or the saturation of a needed resource like the CPU. +- The difference in time between `write_lag` and `flush_lag` indicates that WAL bytes have been sent to the underlying storage system, but it has not reported that they were flushed. + This data is most likely not fully written to a persistent storage, and likely held in some kind of volatile write cache. +- The difference between `flush_lag` and `replay_lag` indicates WAL bytes that have been successfully persisted to storage, but could not be replayed by the database system. diff --git a/doc/administration/geo/replication/upgrading_the_geo_sites.md b/doc/administration/geo/replication/upgrading_the_geo_sites.md index 644232a2c9e..ce0ad736071 100644 --- a/doc/administration/geo/replication/upgrading_the_geo_sites.md +++ b/doc/administration/geo/replication/upgrading_the_geo_sites.md @@ -33,7 +33,7 @@ and all **secondary** sites: 1. SSH into each node of the **primary** site. 1. [Upgrade GitLab on the **primary** site](../../../update/package/index.md#upgrade-using-the-official-repositories). 1. Perform testing on the **primary** site, particularly if you paused replication in step 1 to protect DR. [There are some suggestions for post-upgrade testing](../../../update/plan_your_upgrade.md#pre-upgrade-and-post-upgrade-checks) in the upgrade documentation. -1. Ensure that the secrets in the `/etc/gitlab/gitlab-secrets.json` file of both the primary site and the secondary site are the same. The file must be the same on all of a site’s nodes. +1. Ensure that the secrets in the `/etc/gitlab/gitlab-secrets.json` file of both the primary site and the secondary site are the same. The file must be the same on all of a site's nodes. 1. SSH into each node of **secondary** sites. 1. [Upgrade GitLab on each **secondary** site](../../../update/package/index.md#upgrade-using-the-official-repositories). 1. If you paused replication in step 1, [resume replication on each **secondary**](../index.md#pausing-and-resuming-replication). diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md index 3081d1c2485..2ab96a3d33d 100644 --- a/doc/administration/geo/secondary_proxy/index.md +++ b/doc/administration/geo/secondary_proxy/index.md @@ -131,6 +131,10 @@ and cannot be configured per Geo site. Therefore, all runners clone from the pri which Geo site they register on. For information about GitLab CI using a specific Geo secondary to clone from, see issue [3294](https://gitlab.com/gitlab-org/gitlab/-/issues/3294#note_1009488466). +- When secondary proxying is used together with separate URLs, + [signing in the secondary site using SAML](../replication/single_sign_on.md#saml-with-separate-url-with-proxying-enabled) + is only supported if the SAML Identity Provider (IdP) allows an application to be configured with multiple callback URLs. + ## Behavior of secondary sites when the primary Geo site is down Considering that web traffic is proxied to the primary, the behavior of the secondary sites differs when the primary diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index 31d0fbdffe0..be6e327732d 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -75,7 +75,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ```ruby ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' ``` @@ -193,8 +193,8 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o | `postgresql['md5_auth_cidr_addresses']` | **Primary** and **Secondary** sites' public or VPC private addresses. | If you are using Google Cloud Platform, SoftLayer, or any other vendor that - provides a virtual private cloud (VPC), you can use the **primary** and **secondary** sites' - private addresses (corresponds to "internal address" for Google Cloud Platform) for + provides a virtual private cloud (VPC), we recommend using the **primary** + and **secondary** sites' "private" or "internal" addresses for `postgresql['md5_auth_cidr_addresses']` and `postgresql['listen_address']`. The `listen_address` option opens PostgreSQL up to network connections with the interface @@ -468,7 +468,8 @@ data before running `pg_basebackup`. sudo -i ``` -1. Choose a database-friendly name to use for your **secondary** site to +1. Choose a [database-friendly name](https://www.postgresql.org/docs/13/warm-standby.html#STREAMING-REPLICATION-SLOTS-MANIPULATION) + to use for your **secondary** site to use as the replication slot name. For example, if your domain is `secondary.geo.example.com`, use `secondary_example` as the slot name as shown in the commands below. diff --git a/doc/administration/geo/setup/external_database.md b/doc/administration/geo/setup/external_database.md index 81541d1cb9c..50383546da3 100644 --- a/doc/administration/geo/setup/external_database.md +++ b/doc/administration/geo/setup/external_database.md @@ -39,7 +39,7 @@ developed and tested. We aim to be compatible with most external ## ## The unique identifier for the Geo site. See - ## https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html#common-settings + ## https://docs.gitlab.com/ee/administration/geo_nodes.html#common-settings ## gitlab_rails['geo_node_name'] = '<site_name_here>' ``` diff --git a/doc/administration/geo/setup/index.md b/doc/administration/geo/setup/index.md index 359f706a8aa..8ac64a963bb 100644 --- a/doc/administration/geo/setup/index.md +++ b/doc/administration/geo/setup/index.md @@ -34,8 +34,8 @@ If both Geo sites are based on the [1K reference architecture](../../reference_a - [Using Linux package PostgreSQL instances](database.md) . - [Using external PostgreSQL instances](external_database.md) 1. [Configure GitLab](../replication/configuration.md) to set the **primary** and **secondary** sites. -1. Recommended: [Configure unified URLs](../secondary_proxy/index.md) to use a single, unified URL for all Geo sites. -1. Optional: [Configure Object storage replication](../../object_storage.md) +1. Recommended: [Configure unified URLs](../secondary_proxy/index.md#set-up-a-unified-url-for-geo-sites) to use a single, unified URL for all Geo sites. +1. Optional: [Configure Object storage replication](../replication/object_storage.md) 1. Optional: [Configure a secondary LDAP server](../../auth/ldap/index.md) for the **secondary** sites. See [notes on LDAP](../index.md#ldap). 1. Optional: [Configure Container Registry for the secondary site](../replication/container_registry.md). 1. Follow the [Using a Geo Site](../replication/usage.md) guide. diff --git a/doc/administration/geo_sites.md b/doc/administration/geo_sites.md new file mode 100644 index 00000000000..81cc3a87941 --- /dev/null +++ b/doc/administration/geo_sites.md @@ -0,0 +1,119 @@ +--- +stage: Systems +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Geo sites Admin Area **(PREMIUM SELF)** + +You can configure various settings for GitLab Geo sites. For more information, see +[Geo documentation](../administration/geo/index.md). + +On either the primary or secondary site: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Geo > Sites**. + +## Common settings + +All Geo sites have the following settings: + +| Setting | Description | +| --------| ----------- | +| Primary | This marks a Geo site as **primary** site. There can be only one **primary** site. | +| Name | The unique identifier for the Geo site. It's highly recommended to use a physical location as a name. Good examples are "London Office" or "us-east-1". Avoid words like "primary", "secondary", "Geo", or "DR". This makes the failover process easier because the physical location does not change, but the Geo site role can. All nodes in a single Geo site use the same site name. Nodes use the `gitlab_rails['geo_node_name']` setting in `/etc/gitlab/gitlab.rb` to lookup their Geo site record in the PostgreSQL database. If `gitlab_rails['geo_node_name']` is not set, the node's `external_url` with trailing slash is used as fallback. The value of `Name` is case-sensitive, and most characters are allowed. | +| URL | The instance's user-facing URL. | + +The site you're currently browsing is indicated with a blue `Current` label, and +the **primary** node is listed first as `Primary site`. + +## Secondary site settings + +**Secondary** sites have a number of additional settings available: + +| Setting | Description | +|---------------------------|-------------| +| Selective synchronization | Enable Geo [selective sync](../administration/geo/replication/configuration.md#selective-synchronization) for this **secondary** site. | +| Repository sync capacity | Number of concurrent requests this **secondary** site makes to the **primary** site when backfilling repositories. | +| File sync capacity | Number of concurrent requests this **secondary** site makes to the **primary** site when backfilling files. | + +## Geo backfill + +**Secondary** sites are notified of changes to repositories and files by the **primary** site, +and always attempt to synchronize those changes as quickly as possible. + +Backfill is the act of populating the **secondary** site with repositories and files that +existed *before* the **secondary** site was added to the database. Because there may be +extremely large numbers of repositories and files, it's not feasible to attempt to +download them all at once; so, GitLab places an upper limit on the concurrency of +these operations. + +How long the backfill takes is dependent on the maximum concurrency, but higher +values place more strain on the **primary** site. The limits are configurable. +If your **primary** site has lots of surplus capacity, +you can increase the values to complete backfill in a shorter time. If it's +under heavy load and backfill reduces its availability for standard requests, +you can decrease them. + +## Set up the internal URLs + +> Setting up internal URLs in secondary sites was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77179) in GitLab 14.7. + +You can set up a different URL for synchronization between the primary and secondary site. + +The **primary** site's Internal URL is used by **secondary** sites to contact it +(to sync repositories, for example). The name Internal URL distinguishes it from +[External URL](https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab), +which is used by users. Internal URL does not need to be a private address. + +When [Geo secondary proxying](../administration/geo/secondary_proxy/index.md) is enabled, +the primary uses the secondary's internal URL to contact it directly. + +The internal URL defaults to external URL. To change it: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Geo > Sites**. +1. Select **Edit** on the site you want to customize. +1. Edit the internal URL. +1. Select **Save changes**. + +When enabled, the Admin Area for Geo shows replication details for each site directly +from the primary site's UI, and through the Geo secondary proxy, if enabled. + +WARNING: +We recommend using an HTTPS connection while configuring the Geo sites. To avoid +breaking communication between **primary** and **secondary** sites when using +HTTPS, customize your Internal URL to point to a load balancer with TLS +terminated at the load balancer. + +WARNING: +Starting with GitLab 13.3 and [until 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/325522), +if you use an internal URL that is not accessible to the users, the +OAuth authorization flow does not work properly, because users are redirected +to the internal URL instead of the external one. + +## Multiple secondary sites behind a load balancer + +**Secondary** sites can use identical external URLs if +a unique `name` is set for each Geo site. The `gitlab.rb` setting +`gitlab_rails['geo_node_name']` must: + +- Be set for each GitLab instance that runs `puma`, `sidekiq`, or `geo_logcursor`. +- Match a Geo site name. + +The load balancer must use sticky sessions to avoid authentication +failures and cross-site request errors. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/get_started.md b/doc/administration/get_started.md index 60291732a20..bf3d38657f8 100644 --- a/doc/administration/get_started.md +++ b/doc/administration/get_started.md @@ -41,7 +41,7 @@ Get started: - [Add members](../user/group/index.md#add-users-to-a-group) to the group. - Create a [subgroup](../user/group/subgroups/index.md#create-a-subgroup). - [Add members](../user/group/subgroups/index.md#subgroup-membership) to the subgroup. -- Enable [external authorization control](../user/admin_area/settings/external_authorization.md#configuration). +- Enable [external authorization control](../administration/settings/external_authorization.md#configuration). **More resources** @@ -76,16 +76,16 @@ While this isn't an exhaustive list, following these steps gives you a solid sta - Use a long root password, stored in a vault. - Install trusted SSL certificate and establish a process for renewal and revocation. - [Configure SSH key restrictions](../security/ssh_keys_restrictions.md#restrict-allowed-ssh-key-technologies-and-minimum-length) per your organization's guidelines. -- [Disable new sign-ups](../user/admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups). +- [Disable new sign-ups](settings/sign_up_restrictions.md#disable-new-sign-ups). - Require email confirmation. - Set password length limit, configure SSO or SAML user management. - Limit email domains if allowing sign-up. - Require two-factor authentication (2FA). -- [Disable password authentication](../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled) for Git over HTTPS. -- Set up [email notification for unknown sign-ins](../user/admin_area/settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins). +- [Disable password authentication](settings/sign_in_restrictions.md#password-authentication-enabled) for Git over HTTPS. +- Set up [email notification for unknown sign-ins](settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins). - Configure [user and IP rate limits](https://about.gitlab.com/blog/2020/05/20/gitlab-instance-security-best-practices/#user-and-ip-rate-limits). - Limit [webhooks local access](https://about.gitlab.com/blog/2020/05/20/gitlab-instance-security-best-practices/#webhooks). -- Set [rate limits for protected paths](../user/admin_area/settings/protected_paths.md). +- Set [rate limits for protected paths](settings/protected_paths.md). - Sign up for [Security Alerts](https://about.gitlab.com/company/preference-center/) from the Communication Preference Center. - Keep track of security best practices on our [blog page](https://about.gitlab.com/blog/2020/05/20/gitlab-instance-security-best-practices/). @@ -130,7 +130,7 @@ The routine differs, depending on whether you deployed with the Linux package or When backing up (single node) GitLab server installed using the Linux package, you can use a single Rake task. -Learn about [backing up Linux package or Helm variations](../raketasks/backup_restore.md). +Learn about [backing up Linux package or Helm variations](../administration/backup_restore/index.md). This process backs up your entire instance, but does not back up the configuration files. Ensure those are backed up separately. Keep your configuration files and backup archives in a separate location to ensure the encryption keys are not kept with the encrypted data. @@ -163,7 +163,7 @@ For more information about GitLab SaaS backups, see our [Backup FAQ page](https: ### Alternative backup strategies In some situations the Rake task for backups may not be the most optimal solution. Here are some -[alternatives](../raketasks/backup_restore.md) to consider if the Rake task does not work for you. +[alternatives](../administration/backup_restore/index.md) to consider if the Rake task does not work for you. #### Option 1: File system snapshot @@ -236,10 +236,10 @@ Rate limits also improve the security of your application. You can make changes to your default rate limits from the Admin Area. For more information about configuration, see the [Admin Area page](../security/rate_limits.md#configurable-limits). -- Define [issues rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md) to set a maximum number of issue creation requests per minute, per user. -- Enforce [user and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) for unauthenticated web requests. -- Review the [rate limit on raw endpoints](../user/admin_area/settings/rate_limits_on_raw_endpoints.md). The default setting is 300 requests per minute for raw file access. -- Review the [import/export rate limits](../user/admin_area/settings/import_export_rate_limits.md) of the six active defaults. +- Define [issues rate limits](settings/rate_limit_on_issues_creation.md) to set a maximum number of issue creation requests per minute, per user. +- Enforce [user and IP rate limits](settings/user_and_ip_rate_limits.md) for unauthenticated web requests. +- Review the [rate limit on raw endpoints](settings/rate_limits_on_raw_endpoints.md). The default setting is 300 requests per minute for raw file access. +- Review the [import/export rate limits](settings/import_export_rate_limits.md) of the six active defaults. For more information about API and rate limits, see our [API page](../api/rest/index.md). diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index 5c6fc370052..c7fa40014d0 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -377,9 +377,6 @@ This can be risky because anything that prevents your Gitaly clients from reachi servers causes all Gitaly requests to fail. For example, any sort of network, firewall, or name resolution problems. -Additionally, you must [disable Rugged](../nfs.md#improving-nfs-performance-with-gitlab) -if previously enabled manually. - Gitaly makes the following assumptions: - Your `gitaly1.internal` Gitaly server can be reached at `gitaly1.internal:8075` from your Gitaly @@ -835,7 +832,7 @@ For consistent and stable performance, you should first explore other options su adjusting node specifications, and [reviewing large repositories](../../user/project/repository/managing_large_repositories.md) or workloads. FLAG: -On self-managed GitLab, by default repository cgroups are not available. To make it available, ask an administrator to +On self-managed GitLab, by default repository cgroups are not available. To make it available, an administrator can [enable the feature flag](../feature_flags.md) named `gitaly_run_cmds_in_cgroup`. When enabling cgroups for memory, you should ensure that no swap is configured on the Gitaly nodes as diff --git a/doc/administration/gitaly/gitaly_geo_capabilities.md b/doc/administration/gitaly/gitaly_geo_capabilities.md index e4147eec162..a98477318f2 100644 --- a/doc/administration/gitaly/gitaly_geo_capabilities.md +++ b/doc/administration/gitaly/gitaly_geo_capabilities.md @@ -17,7 +17,6 @@ The following tables are intended to guide you to choose the right combination o |------------|--------------|----------------|-----------------|-------------|-----------------| |Gitaly Cluster | Very high - tolerant of node failures | RTO for a single node of 10 s with no manual intervention | Data is stored on multiple nodes | Good - While writes may take slightly longer due to voting, read distribution improves read speeds | **Trade-off** - Slight decrease in write speed for redundant, strongly-consistent storage solution. **Risks** - [Does not support snapshot backups](../gitaly/index.md#snapshot-backup-and-recovery-limitations), GitLab backup task can be slow for large data sets | |Gitaly Shards | Single storage location is a single point of failure | Would need to restore only shards which failed | Single point of failure | Good - can allocate repositories to shards to spread load | **Trade-off** - Need to manually configure repositories into different shards to balance loads / storage space **Risks** - Single point of failure relies on recovery process when single-node failure occurs | -|Gitaly + NFS | Single storage location is a single point of failure | Single node failure requires restoration from backup | Single point of failure | Average - NFS is not ideally suited to large quantities of small reads / writes which can have a detrimental impact on performance | **Trade-off** - Familiar administration though NFS is not ideally suited to Git demands **Risks** - Many instances of NFS compatibility issues which provide very poor customer experiences | ## Geo capabilities diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 2a3c3da24b3..db11ac8c769 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -52,13 +52,10 @@ Before deploying Gitaly Cluster, review: - [Configuration guidance](configure_gitaly.md) and [Repository storage options](../repository_storage_paths.md) to make sure that Gitaly Cluster is the best setup for you. -If you have: +If you have not yet migrated to Gitaly Cluster, you have two options: -- Not yet migrated to Gitaly Cluster and want to continue using NFS, remain on the service you are using. However, NFS - is [no longer supported](../../update/removals.md#nfs-as-git-repository-storage-is-no-longer-supported). -- Not yet migrated to Gitaly Cluster but want to migrate away from NFS, you have two options: - - A sharded Gitaly instance. - - Gitaly Cluster. +- A sharded Gitaly instance. +- Gitaly Cluster. Contact your [Customer Success Manager](https://about.gitlab.com/job-families/sales/customer-success-management/) or customer support if you have any questions. @@ -72,15 +69,15 @@ the current status of these issues, refer to the referenced issues and epics. | Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. | No known solution prior to GitLab 15.0. In GitLab 15.0 to 15.2, enable the [`gitaly_praefect_generated_replica_paths` feature flag](#praefect-generated-replica-paths-gitlab-150-and-later) on your Geo primary site. In GitLab 15.3, the feature flag is enabled by default. | | Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform standard operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting upgrade assistance](https://about.gitlab.com/support/scheduling-upgrade-assistance/) so your upgrade plan can be reviewed by support. | | Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind results in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup:<br/><br/>1. [Shut down GitLab](../read_only_gitlab.md#shut-down-the-gitlab-ui).<br/>2. Snapshot all Gitaly Cluster nodes at the same time.<br/>3. Take a database dump of the Praefect database. | -| Rebuilding or replacing an existing Gitaly Cluster node | There is no way to replace existing nodes in place because the Praefect database is relied on to determine the current state of each Gitaly node. Changing how Gitaly Cluster stores repositories is proposed in issue [4218](https://gitlab.com/gitlab-org/gitaly/-/issues/4218). Turning on [repository verification](praefect.md#repository-verification) is proposed in issue [4429](https://gitlab.com/gitlab-org/gitaly/-/issues/4429) so Praefect can detect that data is missing and needs to replicated to a new Gitaly node. | No known solution at this time. | +| Limitations when running in Kubernetes, Amazon ECS, or similar | Praefect (Gitaly Cluster) is not supported and Gitaly has known limitations. For more information, see [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127). | Use our [reference architectures](../reference_architectures/index.md). | ### Snapshot backup and recovery limitations Gitaly Cluster does not support snapshot backups. Snapshot backups can cause issues where the Praefect database becomes out of sync with the disk storage. Because of how Praefect rebuilds the replication metadata of Gitaly disk information -during a restore, you should use the [official backup and restore Rake tasks](../../raketasks/backup_restore.md). +during a restore, you should use the [official backup and restore Rake tasks](../../administration/backup_restore/index.md). -The [incremental backup method](../../raketasks/backup_gitlab.md#incremental-repository-backups) +The [incremental backup method](../../administration/backup_restore/backup_gitlab.md#incremental-repository-backups) can be used to speed up Gitaly Cluster backups. If you are unable to use either method, contact customer support for restoration help. @@ -175,7 +172,7 @@ best suited by using Gitaly Cluster. ### Backing up repositories -When backing up or syncing repositories using tools other than GitLab, you must [prevent writes](../../raketasks/backup_restore.md#prevent-writes-and-copy-the-git-repository-data) +When backing up or syncing repositories using tools other than GitLab, you must [prevent writes](../../administration/backup_restore/backup_gitlab.md#prevent-writes-and-copy-the-git-repository-data) while copying repository data. ## Gitaly Cluster @@ -641,7 +638,7 @@ code. This re-introduced code is informally referred to as the "Rugged patches". FLAG: On self-managed GitLab, by default automatic detection of whether Rugged should be used (per storage) is not available. -To make it available, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named +To make it available, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `skip_rugged_auto_detect`. The Ruby methods that perform direct Git access are behind diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index f80a5192c55..9577c4a4cb2 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -19,6 +19,10 @@ Configure Gitaly Cluster using either: Smaller GitLab installations may need only [Gitaly itself](index.md). +NOTE: +Gitaly Cluster is not yet supported in Kubernetes, Amazon ECS, or similar container environments. For more information, see +[epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127). + ## Requirements The minimum recommended configuration for a Gitaly Cluster requires: @@ -1107,8 +1111,8 @@ For more information on Gitaly server configuration, see our ```ruby # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your front door GitLab URL or an internal load balancer. - # Examples: 'https://gitlab.example.com', 'http://1.2.3.4' - gitlab_rails['internal_api_url'] = 'http://GITLAB_HOST' + # Examples: 'https://gitlab.example.com', 'http://10.0.2.2' + gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' ``` 1. Configure the storage location for Git data by setting `gitaly['configuration'][:storage]` in @@ -1487,7 +1491,7 @@ repository storage redundancy. For a replication factor: -- Of `1`: NFS, Gitaly, and Gitaly Cluster have roughly the same storage requirements. +- Of `1`: Gitaly and Gitaly Cluster have roughly the same storage requirements. - More than `1`: The amount of required storage is `used space * replication factor`. `used space` should include any planned future growth. @@ -1628,20 +1632,8 @@ You should use [repository-specific primary nodes](#repository-specific-primary- > - [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/3492) in GitLab 13.12, with primary elections run when Praefect starts or the cluster's consensus of a Gitaly node's health changes. > - [Changed](https://gitlab.com/gitlab-org/gitaly/-/merge_requests/3543) in GitLab 14.1, primary elections are run lazily. -Gitaly Cluster supports electing repository-specific primary Gitaly nodes. Repository-specific -Gitaly primary nodes are enabled in `/etc/gitlab/gitlab.rb` by setting -`praefect['failover_election_strategy'] = 'per_repository'`. - -Praefect's [deprecated election strategies](#deprecated-election-strategies): - -- Elected a primary Gitaly node for each virtual storage, which was used as the primary node for - each repository in the virtual storage. -- Prevented horizontal scaling of a virtual storage. The primary Gitaly node needed a replica of - each repository and thus became the bottleneck. - -The `per_repository` election strategy solves this problem by electing a primary Gitaly node separately for each -repository. Combined with [configurable replication factors](#configure-replication-factor), you can -horizontally scale storage capacity and distribute write load across Gitaly nodes. +Gitaly Cluster elects a primary Gitaly node separately for each repository. Combined with +[configurable replication factors](#configure-replication-factor), you can horizontally scale storage capacity and distribute write load across Gitaly nodes. Primary elections are run lazily. Praefect doesn't immediately elect a new primary node if the current one is unhealthy. A new primary is elected if a request must be served while the current primary is unavailable. diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index dbbed0f60ba..aa487917ef0 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -432,6 +432,11 @@ sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.t > - [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5658) in GitLab 14.4. > - [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5789) in GitLab 14.6, support for immediate replication. +WARNING: +Because of a [known issue](https://gitlab.com/gitlab-org/gitaly/-/issues/5402), you can't add repositories to the +Praefect tracking database with Praefect-generated replica paths (`@cluster`). These repositories are not associated with the repository path used by GitLab and are +inaccessible. + The `track-repository` Praefect sub-command adds repositories on disk to the Praefect tracking database to be tracked. ```shell @@ -484,6 +489,11 @@ This command fails if: > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6319) in GitLab 15.4. +WARNING: +Because of a [known issue](https://gitlab.com/gitlab-org/gitaly/-/issues/5402), you can't add repositories to the +Praefect tracking database with Praefect-generated replica paths (`@cluster`). These repositories are not associated with the repository path used by GitLab and are +inaccessible. + Migrations using the API automatically add repositories to the Praefect tracking database. If you are instead manually copying repositories over from existing infrastructure, you can use the `track-repositories` diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index afef787e9c3..3d8110e1dab 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -12,7 +12,7 @@ Refer to the information below when troubleshooting Gitaly and Gitaly Cluster. The following sections provide possible solutions to Gitaly errors. -See also [Gitaly timeout](../../user/admin_area/settings/gitaly_timeouts.md) settings, +See also [Gitaly timeout](../settings/gitaly_timeouts.md) settings, and our advice on [parsing the `gitaly/current` file](../logs/log_parsing.md#parsing-gitalycurrent). ### Check versions when using standalone Gitaly servers diff --git a/doc/user/admin_area/img/abuse_report_blocked_user.png b/doc/administration/img/abuse_report_blocked_user.png Binary files differindex 435d8dfe821..435d8dfe821 100644 --- a/doc/user/admin_area/img/abuse_report_blocked_user.png +++ b/doc/administration/img/abuse_report_blocked_user.png diff --git a/doc/user/admin_area/img/abuse_reports_page_v13_11.png b/doc/administration/img/abuse_reports_page_v13_11.png Binary files differindex ef57f45ab77..ef57f45ab77 100644 --- a/doc/user/admin_area/img/abuse_reports_page_v13_11.png +++ b/doc/administration/img/abuse_reports_page_v13_11.png diff --git a/doc/user/admin_area/img/admin_labels_v14_7.png b/doc/administration/img/admin_labels_v14_7.png Binary files differindex 01a4ea0c2cc..01a4ea0c2cc 100644 --- a/doc/user/admin_area/img/admin_labels_v14_7.png +++ b/doc/administration/img/admin_labels_v14_7.png diff --git a/doc/user/admin_area/img/broadcast_messages_banner_v15_0.png b/doc/administration/img/broadcast_messages_banner_v15_0.png Binary files differindex e1b350142b3..e1b350142b3 100644 --- a/doc/user/admin_area/img/broadcast_messages_banner_v15_0.png +++ b/doc/administration/img/broadcast_messages_banner_v15_0.png diff --git a/doc/user/admin_area/img/broadcast_messages_notification_v12_10.png b/doc/administration/img/broadcast_messages_notification_v12_10.png Binary files differindex fb03748c892..fb03748c892 100644 --- a/doc/user/admin_area/img/broadcast_messages_notification_v12_10.png +++ b/doc/administration/img/broadcast_messages_notification_v12_10.png diff --git a/doc/user/admin_area/img/cohorts_v13_9_a.png b/doc/administration/img/cohorts_v13_9_a.png Binary files differindex 1a4590290b9..1a4590290b9 100644 --- a/doc/user/admin_area/img/cohorts_v13_9_a.png +++ b/doc/administration/img/cohorts_v13_9_a.png diff --git a/doc/user/admin_area/img/credentials_inventory_gpg_keys_v14_9.png b/doc/administration/img/credentials_inventory_gpg_keys_v14_9.png Binary files differindex 6c4c8f30df6..6c4c8f30df6 100644 --- a/doc/user/admin_area/img/credentials_inventory_gpg_keys_v14_9.png +++ b/doc/administration/img/credentials_inventory_gpg_keys_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_personal_access_tokens_v14_9.png b/doc/administration/img/credentials_inventory_personal_access_tokens_v14_9.png Binary files differindex 254d520d538..254d520d538 100644 --- a/doc/user/admin_area/img/credentials_inventory_personal_access_tokens_v14_9.png +++ b/doc/administration/img/credentials_inventory_personal_access_tokens_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_project_access_tokens_v14_9.png b/doc/administration/img/credentials_inventory_project_access_tokens_v14_9.png Binary files differindex ae204ac09ef..ae204ac09ef 100644 --- a/doc/user/admin_area/img/credentials_inventory_project_access_tokens_v14_9.png +++ b/doc/administration/img/credentials_inventory_project_access_tokens_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v14_9.png b/doc/administration/img/credentials_inventory_ssh_keys_v14_9.png Binary files differindex 8f2f11515eb..8f2f11515eb 100644 --- a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v14_9.png +++ b/doc/administration/img/credentials_inventory_ssh_keys_v14_9.png diff --git a/doc/user/admin_area/img/email1.png b/doc/administration/img/email1.png Binary files differindex e79ccc3e9a9..e79ccc3e9a9 100644 --- a/doc/user/admin_area/img/email1.png +++ b/doc/administration/img/email1.png diff --git a/doc/user/admin_area/img/email2.png b/doc/administration/img/email2.png Binary files differindex d073c0e42da..d073c0e42da 100644 --- a/doc/user/admin_area/img/email2.png +++ b/doc/administration/img/email2.png diff --git a/doc/user/admin_area/img/export_permissions_v13_11.png b/doc/administration/img/export_permissions_v13_11.png Binary files differindex d9bbe8c3daf..d9bbe8c3daf 100644 --- a/doc/user/admin_area/img/export_permissions_v13_11.png +++ b/doc/administration/img/export_permissions_v13_11.png diff --git a/doc/user/admin_area/img/impersonate_user_button_v13_8.png b/doc/administration/img/impersonate_user_button_v13_8.png Binary files differindex 475315a0c0f..475315a0c0f 100644 --- a/doc/user/admin_area/img/impersonate_user_button_v13_8.png +++ b/doc/administration/img/impersonate_user_button_v13_8.png diff --git a/doc/user/admin_area/img/index_runners_search_or_filter_v14_5.png b/doc/administration/img/index_runners_search_or_filter_v14_5.png Binary files differindex 10b8cc01103..10b8cc01103 100644 --- a/doc/user/admin_area/img/index_runners_search_or_filter_v14_5.png +++ b/doc/administration/img/index_runners_search_or_filter_v14_5.png diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index df364a3f737..679042c3114 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -19,11 +19,9 @@ Read more about [configuring rate limits](../security/rate_limits.md). ### Issue creation -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28129) in GitLab 12.10. - This setting limits the request rate to the issue creation endpoint. -Read more about [issue creation rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md). +Read more about [issue creation rate limits](settings/rate_limit_on_issues_creation.md). - **Default rate limit**: Disabled by default. @@ -31,17 +29,15 @@ Read more about [issue creation rate limits](../user/admin_area/settings/rate_li This setting limits the request rate per user or IP. -Read more about [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md). +Read more about [User and IP rate limits](settings/user_and_ip_rate_limits.md). - **Default rate limit**: Disabled by default. ### By raw endpoint -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30829) in GitLab 12.2. - This setting limits the request rate per endpoint. -Read more about [raw endpoint rate limits](../user/admin_area/settings/rate_limits_on_raw_endpoints.md). +Read more about [raw endpoint rate limits](settings/rate_limits_on_raw_endpoints.md). - **Default rate limit**: 300 requests per project, per commit and per file path. @@ -63,16 +59,14 @@ GitLab rate limits the following paths by default: '/admin/session' ``` -Read more about [protected path rate limits](../user/admin_area/settings/protected_paths.md). +Read more about [protected path rate limits](settings/protected_paths.md). - **Default rate limit**: After 10 requests, the client must wait 60 seconds before trying again. ### Package Registry -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/57029) in GitLab 13.12. - This setting limits the request rate on the Packages API per user or IP. For more information, see -[Package Registry Rate Limits](../user/admin_area/settings/package_registry_rate_limits.md). +[Package Registry Rate Limits](settings/package_registry_rate_limits.md). - **Default rate limit**: Disabled by default. @@ -92,7 +86,7 @@ requests per user. For more information, read > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75918) in GitLab 14.6. [Feature flag `files_api_throttling`](https://gitlab.com/gitlab-org/gitlab/-/issues/338903) removed. This setting limits the request rate on the Packages API per user or IP address. For more information, read -[Files API rate limits](../user/admin_area/settings/files_api_rate_limits.md). +[Files API rate limits](settings/files_api_rate_limits.md). - **Default rate limit**: Disabled by default. @@ -101,26 +95,24 @@ This setting limits the request rate on the Packages API per user or IP address. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68645) in GitLab 14.4. This setting limits the request rate on deprecated API endpoints per user or IP address. For more information, read -[Deprecated API rate limits](../user/admin_area/settings/deprecated_api_rate_limits.md). +[Deprecated API rate limits](settings/deprecated_api_rate_limits.md). - **Default rate limit**: Disabled by default. ### Import/Export -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35728) in GitLab 13.2. - This setting limits the import/export actions for groups and projects. | Limit | Default (per minute per user) | |-------------------------|-------------------------------| -| Project Import | 6 | -| Project Export | 6 | -| Project Export Download | 1 | -| Group Import | 6 | -| Group Export | 6 | -| Group Export Download | 1 | +| Project Import | 6 | +| Project Export | 6 | +| Project Export Download | 1 | +| Group Import | 6 | +| Group Export | 6 | +| Group Export Download | 1 | -Read more about [import/export rate limits](../user/admin_area/settings/import_export_rate_limits.md). +Read more about [import/export rate limits](settings/import_export_rate_limits.md). ### Member Invitations @@ -162,10 +154,10 @@ Set the limit to `0` to disable it. This setting limits search requests as follows: -| Limit | Default (requests per minute) | -|-------------------------|-------------------------------| -| Authenticated user | 300 | -| Unauthenticated user | 100 | +| Limit | Default (requests per minute) | +|----------------------|-------------------------------| +| Authenticated user | 300 | +| Unauthenticated user | 100 | Search requests that exceed the search rate limit per minute return the following error: @@ -179,7 +171,7 @@ This endpoint has been requested too many times. Try again later. This setting limits the request rate to the pipeline creation endpoints. -Read more about [pipeline creation rate limits](../user/admin_area/settings/rate_limit_on_pipelines_creation.md). +Read more about [pipeline creation rate limits](settings/rate_limit_on_pipelines_creation.md). ## Gitaly concurrency limit @@ -191,8 +183,6 @@ Read more about [Gitaly concurrency limits](gitaly/configure_gitaly.md#limit-rpc ## Number of comments per issue, merge request, or commit -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22388) in GitLab 12.4. - There's a limit to the number of comments that can be submitted on an issue, merge request, or commit. When the limit is reached, system notes can still be added so that the history of events is not lost, but the user-submitted @@ -202,8 +192,6 @@ comment fails. ## Size of comments and descriptions of issues, merge requests, and epics -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/61974) in GitLab 12.2. - There is a limit to the size of comments and descriptions of issues, merge requests, and epics. Attempting to add a body of text larger than the limit, results in an error, and the item is also not created. @@ -214,8 +202,6 @@ It's possible that this limit changes to a lower number in the future. ## Size of commit titles and descriptions -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292039) in GitLab 13.9. - Commits with arbitrarily large messages may be pushed to GitLab, but the following display limits apply: @@ -231,9 +217,6 @@ are processed. ## Number of issues in the milestone overview -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39453) in GitLab 12.10. -> - [Set](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/58168) to 500 in GitLab 13.11. - The maximum number of issues loaded on the milestone overview page is 500. When the number exceeds the limit the page displays an alert and links to a paginated [issue list](../user/project/issues/managing_issues.md) of all issues in the milestone. @@ -242,8 +225,6 @@ When the number exceeds the limit the page displays an alert and links to a pagi ## Number of pipelines per Git push -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51401) in GitLab 11.10. - When pushing multiple changes with a single Git push, like multiple tags or branches, only four tag or branch pipelines can be triggered. This limit prevents the accidental creation of a large number of pipelines when using `git push --all` or `git push --mirror`. @@ -259,12 +240,10 @@ instance if too many changes are pushed at once and a flood of pipelines are cre ## Retention of activity history -Activity history for projects and individuals' profiles was limited to one year until [GitLab 11.4](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52246) when it was extended to two years, and in [GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/33840) to three years. +Activity history for projects and individuals' profiles is limited to three years. ## Number of embedded metrics -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14939) in GitLab 12.7. - There is a limit when embedding metrics in GitLab Flavored Markdown (GLFM) for performance reasons. - **Max limit**: 100 embeds. @@ -343,8 +322,6 @@ Blocked recursive webhook calls are logged in `auth.log` with the message `"Recu ## Pull Mirroring Interval -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/237891) in GitLab 13.7. - The [minimum wait time between pull refreshes](../user/project/repository/mirror/index.md) defaults to 300 seconds (5 minutes). For example, a pull refresh only runs once in a given 300 second period, regardless of how many times you trigger it. @@ -362,8 +339,6 @@ Plan.default.actual_limits.update!(pull_mirror_interval_seconds: 200) ## Incoming emails from auto-responders -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30327) in GitLab 12.4. - GitLab ignores all incoming emails sent from auto-responders by looking for the `X-Autoreply` header. Such emails don't create comments on issues or merge requests. @@ -376,8 +351,6 @@ and to limit memory consumption. ## Max offset allowed by the REST API for offset-based pagination -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34565) in GitLab 13.0. - When using offset-based pagination in the REST API, there is a limit to the maximum requested offset into the set of results. This limit is only applied to endpoints that also support keyset-based pagination. More information about pagination options can be @@ -401,8 +374,6 @@ Set the limit to `0` to disable it. ### Number of jobs in active pipelines -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32823) in GitLab 12.6. - The total number of jobs in active pipelines can be limited per project. This limit is checked each time a new pipeline is created. An active pipeline is any pipeline in one of the following states: @@ -433,8 +404,6 @@ Set the limit to `0` to disable it. ### Maximum time jobs can run -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16777) in GitLab 12.3. - The default maximum time that jobs can run for is 60 minutes. Jobs that run for more than 60 minutes time out. @@ -447,8 +416,6 @@ You can change the maximum time a job can run before it times out: ### Maximum number of deployment jobs in a pipeline -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46931) in GitLab 13.7. - You can limit the maximum number of deployment jobs in a pipeline. A deployment is any job with an [`environment`](../ci/environments/index.md) specified. The number of deployments in a pipeline is checked at pipeline creation. Pipelines that have @@ -470,8 +437,6 @@ Set the limit to `0` to disable it. ### Number of CI/CD subscriptions to a project -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.9. - The total number of subscriptions can be limited per project. This limit is checked each time a new subscription is created. @@ -516,8 +481,6 @@ This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd). ### Number of pipeline schedules -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29566) in GitLab 12.10. - The total number of pipeline schedules can be limited per project. This limit is checked each time a new pipeline schedule is created. If a new pipeline schedule would cause the total number of pipeline schedules to exceed the limit, the @@ -585,8 +548,6 @@ This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd). ### Number of instance level variables -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216097) in GitLab 13.1. - The total number of instance level CI/CD variables is limited at the instance level. This limit is checked each time a new instance level variable is created. If a new variable would cause the total number of variables to exceed the limit, the new variable is not created. @@ -639,12 +600,10 @@ Plan.default.actual_limits.update!(project_ci_variables: 10000) ### Maximum file size per type of artifact -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37226) in GitLab 13.3. - Job artifacts defined with [`artifacts:reports`](../ci/yaml/index.md#artifactsreports) that are uploaded by the runner are rejected if the file size exceeds the maximum file size limit. The limit is determined by comparing the project's -[maximum artifact size setting](../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size) +[maximum artifact size setting](../administration/settings/continuous_integration.md#maximum-artifacts-size) with the instance limit for the given artifact type, and choosing the smaller value. Limits are set in megabytes, so the smallest possible value that can be defined is `1 MB`. @@ -655,35 +614,35 @@ setting is used: | Artifact limit name | Default value | |---------------------------------------------|---------------| -| `ci_max_artifact_size_accessibility` | 0 | -| `ci_max_artifact_size_api_fuzzing` | 0 | -| `ci_max_artifact_size_archive` | 0 | -| `ci_max_artifact_size_browser_performance` | 0 | -| `ci_max_artifact_size_cluster_applications` | 0 | -| `ci_max_artifact_size_cobertura` | 0 | -| `ci_max_artifact_size_codequality` | 0 | -| `ci_max_artifact_size_container_scanning` | 0 | -| `ci_max_artifact_size_coverage_fuzzing` | 0 | -| `ci_max_artifact_size_dast` | 0 | -| `ci_max_artifact_size_dependency_scanning` | 0 | -| `ci_max_artifact_size_dotenv` | 0 | -| `ci_max_artifact_size_junit` | 0 | -| `ci_max_artifact_size_license_management` | 0 | -| `ci_max_artifact_size_license_scanning` | 0 | -| `ci_max_artifact_size_load_performance` | 0 | -| `ci_max_artifact_size_lsif` | 100 MB ([Introduced at 20 MB](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37226) in GitLab 13.3 and [raised to 100 MB](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46980) in GitLab 13.6.) | -| `ci_max_artifact_size_metadata` | 0 | -| `ci_max_artifact_size_metrics_referee` | 0 | -| `ci_max_artifact_size_metrics` | 0 | -| `ci_max_artifact_size_network_referee` | 0 | -| `ci_max_artifact_size_performance` | 0 | -| `ci_max_artifact_size_requirements` | 0 | -| `ci_max_artifact_size_requirements_v2` | 0 | -| `ci_max_artifact_size_sast` | 0 | -| `ci_max_artifact_size_secret_detection` | 0 | -| `ci_max_artifact_size_terraform` | 5 MB ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37018) in GitLab 13.3) | -| `ci_max_artifact_size_trace` | 0 | -| `ci_max_artifact_size_cyclonedx` | 1 MB | +| `ci_max_artifact_size_accessibility` | 0 | +| `ci_max_artifact_size_api_fuzzing` | 0 | +| `ci_max_artifact_size_archive` | 0 | +| `ci_max_artifact_size_browser_performance` | 0 | +| `ci_max_artifact_size_cluster_applications` | 0 | +| `ci_max_artifact_size_cobertura` | 0 | +| `ci_max_artifact_size_codequality` | 0 | +| `ci_max_artifact_size_container_scanning` | 0 | +| `ci_max_artifact_size_coverage_fuzzing` | 0 | +| `ci_max_artifact_size_dast` | 0 | +| `ci_max_artifact_size_dependency_scanning` | 0 | +| `ci_max_artifact_size_dotenv` | 0 | +| `ci_max_artifact_size_junit` | 0 | +| `ci_max_artifact_size_license_management` | 0 | +| `ci_max_artifact_size_license_scanning` | 0 | +| `ci_max_artifact_size_load_performance` | 0 | +| `ci_max_artifact_size_lsif` | 100 MB | +| `ci_max_artifact_size_metadata` | 0 | +| `ci_max_artifact_size_metrics_referee` | 0 | +| `ci_max_artifact_size_metrics` | 0 | +| `ci_max_artifact_size_network_referee` | 0 | +| `ci_max_artifact_size_performance` | 0 | +| `ci_max_artifact_size_requirements` | 0 | +| `ci_max_artifact_size_requirements_v2` | 0 | +| `ci_max_artifact_size_sast` | 0 | +| `ci_max_artifact_size_secret_detection` | 0 | +| `ci_max_artifact_size_terraform` | 5 MB | +| `ci_max_artifact_size_trace` | 0 | +| `ci_max_artifact_size_cyclonedx` | 1 MB | For example, to set the `ci_max_artifact_size_junit` limit to 10 MB on a self-managed installation, run the following in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): @@ -732,10 +691,10 @@ GitLab SaaS subscribers have different limits defined per plan, affecting all pr Self-managed GitLab Premium and Ultimate limits are defined by a default plan that affects all projects: -| Runner scope | Default value | -|---------------------------------------------|---------------| -| `ci_registered_group_runners` | 1000 | -| `ci_registered_project_runners` | 1000 | +| Runner scope | Default value | +|---------------------------------|---------------| +| `ci_registered_group_runners` | 1000 | +| `ci_registered_project_runners` | 1000 | To update these limits, run the following in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): @@ -808,7 +767,7 @@ You can change these limits in the [GitLab Rails console](operations/rails_conso You can set a limit on the maximum number of variables inside of a dotenv artifact. This limit is checked every time a dotenv file is exported as an artifact. -Set the limit to `0` to disable it. Defaults to `0` on self-managed instances. +Set the limit to `0` to disable it. Defaults to `20` on self-managed instances. To set this limit to `100` on a self-managed instance, run the following command in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): @@ -839,28 +798,20 @@ Plan.default.actual_limits.update!(dotenv_size: 5.kilobytes) ### Limit inbound incident management alerts -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17859) in GitLab 12.5. - This setting limits the number of inbound alert payloads over a period of time. -Read more about [incident management rate limits](../user/admin_area/settings/rate_limit_on_pipelines_creation.md). +Read more about [incident management rate limits](settings/rate_limit_on_pipelines_creation.md). ### Prometheus Alert JSON payloads -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/19940) in GitLab 12.6. - Prometheus alert payloads sent to the `notify.json` endpoint are limited to 1 MB in size. ### Generic Alert JSON payloads -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16441) in GitLab 12.4. - Alert payloads sent to the `notify.json` endpoint are limited to 1 MB in size. ### Metrics dashboard YAML files -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/34834) in GitLab 13.2. - The memory occupied by a parsed metrics dashboard YAML file cannot exceed 1 MB. The maximum depth of each YAML file is limited to 100. The maximum depth of a YAML @@ -891,8 +842,6 @@ panel_groups: ## Environment Dashboard limits **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33895) in GitLab 13.4. - See [Environment Dashboard](../ci/environments/environments_dashboard.md#adding-a-project-to-the-dashboard) for the maximum number of displayed projects. ## Environment data on deploy boards @@ -907,7 +856,7 @@ Kubernetes aren't shown. GitLab has limits around: -- The patch size for a single file. [This is configurable on self-managed instance](../user/admin_area/diff_limits.md). +- The patch size for a single file. [This is configurable on self-managed instance](../administration/diff_limits.md). - The total size of all the diffs for a merge request. An upper and lower limit applies to each of these: @@ -932,8 +881,6 @@ Reports that go over the 20 MB limit aren't loaded. Affected reports: ### Maximum file size indexed -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8638) in GitLab 13.3. - You can set a limit on the content of repository files that are indexed in Elasticsearch. Any files larger than this limit only index the filename. The file content is neither indexed nor searchable. @@ -949,8 +896,6 @@ is pre-allocated during indexing. ### Maximum field length -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201826) in GitLab 12.8. - You can set a limit on the content of text fields indexed for advanced search. Setting a maximum helps to reduce the load of the indexing processes. If any text field exceeds this limit, then the text is truncated to this number of @@ -983,12 +928,10 @@ See the limits in the [Add a design to an issue](../user/project/issues/design_m ### Max push size -The maximum allowed [push size](../user/admin_area/settings/account_and_limit_settings.md#max-push-size) is set to 5 GB. +The maximum allowed [push size](../administration/settings/account_and_limit_settings.md#max-push-size) is set to 5 GB. ### Webhooks and Project Services -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31009) in GitLab 12.4. - Total number of changes (branches or tags) in a single push. If changes are more than the specified limit, hooks are not executed. @@ -999,19 +942,15 @@ More information can be found in these documentations: ### Activities -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31007) in GitLab 12.4. - Total number of changes (branches or tags) in a single push to determine whether individual push events or a bulk push event are created. -More information can be found in the [Push event activities limit and bulk push events documentation](../user/admin_area/settings/push_event_activities_limit.md). +More information can be found in the [Push event activities limit and bulk push events documentation](settings/push_event_activities_limit.md). ## Package Registry Limits ### File Size Limits -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218017) in GitLab 13.4. - The default maximum file size for a package that's uploaded to the [GitLab Package Registry](../user/packages/package_registry/index.md) varies by format: - Conan: 3 GB @@ -1074,14 +1013,6 @@ varies by file type: - Image blob: 5 GB - Image manifest: 10 MB -## Branch retargeting on merge - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320902) in GitLab 13.9. - -If a branch is merged while open merge requests still point to it, GitLab can -retarget merge requests pointing to the now-merged branch. For more information, see -[Update merge requests when target branch merges](../user/project/merge_requests/index.md#update-merge-requests-when-target-branch-merges). - ## Maximum number of assignees and reviewers > - Maximum assignees [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368936) in GitLab 15.6. @@ -1117,3 +1048,108 @@ The [secure files API](../api/secure_files.md) enforces the following limits: The [changelog API](../api/repositories.md#add-changelog-data-to-a-changelog-file) enforces the following limits: - The commit range between `from` and `to` cannot exceed 15000 commits. + +## Value Stream Analytics limits + +- Each namespace (such as a group or a project) can have a maximum of 50 value streams. +- Each value stream can have a maximum of 15 stages. + +## List all instance limits + +To list all instance limit values, run the following from the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + +```ruby +Plan.default.actual_limits +``` + +Sample output: + +```ruby +id: 1, +plan_id: 1, +ci_pipeline_size: 0, +ci_active_jobs: 0, +project_hooks: 100, +group_hooks: 50, +ci_project_subscriptions: 3, +ci_pipeline_schedules: 10, +offset_pagination_limit: 50000, +ci_instance_level_variables: "[FILTERED]", +storage_size_limit: 0, +ci_max_artifact_size_lsif: 100, +ci_max_artifact_size_archive: 0, +ci_max_artifact_size_metadata: 0, +ci_max_artifact_size_trace: "[FILTERED]", +ci_max_artifact_size_junit: 0, +ci_max_artifact_size_sast: 0, +ci_max_artifact_size_dependency_scanning: 350, +ci_max_artifact_size_container_scanning: 150, +ci_max_artifact_size_dast: 0, +ci_max_artifact_size_codequality: 0, +ci_max_artifact_size_license_management: 0, +ci_max_artifact_size_license_scanning: 100, +ci_max_artifact_size_performance: 0, +ci_max_artifact_size_metrics: 0, +ci_max_artifact_size_metrics_referee: 0, +ci_max_artifact_size_network_referee: 0, +ci_max_artifact_size_dotenv: 0, +ci_max_artifact_size_cobertura: 0, +ci_max_artifact_size_terraform: 5, +ci_max_artifact_size_accessibility: 0, +ci_max_artifact_size_cluster_applications: 0, +ci_max_artifact_size_secret_detection: "[FILTERED]", +ci_max_artifact_size_requirements: 0, +ci_max_artifact_size_coverage_fuzzing: 0, +ci_max_artifact_size_browser_performance: 0, +ci_max_artifact_size_load_performance: 0, +ci_needs_size_limit: 2, +conan_max_file_size: 3221225472, +maven_max_file_size: 3221225472, +npm_max_file_size: 524288000, +nuget_max_file_size: 524288000, +pypi_max_file_size: 3221225472, +generic_packages_max_file_size: 5368709120, +golang_max_file_size: 104857600, +debian_max_file_size: 3221225472, +project_feature_flags: 200, +ci_max_artifact_size_api_fuzzing: 0, +ci_pipeline_deployments: 500, +pull_mirror_interval_seconds: 300, +daily_invites: 0, +rubygems_max_file_size: 3221225472, +terraform_module_max_file_size: 1073741824, +helm_max_file_size: 5242880, +ci_registered_group_runners: 1000, +ci_registered_project_runners: 1000, +ci_daily_pipeline_schedule_triggers: 0, +ci_max_artifact_size_cluster_image_scanning: 0, +ci_jobs_trace_size_limit: "[FILTERED]", +pages_file_entries: 200000, +dast_profile_schedules: 1, +external_audit_event_destinations: 5, +dotenv_variables: "[FILTERED]", +dotenv_size: 5120, +pipeline_triggers: 25000, +project_ci_secure_files: 100, +repository_size: 0, +security_policy_scan_execution_schedules: 0, +web_hook_calls_mid: 0, +web_hook_calls_low: 0, +project_ci_variables: "[FILTERED]", +group_ci_variables: "[FILTERED]", +ci_max_artifact_size_cyclonedx: 1, +rpm_max_file_size: 5368709120, +pipeline_hierarchy_size: 1000, +ci_max_artifact_size_requirements_v2: 0, +enforcement_limit: 0, +notification_limit: 0, +dashboard_limit_enabled_at: nil, +web_hook_calls: 0, +project_access_token_limit: 0, +google_cloud_logging_configurations: 5, +ml_model_max_file_size: 10737418240, +limits_history: {} +``` + +Some limit values display as `[FILTERED]` in the list due to +[filtering in the Rails console](operations/rails_console.md#filtered-console-output). diff --git a/doc/administration/integration/diagrams_net.md b/doc/administration/integration/diagrams_net.md index a4e8528fb25..335b26565e6 100644 --- a/doc/administration/integration/diagrams_net.md +++ b/doc/administration/integration/diagrams_net.md @@ -8,7 +8,7 @@ type: reference, howto # Diagrams.net **(FREE)** With the [diagrams.net](https://www.diagrams.net/) integration, you can create and embed SVG diagrams in wikis. -The diagram editor is available in both the Markdown editor and the content editor. +The diagram editor is available in both the plain text editor and the rich text editor. On GitLab.com, this integration is enabled for all SaaS users and does not require any additional configuration. diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 9c5e07eedaa..f05197d45e7 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -157,7 +157,6 @@ using Tomcat or Jetty. Prerequisites: - JRE/JDK version 11 or later. -- Apache Maven version 3.0.2 or later. - (Recommended) Jetty version 11 or later. - (Recommended) Tomcat version 10 or later. @@ -167,11 +166,11 @@ PlantUML recommends to install Tomcat 10 or above. The scope of this page only includes setting up a basic Tomcat server. For more production-ready configurations, see the [Tomcat Documentation](https://tomcat.apache.org/tomcat-10.1-doc/index.html). -1. Install JDK/JRE 11 and Maven: +1. Install JDK/JRE 11: ```shell sudo apt update - sudo apt-get install graphviz default-jdk git-core maven + sudo apt-get install graphviz default-jdk git-core ``` 1. Add a user for Tomcat: @@ -183,8 +182,8 @@ see the [Tomcat Documentation](https://tomcat.apache.org/tomcat-10.1-doc/index.h 1. Install and configure Tomcat 10: ```shell - cd /tmp & wget https://dlcdn.apache.org/tomcat/tomcat-10/v10.1.9/bin/apache-tomcat-10.1.9.tar.gz - sudo tar xzvf apache-tomcat-10*tar.gz -C /opt/tomcat --strip-components=1 + wget https://dlcdn.apache.org/tomcat/tomcat-10/v10.1.9/bin/apache-tomcat-10.1.9.tar.gz -P /tmp + sudo tar xzvf /tmp/apache-tomcat-10*tar.gz -C /opt/tomcat --strip-components=1 sudo chown -R tomcat:tomcat /opt/tomcat/ sudo chmod -R u+x /opt/tomcat/bin ``` @@ -248,7 +247,7 @@ see the [Tomcat Documentation](https://tomcat.apache.org/tomcat-10.1-doc/index.h tcp6 0 0 :::8005 :::* LISTEN 14935/java ``` -1. Modify your NGINX configuration. The `proxy_pass` port matches the Connector port in the `server.xml`: +1. Modify your NGINX configuration in `/etc/gitlab/gitlab.rb`. Ensure the `proxy_pass` port matches the Connector port in `server.xml`: ```shell nginx['custom_gitlab_server_config'] = "location /-/plantuml { @@ -269,19 +268,22 @@ see the [Tomcat Documentation](https://tomcat.apache.org/tomcat-10.1-doc/index.h 1. Install PlantUML and copy the `.war` file: + Use the [latest release](https://github.com/plantuml/plantuml-server/releases) of plantuml-jsp (example: plantuml-jsp-v1.2023.8.war). For context, see [this issue](https://github.com/plantuml/plantuml-server/issues/265). + ```shell - cd / & git clone https://github.com/plantuml/plantuml-server.git - cd plantuml-server - mvn package - cp /plantuml-server/target/plantuml.war /opt/tomcat/webapps/plantuml.war - chown tomcat:tomcat /opt/tomcat/webapps/plantuml.war - systemctl restart tomcat + cd / + wget https://github.com/plantuml/plantuml-server/releases/download/v1.2023.8/plantuml-jsp-v1.2023.8.war + sudo cp plantuml-jsp-v1.2023.8.war /opt/tomcat/webapps/plantuml.war + sudo chown tomcat:tomcat /opt/tomcat/webapps/plantuml.war + sudo systemctl restart tomcat ``` The Tomcat service should restart. After the restart is complete, the PlantUML integration is ready and listening for requests on port `8005`: `http://localhost:8005/plantuml` +To test if the PlantUML server is working, run `curl --location --verbose "http://localhost:8005/plantuml/"`. + To change the Tomcat defaults, edit the `/opt/tomcat/conf/server.xml` file. NOTE: @@ -293,6 +295,7 @@ the configuration below accordingly. PlantUML has features that allow fetching network resources. If you self-host the PlantUML server, put network controls in place to isolate it. +For example, make use of PlantUML's [security profiles](https://plantuml.com/security). ```plaintext @startuml @@ -320,7 +323,7 @@ these steps: - For PlantUML servers running v1.2020.9 and above, such as [plantuml.com](https://plantuml.com), you must set the `PLANTUML_ENCODING` environment variable to enable the `deflate` - compression. In Linux package installations, you can set this value in `/etc/gitlab.rb` with + compression. In Linux package installations, you can set this value in `/etc/gitlab/gitlab.rb` with this command: ```ruby @@ -337,6 +340,6 @@ these steps: PLANTUML_ENCODING: deflate ``` -- For GitLab versions 13.1 and later, PlantUML integration now - [requires a header prefix in the URL](https://github.com/plantuml/plantuml/issues/117#issuecomment-6235450160) +- `deflate` is the default encoding type for PlantUML. To use a different encoding type, PlantUML integration + [requires a header prefix in the URL](https://plantuml.com/text-encoding) to distinguish different encoding types. diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index e5920520be7..1ab45d6ce99 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -13,7 +13,7 @@ WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. - Read more about the non-deprecated [Web Terminals accessible through the Web IDE](../../user/project/web_ide/index.md). - Read more about the non-deprecated [Web Terminals accessible from a running CI job](../../ci/interactive_web_terminal/index.md). @@ -114,5 +114,5 @@ lifetime in your GitLab instance: 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). 1. Select **Admin Area**. -1. Select [**Settings > Web terminal**](../../user/admin_area/settings/index.md#general). +1. Select [**Settings > Web terminal**](../../administration/settings/index.md#general). 1. Set a `max session time`. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 099e7ec9e6f..106334b226d 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -282,7 +282,7 @@ To migrate back to local storage, you must If [`artifacts:expire_in`](../ci/yaml/index.md#artifactsexpire_in) is used to set an expiry for the artifacts, they are marked for deletion right after that date passes. -Otherwise, they expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md). +Otherwise, they expire per the [default artifacts expiration setting](../administration/settings/continuous_integration.md). Artifacts are cleaned up by the `expire_build_artifacts_worker` cron job which Sidekiq runs every 7 minutes (`*/7 * * * *` in [Cron](../topics/cron/index.md) syntax). @@ -376,7 +376,7 @@ To change the default schedule on which the artifacts are expired: ## Set the maximum file size of the artifacts If artifacts are enabled, you can change the maximum file size of the -artifacts through the [Admin Area settings](../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size). +artifacts through the [Admin Area settings](../administration/settings/continuous_integration.md#maximum-artifacts-size). ## Storage statistics diff --git a/doc/administration/labels.md b/doc/administration/labels.md new file mode 100644 index 00000000000..adc621a2982 --- /dev/null +++ b/doc/administration/labels.md @@ -0,0 +1,28 @@ +--- +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Labels administration **(FREE SELF)** + +To manage labels for the GitLab instance, select **Labels** (**{labels}**) from the Admin Area sidebar. For more details on how to manage labels, see [Labels](../user/project/labels.md). + +Labels created in the Admin Area are automatically added to new projects. +They are not available to new groups. +Updating or adding labels in the Admin Area does not modify labels in existing projects. + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/lfs/index.md b/doc/administration/lfs/index.md index e4475e877d8..4659917cf8b 100644 --- a/doc/administration/lfs/index.md +++ b/doc/administration/lfs/index.md @@ -391,9 +391,11 @@ To delete these references: ls -al /var/opt/gitlab/gitlab-rails/shared/lfs-objects/00/66/22269c61b41bf14a22bbe0e43be3acf86a4a446afb4250c3794ea47541a7 ``` -1. If the file is not present, remove the database record via the rails console: +1. If the file is not present, remove the database records via the rails console: ```ruby + # First delete the parent records and then destroy the record itself + lfs_object.lfs_objects_projects.destroy_all lfs_object.destroy ``` diff --git a/doc/administration/license.md b/doc/administration/license.md new file mode 100644 index 00000000000..732c2840217 --- /dev/null +++ b/doc/administration/license.md @@ -0,0 +1,83 @@ +--- +stage: Fulfillment +group: Provision +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Activate GitLab Enterprise Edition (EE) **(PREMIUM SELF)** + +When you install a new GitLab instance without a license, only Free features +are enabled. To enable more features in GitLab Enterprise Edition (EE), activate +your instance with an activation code. + +## Activate GitLab EE + +In GitLab Enterprise Edition 14.1 and later, you need an activation code to activate +your instance. + +Prerequisite: + +- You must [purchase a subscription](https://about.gitlab.com/pricing/). +- You must be running GitLab Enterprise Edition (EE). +- You must have GitLab 14.1 or later. +- Your instance must be connected to the internet. + +To activate your instance with an activation code: + +1. Copy the activation code, a 24-character alphanumeric string, from either: + - Your subscription confirmation email. + - The [Customers Portal](https://customers.gitlab.com/customers/sign_in), on the **Manage Purchases** page. +1. Sign in to your GitLab self-managed instance. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Subscription**. +1. Paste the activation code in **Activation code**. +1. Read and accept the terms of service. +1. Select **Activate**. + +The subscription is activated. + +If you have an offline environment, +[activate GitLab EE with a license file or key](license_file.md) instead. + +If you have questions or need assistance activating your instance, +[contact GitLab Support](https://about.gitlab.com/support/#contact-support). + +When [the license expires](../administration/license_file.md#what-happens-when-your-license-expires), +some functionality is locked. + +## Verify your GitLab edition + +To verify the edition, sign in to GitLab and select +**Help** (**{question-o}**) > **Help**. The GitLab edition and version are listed +at the top of the page. + +If you are running GitLab Community Edition, you can upgrade your installation to GitLab +EE. For more details, see [Upgrading between editions](../update/index.md#upgrading-between-editions). +If you have questions or need assistance upgrading from GitLab Community Edition (CE) to EE, +[contact GitLab Support](https://about.gitlab.com/support/#contact-support). + +## Troubleshooting + +### Cannot activate instance due to connectivity error + +This error occurs when you use an activation code to activate your instance, but your instance is unable to connect to the GitLab servers. + +You may have connectivity issues due to the following reasons: + +- **You have an offline environment**: + - Configure your setup to allow connection to GitLab servers. If connection to GitLab servers is not possible, contact your Sales Representative to request a license key. You can also contact [GitLab support](https://about.gitlab.com/support/#contact-support) if you need help finding your Sales Representative. +- **Customers Portal is not operational**: + - To check for performance or service disruptions, check the Customers Portal [status](https://status.gitlab.com/). +- **Firewall settings**: + - Check if your GitLab instance has an encrypted connection to `customers.gitlab.com` (with IP addresses 172.64.146.11 and 104.18.41.245) on port 443: + + ```shell + curl --verbose "https://customers.gitlab.com/" + ``` + + - If the curl command returns a failure, either: + - [Configure a proxy](https://docs.gitlab.com/omnibus/settings/environment-variables.html) in `gitlab.rb` to point to your server. + - Contact your network administrator to make changes to the proxy. + - If an SSL inspection appliance is used, you must add the appliance's root CA certificate to `/etc/gitlab/trusted-certs` on the server, then run `gitlab-ctl reconfigure`. +
\ No newline at end of file diff --git a/doc/administration/license_file.md b/doc/administration/license_file.md new file mode 100644 index 00000000000..5f82536698b --- /dev/null +++ b/doc/administration/license_file.md @@ -0,0 +1,269 @@ +--- +stage: Fulfillment +group: Provision +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +<!-- To promote the workflow described in license.md, this page is not included in global left nav. --> + +# Activate GitLab EE with a license file or key + +If you receive a license file from GitLab (for example, for a trial), you can +upload it to your instance or add it during installation. The license file is +a base64-encoded ASCII text file with a `.gitlab-license` extension. + +The first time you sign in to your GitLab instance, a note with a +link to the **Add license** page should be displayed. + +Otherwise, to add your license: + +1. Sign in to GitLab as an administrator. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. In the **Add License** area, add a license by either uploading the file or entering the key. +1. Select the **Terms of Service** checkbox. +1. Select **Add license**. + +NOTE: +In GitLab 14.7.x to 14.9.x, you can add the license file with the UI. +In GitLab 14.1.x to 14.7, if you have already activated your subscription with an activation code, you cannot access **Add License** from the Admin Area. You must access **Add License** directly from the URL, `<YourGitLabURL>/admin/license/new`. + +## Activate subscription during installation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114572) in GitLab 16.0. + +To activate your subscription during installation, set the `GITLAB_ACTIVATION_CODE` environment variable with the activation code: + +```shell +export GITLAB_ACTIVATION_CODE=your_activation_code +``` + +## Add license file during installation + +If you have a license, you can also import it when you install GitLab. + +- For self-compiled installations: + - Place the `Gitlab.gitlab-license` file in the `config/` directory. + - To specify a custom location and filename for the license, set the + `GITLAB_LICENSE_FILE` environment variable with the path to the file: + + ```shell + export GITLAB_LICENSE_FILE="/path/to/license/file" + ``` + +- For Linux package installations: + - Place the `Gitlab.gitlab-license` file in the `/etc/gitlab/` directory. + - To specify a custom location and filename for the license, add this entry to `gitlab.rb`: + + ```ruby + gitlab_rails['initial_license_file'] = "/path/to/license/file" + ``` + +WARNING: +These methods only add a license at the time of installation. To renew or upgrade +a license, add the license in the **Admin Area** in the web user interface. + +## Submit license usage data + +If you use a license file or key to activate your instance in an offline environment, you must submit your license +usage data monthly. +To submit the data, [export your license usage](../subscriptions/self_managed/index.md#export-your-license-usage) +and send it by email to the renewals service, `renewals-service@customers.gitlab.com`. + +If you don't submit your data each month after your subscription start date, an email is sent to the address +associated with your subscription and a banner displays to remind you to submit your data. The banner displays +in the **Admin Area** on the **Dashboard** and on the **Subscription** pages. You can only dismiss it until the +following month after you submit your license usage data. + +## What happens when your license expires + +Fifteen days before the license expires, a notification banner with the upcoming expiration +date displays to GitLab administrators. + +When your license expires, GitLab locks features, like Git pushes +and issue creation. Your instance becomes read-only and +an expiration message displays to all administrators. You have a 14-day grace period +before this occurs. + +To resume functionality, [renew your subscription](../subscriptions/self_managed/index.md#renew-subscription-manually). + +If the license has been expired for more than 30 days, you must purchase a [new subscription](../subscriptions/self_managed/index.md) to resume functionality. + +To go back to Free features, [delete all expired licenses](#remove-a-license). + +## Remove a license + +To remove a license from a self-managed instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Subscription**. +1. Select **Remove license**. + +Repeat these steps to remove all licenses, including those applied in the past. + +## View license details and history + +To view your license details: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Subscription**. + +You can add and view more than one license, but only the latest license in +the current date range is the active license. + +When you add a future-dated license, it doesn't take effect until its applicable date. +You can view all active subscriptions in the **Subscription history** table. + +You can also [export](../subscriptions/self_managed/index.md) your license usage information to a CSV file. + +NOTE: +In GitLab 13.6 and earlier, a banner about an expiring license may continue to display +when you add a new license. This happens when the start date of the new license +is in the future and the expiring one is still active. +The banner disappears after the new license becomes active. + +## License commands in the Rails console + +The following commands can be run in the [Rails console](../administration/operations/rails_console.md#starting-a-rails-console-session). + +WARNING: +Any command that changes data directly could be damaging if not run correctly, or under the right conditions. +We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case. + +### See current license information + +```ruby +# License information (name, company, email address) +License.current.licensee + +# Plan: +License.current.plan + +# Uploaded: +License.current.created_at + +# Started: +License.current.starts_at + +# Expires at: +License.current.expires_at + +# Is this a trial license? +License.current.trial? + +# License ID for lookup on CustomersDot +License.current.license_id + +# License data in Base64-encoded ASCII format +License.current.data + +# Confirm the current billable seat count excluding guest users. This is useful for customers who use an Ultimate subscription tier where Guest seats are not counted. +User.active.without_bots.excluding_guests.count + +``` + +#### Interaction with licenses that start in the future + +```ruby +# Future license data follows the same format as current license data it just uses a different modifier for the License prefix +License.future_dated +``` + +### Check if a project feature is available on the instance + +Features listed in [`features.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/gitlab_subscriptions/features.rb). + +```ruby +License.current.feature_available?(:jira_dev_panel_integration) +``` + +#### Check if a project feature is available in a project + +Features listed in [`features.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/gitlab_subscriptions/features.rb). + +```ruby +p = Project.find_by_full_path('<group>/<project>') +p.feature_available?(:jira_dev_panel_integration) +``` + +### Add a license through the console + +#### Using a `key` variable + +```ruby +key = "<key>" +license = License.new(data: key) +license.save +License.current # check to make sure it applied +``` + +#### Using a license file + +```ruby +license_file = File.open("/tmp/Gitlab.license") + +key = license_file.read.gsub("\r\n", "\n").gsub(/\n+$/, '') + "\n" + +license = License.new(data: key) +license.save +License.current # check to make sure it applied +``` + +These snippets can be saved to a file and executed [using the Rails Runner](operations/rails_console.md#using-the-rails-runner) so the +license can be applied via shell automation scripts. + +This is needed for example in a known edge-case with +[expired license and multiple LDAP servers](../administration/auth/ldap/ldap-troubleshooting.md#expired-license-causes-errors-with-multiple-ldap-servers). + +### Remove licenses + +To clean up the [License History table](../administration/license_file.md#view-license-details-and-history): + +```ruby +TYPE = :trial? +# or :expired? + +License.select(&TYPE).each(&:destroy!) + +# or even License.all.each(&:destroy!) +``` + +## Troubleshooting + +### No Subscription area in the Admin Area + +You cannot add your license because there is no **Subscription** area. +This issue might occur if: + +- You're running GitLab Community Edition. Before you add your license, you + must [upgrade to Enterprise Edition](../update/index.md#community-to-enterprise-edition). +- You're using GitLab.com. You cannot add a self-managed license to GitLab.com. + To use paid features on GitLab.com, [purchase a separate subscription](../subscriptions/gitlab_com/index.md). + +### Users exceed license limit upon renewal + +GitLab displays a message prompting you to purchase +additional users. This issue occurs if you add a license that does not have enough +users to cover the number of users in your instance. + +To fix this issue, purchase additional seats to cover those users. +For more information, read the [licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). + +In GitLab 14.2 and later, for instances that use a license file, the following +rules apply: + +- If the users over license are less than or equal to 10% of the users in the license + file, the license is applied and you pay the overage in the next renewal. +- If the users over license are more than 10% of the users in the license file, + you cannot apply the license without purchasing more users. + +For example, if you purchase a license for 100 users, you can have 110 users when you add +your license. However, if you have 111 users, you must purchase more users before you can add +the license. + +### `Start GitLab Ultimate trial` still displays after adding license + +To fix this issue, restart [Puma or your entire GitLab instance](../administration/restart_gitlab.md). diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md index e43fe851aa2..a862fd46a3f 100644 --- a/doc/administration/load_balancer.md +++ b/doc/administration/load_balancer.md @@ -113,7 +113,7 @@ Configure DNS for an alternate SSH hostname such as `altssh.gitlab.example.com`. ## Readiness check -It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../user/admin_area/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma doesn't accept requests. +It is strongly recommend that multi-node deployments configure load balancers to use the [readiness check](../administration/monitoring/health_check.md#readiness) to ensure a node is ready to accept traffic, before routing traffic to it. This is especially important when utilizing Puma, as there is a brief period during a restart where Puma doesn't accept requests. WARNING: Using the `all=1` parameter with the readiness check in GitLab versions 15.4 to 15.8 may cause [increased Praefect memory usage](https://gitlab.com/gitlab-org/gitaly/-/issues/4751) and lead to memory errors. diff --git a/doc/administration/logs/index.md b/doc/administration/logs/index.md index 8dcb25e22df..449f33fbbef 100644 --- a/doc/administration/logs/index.md +++ b/doc/administration/logs/index.md @@ -754,8 +754,8 @@ This file is located at: This log records: -- Requests over the [Rate Limit](../../user/admin_area/settings/rate_limits_on_raw_endpoints.md) on raw endpoints. -- [Protected paths](../../user/admin_area/settings/protected_paths.md) abusive requests. +- Requests over the [Rate Limit](../settings/rate_limits_on_raw_endpoints.md) on raw endpoints. +- [Protected paths](../settings/protected_paths.md) abusive requests. - In GitLab versions [12.3](https://gitlab.com/gitlab-org/gitlab/-/issues/29239) and later, user ID and username, if available. @@ -1129,7 +1129,7 @@ GitLab also tracks [Prometheus metrics for Praefect](../gitaly/monitoring.md#mon For Omnibus installations, the backup log is located at `/var/log/gitlab/gitlab-rails/backup_json.log`. -This log is populated when a [GitLab backup is created](../../raketasks/backup_restore.md). You can use this log to understand how the backup process performed. +This log is populated when a [GitLab backup is created](../../administration/backup_restore/index.md). You can use this log to understand how the backup process performed. ## Performance bar stats diff --git a/doc/administration/maintenance_mode/index.md b/doc/administration/maintenance_mode/index.md index 3bbebe7ecce..336067d1891 100644 --- a/doc/administration/maintenance_mode/index.md +++ b/doc/administration/maintenance_mode/index.md @@ -128,8 +128,12 @@ For most JSON requests, `POST`, `PUT`, `PATCH`, and `DELETE` are blocked, and th ### GraphQL API +> The `GeoRegistriesUpdate` mutation addition in the allowlist was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124259) in GitLab 16.2. + `POST /api/graphql` requests are allowed but mutations are blocked with the error message `You cannot perform write operations on a read-only instance`. +The only mutation that is allowed is the `GeoRegistriesUpdate` which is used to resync and reverify registries. + ### Continuous Integration - No new jobs or pipelines start, scheduled or otherwise. @@ -194,7 +198,8 @@ When primary is in Maintenance Mode, secondary also automatically goes into Main It is important that you do not disable replication before enabling Maintenance Mode. -Replication and verification continues to work but proxied Git pushes to primary do not work. +Replication, verification and manual actions to resync and reverify registries through the Admin UI +continue to work, but proxied Git pushes to primary don't. ### Secure features diff --git a/doc/administration/merge_requests_approvals.md b/doc/administration/merge_requests_approvals.md new file mode 100644 index 00000000000..6cd0edf22eb --- /dev/null +++ b/doc/administration/merge_requests_approvals.md @@ -0,0 +1,43 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference, concepts +--- + +# Merge request approvals **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39060) in GitLab 12.8. + +Merge request approval rules prevent users from overriding certain settings on the project level. +When enabled at the instance level, these settings [cascade](../user/project/merge_requests/approvals/settings.md#settings-cascading) +and can no longer be changed: + +- In projects. +- In groups. Cascading to groups was [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) + in GitLab 14.5. + +To enable merge request approval settings for an instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Push Rules**. +1. Expand **Merge request approvals**. +1. Choose the required options. +1. Select **Save changes**. + +## Available rules + +Merge request approval settings that can be set at an instance level are: + +- **Prevent approval by author**. Prevents project maintainers from allowing request authors to + merge their own merge requests. +- **Prevent approvals by users who add commits**. Prevents project maintainers from allowing users + to approve merge requests if they have submitted any commits to the source branch. +- **Prevent editing approval rules in projects and merge requests**. Prevents users from modifying + the approvers list in project settings or in individual merge requests. + +See also the following, which are affected by instance-level rules: + +- [Project merge request approval rules](../user/project/merge_requests/approvals/index.md). +- [Group merge request approval settings](../user/group/manage.md#group-merge-request-approval-settings) available in GitLab 13.9 and later. diff --git a/doc/administration/moderate_users.md b/doc/administration/moderate_users.md new file mode 100644 index 00000000000..42f1f26586f --- /dev/null +++ b/doc/administration/moderate_users.md @@ -0,0 +1,393 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Moderate users (administration) **(FREE SELF)** + +This is the administration documentation. For information about moderating users at the group level, see the [group-level documentation](../user/group/moderate_users.md). + +GitLab administrators can moderate user access by approving, blocking, banning, or deactivating +users. + +## Users pending approval + +A user in _pending approval_ state requires action by an administrator. A user sign up can be in a +pending approval state because an administrator has enabled any of the following options: + +- [Require administrator approval for new sign-ups](../administration/settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) setting. +- [User cap](../administration/settings/sign_up_restrictions.md#user-cap). +- [Block auto-created users (OmniAuth)](../integration/omniauth.md#configure-common-settings) +- [Block auto-created users (LDAP)](../administration/auth/ldap/index.md#basic-configuration-settings) + +When a user registers for an account while this setting is enabled: + +- The user is placed in a **Pending approval** state. +- The user sees a message telling them their account is awaiting approval by an administrator. + +A user pending approval: + +- Is functionally identical to a [blocked](#block-a-user) user. +- Cannot sign in. +- Cannot access Git repositories or the GitLab API. +- Does not receive any notifications from GitLab. +- Does not consume a [seat](../subscriptions/self_managed/index.md#billable-users). + +An administrator must [approve their sign up](#approve-or-reject-a-user-sign-up) to allow them to +sign in. + +### View user sign ups pending approval + +To view user sign ups pending approval: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Pending approval** tab. + +### Approve or reject a user sign up + +A user sign up pending approval can be approved or rejected from the Admin Area. + +To approve or reject a user sign up: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Pending approval** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Approve** or **Reject**. + +Approving a user: + +- Activates their account. +- Changes the user's state to active. +- Consumes a subscription [seat](../subscriptions/self_managed/index.md#billable-users). + +## Block and unblock users + +GitLab administrators can block and unblock users. + +### Block a user + +To completely prevent access of a user to the GitLab instance, +administrators can choose to block the user. + +Users can be blocked [via an abuse report](../administration/review_abuse_reports.md#blocking-users), +by removing them in LDAP, or directly from the Admin Area. To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Block**. + +A blocked user: + +- Cannot sign in. +- Cannot access Git repositories or the API. +- Does not receive any notifications from GitLab. +- Cannot use [slash commands](../user/project/integrations/gitlab_slack_application.md#slash-commands). +- Does not consume a [seat](../subscriptions/self_managed/index.md#billable-users). + +Personal projects, and group and user history of the blocked user are left intact. + +NOTE: +Users can also be blocked using the [GitLab API](../api/users.md#block-user). + +### Unblock a user + +A blocked user can be unblocked from the Admin Area. To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Blocked** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Unblock**. + +The user's state is set to active and they consume a +[seat](../subscriptions/self_managed/index.md#billable-users). + +NOTE: +Users can also be unblocked using the [GitLab API](../api/users.md#unblock-user). + +The unblock option may be unavailable for LDAP users. To enable the unblock option, +the LDAP identity first needs to be deleted: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Blocked** tab. +1. Select a user. +1. Select the **Identities** tab. +1. Find the LDAP provider and select **Delete**. + +## Activate and deactivate users + +GitLab administrators can deactivate and activate users. + +### Deactivate a user + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22257) in GitLab 12.4. + +To temporarily prevent access by a GitLab user that has no recent activity, +administrators can choose to deactivate the user. + +Deactivating a user is functionally identical to [blocking a user](#block-and-unblock-users), +with the following differences: + +- It does not prohibit the user from logging back in via the UI. +- Once a deactivated user logs back into the GitLab UI, their account is set to active. + +A deactivated user: + +- Cannot access Git repositories or the API. +- Does not receive any notifications from GitLab. +- Cannot use [slash commands](../user/project/integrations/gitlab_slack_application.md#slash-commands). +- Does not consume a [seat](../subscriptions/self_managed/index.md#billable-users). + +Personal projects, and group and user history of the deactivated user are left intact. + +NOTE: +Users are notified about account deactivation if +[user deactivation emails](../administration/settings/email.md#user-deactivation-emails) are enabled. + +A user can be deactivated from the Admin Area. To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Deactivate**. + +For the deactivation option to be visible to an administrator, the user: + +- Must have a state of active. +- Must be [dormant](#automatically-deactivate-dormant-users). + +NOTE: +Users can also be deactivated using the [GitLab API](../api/users.md#deactivate-user). + +### Automatically deactivate dormant users + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320875) in GitLab 14.0. +> - Exclusion of GitLab generate bots [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340346) in GitLab 14.5 +> - Customizable time period [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336747) in GitLab 15.4 +> - The lower limit for inactive period set to 90 days [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/100793) in GitLab 15.5 + +Administrators can enable automatic deactivation of users who either: + +- Were created more than a week ago and have not signed in. +- Have no activity for a specified period of time (default and minimum is 90 days). + +To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. +1. Under **Dormant users**, check **Deactivate dormant users after a period of inactivity**. +1. Under **Days of inactivity before deactivation**, enter the number of days before deactivation. Minimum value is 90 days. +1. Select **Save changes**. + +When this feature is enabled, GitLab runs a job once a day to deactivate the dormant users. + +A maximum of 100,000 users can be deactivated per day. + +NOTE: +GitLab generated bots are excluded from the automatic deactivation of dormant users. + +### Automatically delete unconfirmed users **(PREMIUM SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352514) in GitLab 16.1 [with a flag](../administration/feature_flags.md) named `delete_unconfirmed_users_setting`. Disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124982) in GitLab 16.2. + +Prerequisites: + +- You must be an administrator. + +You can enable automatic deletion of users who both: + +- Never confirmed their email address. +- Signed up for GitLab more than a specified number of days in the past. + +You can configure these settings using either the [Settings API](../api/settings.md) or in a Rails console: + +```ruby + Gitlab::CurrentSettings.update(delete_unconfirmed_users: true) + Gitlab::CurrentSettings.update(unconfirmed_users_delete_after_days: 365) +``` + +When the `delete_unconfirmed_users` setting is enabled, GitLab runs a job once an hour to delete the unconfirmed users. +The job only deletes users who signed up more than `unconfirmed_users_delete_after_days` days in the past. + +This job only runs when the `email_confirmation_setting` is set to `soft` or `hard`. + +A maximum of 240,000 users can be deleted per day. + +### Activate a user + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22257) in GitLab 12.4. + +A deactivated user can be activated from the Admin Area. + +To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Deactivated** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Activate**. + +The user's state is set to active and they consume a +[seat](../subscriptions/self_managed/index.md#billable-users). + +NOTE: +A deactivated user can also activate their account themselves by logging back in via the UI. +Users can also be activated using the [GitLab API](../api/users.md#activate-user). + +## Ban and unban users + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327353) in GitLab 14.2 [with a flag](../administration/feature_flags.md) named `ban_user_feature_flag`. Disabled by default. +> - Ban and unban users [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/327353) in GitLab 14.8. Feature flag `ban_user_feature_flag` removed. +> - Hiding merge requests of banned users [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107836) in GitLab 15.8 [with a flag](../administration/feature_flags.md) named `hide_merge_requests_from_banned_users`. Disabled by default. +> - Hiding comments of banned users [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112973) in GitLab 15.11 [with a flag](../administration/feature_flags.md) named `hidden_notes`. Disabled by default. +> - Hiding projects of banned users [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121488) in GitLab 16.2 [with a flag](../administration/feature_flags.md) named `hide_projects_of_banned_users`. Disabled by default. + +GitLab administrators can ban and unban users. Banned users are blocked, and their projects, issues, merge requests, and comments are hidden. + +### Ban a user + +To block a user and hide their contributions, administrators can ban the user. + +Users can be banned using the Admin Area. To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Ban user**. + +The banned user does not consume a [seat](../subscriptions/self_managed/index.md#billable-users). + +### Unban a user + +A banned user can be unbanned using the Admin Area. To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Banned** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Unban user**. + +The user's state is set to active and they consume a +[seat](../subscriptions/self_managed/index.md#billable-users). + +### Delete a user + +Use the Admin Area to delete users. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Banned** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Delete user**. +1. Type the username. +1. Select **Delete user**. + +NOTE: +You can only delete a user if there are inherited or direct owners of a group. You cannot delete a user if they are the only group owner. + +You can also delete a user and their contributions, such as merge requests, issues, and groups of which they are the only group owner. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Banned** tab. +1. Optional. Select a user. +1. Select the **{settings}** **User administration** dropdown list. +1. Select **Delete user and contributions**. +1. Type the username. +1. Select **Delete user and contributions**. + +NOTE: +Before 15.1, additionally groups of which deleted user were the only owner among direct members were deleted. + +## Troubleshooting + +When moderating users, you may need to perform bulk actions on them based on certain conditions. The following rails console scripts show some examples of this. You may [start a rails console session](../administration/operations/rails_console.md#starting-a-rails-console-session) and use scripts similar to the following: + +### Deactivate users that have no recent activity + +Administrators can deactivate users that have no recent activity. + +WARNING: +Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. + +```ruby +days_inactive = 90 +inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago) + +inactive_users.each do |user| + puts "user '#{user.username}': #{user.last_activity_on}" + user.deactivate! +end +``` + +### Block users that have no recent activity + +Administrators can block users that have no recent activity. + +WARNING: +Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. + +```ruby +days_inactive = 90 +inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago) + +inactive_users.each do |user| + puts "user '#{user.username}': #{user.last_activity_on}" + user.block! +end +``` + +### Block or delete users that have no projects or groups + +Administrators can block or delete users that have no projects or groups. + +WARNING: +Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. + +```ruby +users = User.where('id NOT IN (select distinct(user_id) from project_authorizations)') + +# How many users are removed? +users.count + +# If that count looks sane: + +# You can either block the users: +users.each { |user| user.blocked? ? nil : user.block! } + +# Or you can delete them: + # need 'current user' (your user) for auditing purposes +current_user = User.find_by(username: '<your username>') + +users.each do |user| + DeleteUserWorker.perform_async(current_user.id, user.id) +end +``` diff --git a/doc/administration/monitoring/health_check.md b/doc/administration/monitoring/health_check.md new file mode 100644 index 00000000000..4dbbdf6f3c9 --- /dev/null +++ b/doc/administration/monitoring/health_check.md @@ -0,0 +1,146 @@ +--- +stage: Monitor +group: Respond +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Health Check **(FREE SELF)** + +GitLab provides liveness and readiness probes to indicate service health and +reachability to required services. These probes report on the status of the +database connection, Redis connection, and access to the file system. These +endpoints [can be provided to schedulers like Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) to hold +traffic until the system is ready or restart the container as needed. + +## IP allowlist + +To access monitoring resources, the requesting client IP needs to be included in the allowlist. +For details, see [how to add IPs to the allowlist for the monitoring endpoints](../../administration/monitoring/ip_allowlist.md). + +## Using the endpoints locally + +With default allowlist settings, the probes can be accessed from localhost using the following URLs: + +```plaintext +GET http://localhost/-/health +``` + +```plaintext +GET http://localhost/-/readiness +``` + +```plaintext +GET http://localhost/-/liveness +``` + +## Health + +Checks whether the application server is running. +It does not verify the database or other services +are running. This endpoint circumvents Rails Controllers +and is implemented as additional middleware `BasicHealthCheck` +very early into the request processing lifecycle. + +```plaintext +GET /-/health +``` + +Example request: + +```shell +curl "https://gitlab.example.com/-/health" +``` + +Example response: + +```plaintext +GitLab OK +``` + +## Readiness + +The readiness probe checks whether the GitLab instance is ready +to accept traffic via Rails Controllers. The check by default +does validate only instance-checks. + +If the `all=1` parameter is specified, the check also validates +the dependent services (Database, Redis, Gitaly etc.) +and gives a status for each. + +```plaintext +GET /-/readiness +GET /-/readiness?all=1 +``` + +Example request: + +```shell +curl "https://gitlab.example.com/-/readiness" +``` + +Example response: + +```json +{ + "master_check":[{ + "status":"failed", + "message": "unexpected Master check result: false" + }], + ... +} +``` + +On failure, the endpoint returns a `503` HTTP status code. + +This check is being exempt from Rack Attack. + +## Liveness + +WARNING: +In GitLab [12.4](https://about.gitlab.com/upcoming-releases/) +the response body of the Liveness check was changed +to match the example below. + +Checks whether the application server is running. +This probe is used to know if Rails Controllers +are not deadlocked due to a multi-threading. + +```plaintext +GET /-/liveness +``` + +Example request: + +```shell +curl "https://gitlab.example.com/-/liveness" +``` + +Example response: + +On success, the endpoint returns a `200` HTTP status code, and a response like below. + +```json +{ + "status": "ok" +} +``` + +On failure, the endpoint returns a `503` HTTP status code. + +This check is being exempt from Rack Attack. + +## Sidekiq + +Learn how to configure the [Sidekiq health checks](../../administration/sidekiq/sidekiq_health_check.md). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index 1b23d6b7f49..622a772a638 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -15,7 +15,7 @@ Explore our features to monitor your GitLab instance: products. - [GitHub imports](github_imports.md): Monitor the health and progress of the GitHub importer with various Prometheus metrics. -- [Monitoring uptime](../../user/admin_area/monitoring/health_check.md): Check the +- [Monitoring uptime](health_check.md): Check the server status using the health check endpoint. - [IP allowlists](ip_allowlist.md): Configure GitLab for monitoring endpoints that provide health check information when probed. diff --git a/doc/administration/monitoring/ip_allowlist.md b/doc/administration/monitoring/ip_allowlist.md index 72640cd6218..364c1b27d33 100644 --- a/doc/administration/monitoring/ip_allowlist.md +++ b/doc/administration/monitoring/ip_allowlist.md @@ -6,13 +6,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w # IP whitelist **(FREE SELF)** -GitLab provides some [monitoring endpoints](../../user/admin_area/monitoring/health_check.md) +GitLab provides some [monitoring endpoints](health_check.md) that provide health check information when probed. To control access to those endpoints via IP whitelisting, you can add single hosts or use IP ranges: -**Omnibus** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Open `/etc/gitlab/gitlab.rb` and add or uncomment the following: @@ -22,9 +24,7 @@ hosts or use IP ranges: 1. Save the file and [reconfigure](../restart_gitlab.md#reconfigure-a-linux-package-installation) GitLab for the changes to take effect. ---- - -**Helm chart** +:::TabTitle Helm chart (Kubernetes) You can set the required IPs under the `gitlab.webservice.monitoring.ipWhitelist` key. For example: @@ -37,9 +37,7 @@ gitlab: - 0.0.0.0/0 # Default ``` ---- - -**Source** +:::TabTitle Self-compiled (source) 1. Edit `config/gitlab.yml`: @@ -52,3 +50,5 @@ gitlab: ``` 1. Save the file and [restart](../restart_gitlab.md#installations-from-source) GitLab for the changes to take effect. + +::EndTabs diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md index 0d2037f3a92..a1def4764f6 100644 --- a/doc/administration/monitoring/performance/gitlab_configuration.md +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -16,8 +16,8 @@ settings: 1. Add the necessary configuration changes. 1. Restart all GitLab for the changes to take effect: - - For Omnibus GitLab installations: `sudo gitlab-ctl restart` - - For installations from source: `sudo service gitlab restart` + - For Linux package installations: `sudo gitlab-ctl restart` + - For self-compiled installations: `sudo service gitlab restart` NOTE: Removed [in GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30786). Use the diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index b448ac461c8..8b3720ca8a9 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -18,10 +18,10 @@ and Grafana allows you to query the data to display graphs. ## Deprecation of bundled Grafana -Bundled Grafana was an optional Omnibus GitLab service that provided a user interface to GitLab metrics. +Bundled Grafana was an optional service for Linux package installations that provided a user interface to GitLab metrics. -The version of Grafana that is bundled with Omnibus GitLab is no longer supported. If you're using the bundled Grafana, you -should switch to a newer version from [Grafana Labs](https://grafana.com/grafana/). +The version of Grafana that is bundled with Linux package installations is no longer supported. If you're using the +bundled Grafana, you should switch to a newer version from [Grafana Labs](https://grafana.com/grafana/). ### Switch to new Grafana instance @@ -34,7 +34,8 @@ To switch away from bundled Grafana to a newer version of Grafana from Grafana L ### Temporary workaround -In GitLab versions 16.0 to 16.2, you can still force Omnibus GitLab to enable and configure Grafana by setting the following: +In GitLab versions 16.0 to 16.2, you can still force Linux package installations to enable and configure Grafana by +setting the following: - `grafana['enable'] = true`. - `grafana['enable_deprecated_service'] = true`. @@ -92,9 +93,9 @@ GitLab sidebar: and expand **Metrics - Grafana**. 1. Select the **Add a link to Grafana** checkbox. 1. Configure the **Grafana URL**: - - *If Grafana is enabled through Omnibus GitLab and on the same server,* + - If Grafana is enabled through a Linux package installation and on the same server, leave **Grafana URL** unchanged. It should be `/-/grafana`. - - *Otherwise,* enter the full URL of the Grafana instance. + - Otherwise, enter the full URL of the Grafana instance. 1. Select **Save changes**. GitLab displays your link in the **Main menu > Admin > Monitoring > Metrics Dashboard**. diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 3fdd4c24177..8afec54dab2 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -22,7 +22,7 @@ From left to right, the performance bar displays: - **Current Host**: the current host serving the page. - **Database queries**: the time taken (in milliseconds) and the total number of database queries, displayed in the format `00ms / 00 (00 cached) pg`. Select to display - a modal window with more details. You can use this to see the following + a dialog with more details. You can use this to see the following details for each query: - **In a transaction**: shows up below the query if it was executed in the context of a transaction @@ -38,8 +38,7 @@ From left to right, the performance bar displays: [Gitaly](../../gitaly/index.md) calls. Select to display a modal window with more details. - **Rugged calls**: the time taken (in milliseconds) and the total number of - [Rugged](../../nfs.md#improving-nfs-performance-with-gitlab) calls. - Select to display a modal window with more details. + Rugged calls. Select to display a modal window with more details. - **Redis calls**: the time taken (in milliseconds) and the total number of Redis calls. Select to display a modal window with more details. - **Elasticsearch calls**: the time taken (in milliseconds) and the total number of diff --git a/doc/administration/monitoring/prometheus/gitlab_exporter.md b/doc/administration/monitoring/prometheus/gitlab_exporter.md index 0bd13fe5a87..22b73378cab 100644 --- a/doc/administration/monitoring/prometheus/gitlab_exporter.md +++ b/doc/administration/monitoring/prometheus/gitlab_exporter.md @@ -9,12 +9,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Renamed from `GitLab monitor exporter` to `GitLab exporter` in [GitLab 12.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16511). The [GitLab exporter](https://gitlab.com/gitlab-org/gitlab-exporter) enables you to -measure various GitLab metrics pulled from Redis and the database in Omnibus GitLab +measure various GitLab metrics pulled from Redis and the database in Linux package instances. For installations from source you must install and configure it yourself. -To enable the GitLab exporter in an Omnibus GitLab instance: +To enable the GitLab exporter in a Linux package instance: 1. [Enable Prometheus](index.md#configuring-prometheus). 1. Edit `/etc/gitlab/gitlab.rb`. @@ -32,7 +32,7 @@ the GitLab exporter exposed at `localhost:9168`. ## Use a different Rack server -> - Introduced in [Omnibus GitLab 13.8](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4896). +> - Introduced in [GitLab 13.8](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4896). > - WEBrick is now the default Rack server instead of Puma. By default, the GitLab exporter runs on [WEBrick](https://github.com/ruby/webrick), a single-threaded Ruby web server. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 5d6827b79ee..713a1fb3b5d 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -23,8 +23,8 @@ GitLab monitors its own internal service metrics, and makes them available at th `/-/metrics` endpoint. Unlike other [Prometheus](https://prometheus.io) exporters, to access the metrics, the client IP address must be [explicitly allowed](../ip_allowlist.md). -These metrics are enabled and collected for [Omnibus GitLab](https://docs.gitlab.com/omnibus/) -and Chart installations. For source installations, these metrics must be enabled +These metrics are enabled and collected for [Linux package](https://docs.gitlab.com/omnibus/) +and Helm chart installations. For self-compiled installations, these metrics must be enabled manually and collected by a Prometheus server. For enabling and viewing metrics from Sidekiq nodes, see [Sidekiq metrics](#sidekiq-metrics). @@ -206,7 +206,7 @@ configuration option in `gitlab.yml`. These metrics are served from the | `sidekiq_jobs_dead_total` | Counter | 13.7 | Sidekiq dead jobs (jobs that have run out of retries) | `queue`, `boundary`, `external_dependencies`, `feature_category`, `urgency` | | `sidekiq_redis_requests_total` | Counter | 13.1 | Redis requests during a Sidekiq job execution | `queue`, `boundary`, `external_dependencies`, `feature_category`, `job_status`, `urgency` | | `sidekiq_elasticsearch_requests_total` | Counter | 13.1 | Elasticsearch requests during a Sidekiq job execution | `queue`, `boundary`, `external_dependencies`, `feature_category`, `job_status`, `urgency` | -| `sidekiq_jobs_deferred_total` | Counter | 16.1 | Number of jobs being deferred when `defer_sidekiq_jobs` feature flag is enabled | `worker` | +| `sidekiq_jobs_skipped_total` | Counter | 16.2 | Number of jobs being skipped (dropped or deferred) when `drop_sidekiq_jobs` feature flag is enabled or `run_sidekiq_jobs` feature flag is disabled | `worker`, `action` | | `sidekiq_running_jobs` | Gauge | 12.2 | Number of Sidekiq jobs running | `queue`, `boundary`, `external_dependencies`, `feature_category`, `urgency` | | `sidekiq_concurrency` | Gauge | 12.5 | Maximum number of Sidekiq jobs | | | `sidekiq_mem_total_bytes` | Gauge | 15.3 | Number of bytes allocated for both objects consuming an object slot and objects that required a malloc'| | @@ -371,6 +371,17 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_project_wiki_repositories_verification_total` | Gauge | 15.10 | Number of Project Wiki Repositories to attempt to verify on secondary | `url` | | `geo_project_wiki_repositories_verified` | Gauge | 15.10 | Number of Project Wiki Repositories successfully verified on secondary | `url` | | `geo_project_wiki_repositories_verification_failed` | Gauge | 15.10 | Number of Project Wiki Repositories that failed verification on secondary | `url` | +| `geo_project_repositories` | Gauge | 16.2 | Number of Project Repositories on primary | `url` | +| `geo_project_repositories_checksum_total` | Gauge | 16.2 | Number of Project Repositories to checksum on primary | `url` | +| `geo_project_repositories_checksummed` | Gauge | 16.2 | Number of Project Repositories that successfully calculated the checksum on primary | `url` | +| `geo_project_repositories_checksum_failed` | Gauge | 16.2 | Number of Project Repositories that failed to calculate the checksum on primary | `url` | +| `geo_project_repositories_synced` | Gauge | 16.2 | Number of syncable Project Repositories synced on secondary | `url` | +| `geo_project_repositories_failed` | Gauge | 16.2 | Number of syncable Project Repositories failed to sync on secondary | `url` | +| `geo_project_repositories_registry` | Gauge | 16.2 | Number of Project Repositories in the registry | `url` | +| `geo_project_repositories_verification_total` | Gauge | 16.2 | Number of Project Repositories to attempt to verify on secondary | `url` | +| `geo_project_repositories_verified` | Gauge | 16.2 | Number of Project Repositories successfully verified on secondary | `url` | +| `geo_project_repositories_verification_failed` | Gauge | 16.2 | Number of Project Repositories that failed verification on secondary | `url` | + | `gitlab_memwd_violations_total` | Counter | 15.9 | Total number of times a Sidekiq process violated a memory threshold | | | `gitlab_memwd_violations_handled_total` | Counter | 15.9 | Total number of times Sidekiq process memory violations were handled | | | `sidekiq_watchdog_running_jobs_total` | Counter | 15.9 | Current running jobs when RSS limit was reached | `worker_class` | @@ -460,7 +471,6 @@ Some basic Ruby runtime metrics are available: | `puma_pool_capacity` | Gauge | 12.0 | Number of requests the worker is capable of taking right now | | `puma_max_threads` | Gauge | 12.0 | Maximum number of worker threads | | `puma_idle_threads` | Gauge | 12.0 | Number of spawned threads which are not processing a request | -| `puma_killer_terminations_total` | Gauge | 12.0 | Number of workers terminated by PumaWorkerKiller | ## Redis metrics @@ -487,5 +497,5 @@ metrics can't function correctly. This directory's location is configured using environment variable `prometheus_multiproc_dir`. For best performance, create this directory in `tmpfs`. -If GitLab is installed using [Omnibus GitLab](https://docs.gitlab.com/omnibus/) +If GitLab is installed using the [Linux package](https://docs.gitlab.com/omnibus/) and `tmpfs` is available, then GitLab configures the metrics directory for you. diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 0e8315e528a..a9b393aab33 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -9,12 +9,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w [Prometheus](https://prometheus.io) is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products. -GitLab provides out-of-the-box monitoring with Prometheus, providing easy -access to high quality time-series monitoring of GitLab services. +GitLab provides out-of-the-box monitoring with Prometheus, providing access to high quality time-series monitoring of +GitLab services. -Prometheus and the various exporters listed in this page are bundled in the -Omnibus GitLab package. Check each exporter's documentation for the timeline -they got added. For installations from source you must install them +Prometheus and the various exporters listed in this page are bundled in Linux packages. Check each exporter's +documentation for the timeline they got added. For installations from source you must install them yourself. Over subsequent releases additional GitLab metrics are captured. Prometheus services are on by default. @@ -85,7 +84,7 @@ listens on: ### Adding custom scrape configurations -You can configure additional scrape targets for the Omnibus GitLab-bundled +You can configure additional scrape targets for the Linux package-bundled Prometheus by editing `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb` using the [Prometheus scrape target configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E) syntax. @@ -108,16 +107,16 @@ prometheus['scrape_configs'] = [ ] ``` -### Standalone Prometheus using Omnibus GitLab +### Standalone Prometheus using the Linux package -The Omnibus GitLab package can be used to configure a standalone Monitoring node running Prometheus and [Grafana](../performance/grafana_configuration.md). +The Linux package can be used to configure a standalone Monitoring node running Prometheus and [Grafana](../performance/grafana_configuration.md). -The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with Omnibus GitLab: +The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with the Linux +package: 1. SSH into the Monitoring node. -1. [Install](https://about.gitlab.com/install/) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page, but - do not follow the remaining steps. +1. [Install](https://about.gitlab.com/install/) the Linux package you want using **steps 1 and 2** from the GitLab + downloads page, but do not follow the remaining steps. 1. Make sure to collect the IP addresses or DNS records of the Consul server nodes, for the next step. 1. Edit `/etc/gitlab/gitlab.rb` and add the contents: @@ -388,8 +387,7 @@ Read more about the [GitLab Metrics](gitlab_metrics.md). ## Bundled software metrics -Many of the GitLab dependencies bundled in Omnibus GitLab are preconfigured to -export Prometheus metrics. +Many of the GitLab dependencies bundled in the Linux package are preconfigured to export Prometheus metrics. ### Node exporter @@ -435,22 +433,6 @@ The GitLab exporter allows you to measure various GitLab metrics, pulled from Re [Read more about the GitLab exporter](gitlab_exporter.md). -## Configuring Prometheus to monitor Kubernetes - -If your GitLab server is running within Kubernetes, Prometheus collects metrics from the Nodes and [annotated Pods](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config) in the cluster, including performance data on each container. This is particularly helpful if your CI/CD environments run in the same cluster, as you can use the [Prometheus project integration](../../../user/project/integrations/prometheus.md) to monitor them. - -To disable the monitoring of Kubernetes: - -1. Edit `/etc/gitlab/gitlab.rb`. -1. Add (or find and uncomment) the following line and set it to `false`: - - ```ruby - prometheus['monitor_kubernetes'] = false - ``` - -1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to - take effect. - ## Troubleshooting ### `/var/opt/gitlab/prometheus` consumes too much disk space diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index 3e3712c9645..0273c4b03b1 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -15,35 +15,10 @@ is recommended over NFS where possible, due to better performance. When eliminating the usage of NFS, there are [additional steps you need to take](object_storage.md#alternatives-to-file-system-storage) in addition to moving to Object Storage. -File system performance can impact overall GitLab performance, especially for -actions that read or write to Git repositories. For steps you can use to test -file system performance, see -[File System Performance Benchmarking](operations/filesystem_benchmarking.md). - -## Gitaly with NFS not supported - -Technical and engineering support for using NFS to store Git repository data is officially at end-of-life. No product -changes or troubleshooting is provided through engineering, security or paid support channels. - -If an issue is reproducible, or if it happens intermittently but regularly, GitLab Support can investigate providing the -issue can be reproduced without NFS. To reproduce without NFS, migrate the affected repositories to a different Gitaly -shard. For example, a Gitaly Cluster or a standalone Gitaly VM, backed with block storage. - -## Known kernel version incompatibilities - -RedHat Enterprise Linux (RHEL) and CentOS v7.7 and v7.8 ship with kernel -version `3.10.0-1127`, which [contains a bug](https://bugzilla.redhat.com/show_bug.cgi?id=1783554) that causes -[uploads to fail to copy over NFS](https://gitlab.com/gitlab-org/gitlab/-/issues/218999). The -following GitLab versions include a fix to work properly with that -kernel version: - -- [12.10.12](https://about.gitlab.com/releases/2020/06/25/gitlab-12-10-12-released/) -- [13.0.7](https://about.gitlab.com/releases/2020/06/25/gitlab-13-0-7-released/) -- [13.1.1](https://about.gitlab.com/releases/2020/06/24/gitlab-13-1-1-released/) -- 13.2 and up +NFS cannot be used for repository storage. -If you are using that kernel version, be sure to upgrade GitLab to avoid -errors. +For steps you can use to test file system performance, see +[File System Performance Benchmarking](operations/filesystem_benchmarking.md). ## Fast lookup of authorized SSH keys @@ -59,26 +34,6 @@ is moved to NFS. We are investigating the use of [fast lookup as the default](https://gitlab.com/groups/gitlab-org/-/epics/3104). -## Improving NFS performance with GitLab - -NFS performance with GitLab can in some cases be improved with -[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using [Rugged](https://github.com/libgit2/rugged). - -Depending on the GitLab version, GitLab [automatically detects](gitaly/index.md#automatic-detection) if Rugged can and should -be used per storage. - -If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. If you -previously enabled Rugged using the feature flag and you want to use automatic detection instead, you must unset -the feature flag: - -```shell -sudo gitlab-rake gitlab:features:unset_rugged -``` - -From GitLab 12.7, Rugged is only automatically enabled for use with Puma if the -[Puma thread count is set to `1`](../install/requirements.md#puma-settings). To use Rugged with a Puma thread count of -more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code). - ## NFS server Installing the `nfs-kernel-server` package allows you to share directories with @@ -293,7 +248,7 @@ NFS mount point is `/gitlab-nfs`. Then, add the following bind mounts in Using bind mounts requires you to manually make sure the data directories are empty before attempting a restore. Read more about the -[restore prerequisites](../raketasks/backup_restore.md). +[restore prerequisites](../administration/backup_restore/index.md). ### Multiple NFS mounts @@ -315,7 +270,7 @@ provide configuration for [UDP log shipping](https://docs.gitlab.com/omnibus/set Having multiple NFS mounts requires you to manually make sure the data directories are empty before attempting a restore. Read more about the -[restore prerequisites](../raketasks/backup_restore.md). +[restore prerequisites](../administration/backup_restore/index.md). ## Testing NFS @@ -361,33 +316,6 @@ sudo ufw allow from <client_ip_address> to any port nfs ## Known issues -### Upgrade to Gitaly Cluster or disable caching if experiencing data loss - -WARNING: -Engineering support for NFS for Git repositories -[is unavailable](../update/removals.md#nfs-as-git-repository-storage-is-no-longer-supported). - -Customers and users have reported data loss on high-traffic repositories when using NFS for Git repositories. -For example, we have seen: - -- [Inconsistent updates after a push](https://gitlab.com/gitlab-org/gitaly/-/issues/2589). -- `git ls-remote` [returning the wrong (or no branches)](https://gitlab.com/gitlab-org/gitaly/-/issues/3083) -causing Jenkins to intermittently re-run all pipelines for a repository. - -The problem may be partially mitigated by adjusting caching using the following NFS client mount options: - -| Setting | Description | -| ------- | ----------- | -| `lookupcache=positive` | Tells the NFS client to honor `positive` cache results but invalidates any `negative` cache results. Negative cache results cause problems with Git. Specifically, a `git push` can fail to register uniformly across all NFS clients. The negative cache causes the clients to 'remember' that the files did not exist previously. -| `actimeo=0` | Sets the time to zero that the NFS client caches files and directories before requesting fresh information from a server. | -| `noac` | Tells the NFS client not to cache file attributes and forces application writes to become synchronous so that local changes to a file become visible on the server immediately. | - -WARNING: -The `actimeo=0` and `noac` options both result in a significant reduction in performance, possibly leading to timeouts. -You may be able to avoid timeouts and data loss using `actimeo=0` and `lookupcache=positive` _without_ `noac`, however -we expect the performance reduction is still significant. Upgrade to -[Gitaly Cluster](gitaly/praefect.md) as soon as possible. - ### Avoid using cloud-based file systems GitLab strongly recommends against using cloud-based file systems such as: @@ -447,10 +375,3 @@ On Ubuntu 16.04, use: ```shell sudo perf trace --no-syscalls --event 'nfs4:*' -p $(pgrep -fd ',' puma) ``` - -### Warnings `garbage found: .../repositories/@hashed/...git/objects/pack/.nfs...` in Gitaly logs - -If you find any warnings like `garbage found: .../repositories/@hashed/...git/objects/pack/.nfs...` in [Gitaly logs](logs/index.md#gitaly-logs), -this problem occurs if `lookupcache=positive` is not set, which we recommend as an -[NFS mount option](#mount-options). -See [Gitaly issue #3175](https://gitlab.com/gitlab-org/gitaly/-/issues/3175) for more details. diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index f2b966bd180..2bf3ef0275c 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -68,7 +68,7 @@ automatically. Thus, only the following providers can be used: The consolidated form configuration can't be used for backups or Mattermost. Backups can be configured with -[server side encryption](../raketasks/backup_gitlab.md#s3-encrypted-buckets) +[server side encryption](../administration/backup_restore/backup_gitlab.md#s3-encrypted-buckets) separately. See the [table for a complete list](#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form) of supported object storage types. @@ -163,7 +163,7 @@ supported by consolidated form, refer to the following guides: | Object storage type | Supported by consolidated form? | |---------------------|------------------------------------------| | [Project-level Secure Files](secure_files.md#using-object-storage) | **{dotted-circle}** No | -| [Backups](../raketasks/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | +| [Backups](../administration/backup_restore/backup_gitlab.md#upload-backups-to-a-remote-cloud-storage) | **{dotted-circle}** No | | [Container Registry](packages/container_registry.md#use-object-storage) (optional feature) | **{dotted-circle}** No | | [Mattermost](https://docs.mattermost.com/configure/file-storage-configuration-settings.html)| **{dotted-circle}** No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | **{dotted-circle}** No | @@ -174,7 +174,7 @@ supported by consolidated form, refer to the following guides: | [Packages](packages/index.md#use-object-storage) (optional feature) | **{check-circle}** Yes | | [Dependency Proxy](packages/dependency_proxy.md#using-object-storage) (optional feature) | **{check-circle}** Yes | | [Terraform state files](terraform_state.md#using-object-storage) | **{check-circle}** Yes | -| [Pages content](pages/index.md#using-object-storage) | **{check-circle}** Yes | +| [Pages content](pages/index.md#object-storage-settings) | **{check-circle}** Yes | ## Configure the connection settings @@ -825,7 +825,7 @@ See the following additional guides: ### Objects are not included in GitLab backups -As noted in [the backup documentation](../raketasks/backup_restore.md), +As noted in [the backup documentation](../administration/backup_restore/index.md), objects are not included in GitLab backups. You can enable backups with your object storage provider instead. @@ -848,7 +848,7 @@ Helm-based installs require separate buckets to ### S3 API compatibility issues -Not all S3 providers [are fully compatible](../raketasks/backup_gitlab.md#other-s3-providers) +Not all S3 providers [are fully compatible](../administration/backup_restore/backup_gitlab.md#other-s3-providers) with the Fog library that GitLab uses. Symptoms include an error in `production.log`: ```plaintext diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index d54d286c19d..8382f3aa8b5 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -27,8 +27,8 @@ lookup of authorized SSH keys. ## Fast lookup is required for Geo **(PREMIUM)** -Unlike [Cloud Native GitLab](https://docs.gitlab.com/charts/), Omnibus GitLab by default -manages an `authorized_keys` file that is located in the +Unlike [Cloud Native GitLab](https://docs.gitlab.com/charts/), by default Linux package installations +manage an `authorized_keys` file that is located in the `git` user's home directory. For most installations, this file is located under `/var/opt/gitlab/.ssh/authorized_keys`, but you can use the following command to locate the `authorized_keys` on your system: @@ -74,7 +74,7 @@ able to accept a fingerprint. Check the version of OpenSSH on your server with ` Add the following to your `sshd_config` file. This file is usually located at `/etc/ssh/sshd_config`, but it is at `/assets/sshd_config` if you're using -Omnibus Docker: +Docker from a Linux package installation: ```plaintext Match User git # Apply the AuthorizedKeysCommands to the git user only @@ -146,7 +146,8 @@ This overview is brief. Refer to the above instructions for more context. 1. On the left sidebar, select **Settings > Network**. 1. Expand **Performance optimization**. 1. Select the **Use authorized_keys file to authenticate SSH keys** checkbox. -1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. +1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Docker + from a Linux package installation. 1. Reload `sshd`: `sudo service sshd reload`. ## SELinux support and limitations diff --git a/doc/administration/operations/filesystem_benchmarking.md b/doc/administration/operations/filesystem_benchmarking.md index cd4ab1a9cf8..bd37bd4b1a8 100644 --- a/doc/administration/operations/filesystem_benchmarking.md +++ b/doc/administration/operations/filesystem_benchmarking.md @@ -11,7 +11,7 @@ especially for actions that read or write to Git repositories. This information helps benchmark file system performance against known good and bad real-world systems. -Normally when talking about file system performance the biggest concern is +When talking about file system performance the biggest concern is with Network File Systems (NFS). However, even some local disks can have slow I/O. The information on this page can be used for either scenario. diff --git a/doc/administration/operations/gitlab_sshd.md b/doc/administration/operations/gitlab_sshd.md index 5c4af32fc3d..2707c8f08a0 100644 --- a/doc/administration/operations/gitlab_sshd.md +++ b/doc/administration/operations/gitlab_sshd.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # `gitlab-sshd` **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5 as an Experiment for self-managed customers. -> - Ready for production use with [Cloud Native GitLab in GitLab 15.1](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/2540) and [Omnibus GitLab in GitLab 15.9](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5937). +> - Ready for production use with [Cloud Native GitLab in GitLab 15.1](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/2540) and [Linux packages in GitLab 15.9](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5937). `gitlab-sshd` is [a standalone SSH server](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/sshd) written in Go. It is provided as a part of the `gitlab-shell` package. It has a lower memory @@ -47,7 +47,7 @@ The following instructions enable `gitlab-sshd` on a different port than OpenSSH gitlab_sshd['listen_address'] = '[::]:2222' # Adjust the port accordingly ``` -1. Optional. By default, Omnibus GitLab generates SSH host keys for `gitlab-sshd` if +1. Optional. By default, Linux package installations generate SSH host keys for `gitlab-sshd` if they do not exist in `/var/opt/gitlab/gitlab-sshd`. If you wish to disable this automatic generation, add this line: ```ruby diff --git a/doc/administration/operations/index.md b/doc/administration/operations/index.md index 867ef3236ee..be90b0a073f 100644 --- a/doc/administration/operations/index.md +++ b/doc/administration/operations/index.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Keep your GitLab instance up and running. - [Housekeeping](../../administration/housekeeping.md) -- [Activate GitLab EE with license](../../user/admin_area/license_file.md) +- [Activate GitLab EE with license](../../administration/license_file.md) - [Fast SSH key lookup](../../administration/operations/fast_ssh_key_lookup.md) - [File system benchmarking](../../administration/operations/filesystem_benchmarking.md) - [`gitlab-sshd`](../../administration/operations/gitlab_sshd.md) @@ -17,7 +17,7 @@ Keep your GitLab instance up and running. - [Use SSH certificates](../../administration/operations/ssh_certificates.md) - [Enable encrypted configuration](../../administration/encrypted_configuration.md) - [Rake tasks](../../raketasks/index.md) -- [Backup and restore](../../raketasks/backup_restore.md) +- [Backup and restore](../../administration/backup_restore/index.md) - [Inactive project deletion](../../administration/inactive_project_deletion.md) - [Move repositories](../../administration/operations/moving_repositories.md) - [Read-only state](../../administration/read_only_gitlab.md) diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md index e9d829f3f08..c27bedd39de 100644 --- a/doc/administration/operations/moving_repositories.md +++ b/doc/administration/operations/moving_repositories.md @@ -188,14 +188,14 @@ Each of the approaches we list can or does overwrite data in the target director ### Recommended approach in all cases -For either Gitaly or Gitaly Cluster targets, the GitLab [backup and restore capability](../../raketasks/backup_restore.md) +For either Gitaly or Gitaly Cluster targets, the GitLab [backup and restore capability](../../administration/backup_restore/index.md) should be used. Git repositories are accessed, managed, and stored on GitLab servers by Gitaly as a database. Data loss can result from directly accessing and copying Gitaly files using tools like `rsync`. - From GitLab 13.3, backup performance can be improved by - [processing multiple repositories concurrently](../../raketasks/backup_gitlab.md#back-up-git-repositories-concurrently). + [processing multiple repositories concurrently](../../administration/backup_restore/backup_gitlab.md#back-up-git-repositories-concurrently). - Backups can be created of just the repositories using the - [skip feature](../../raketasks/backup_gitlab.md#excluding-specific-directories-from-the-backup). + [skip feature](../../administration/backup_restore/backup_gitlab.md#excluding-specific-directories-from-the-backup). No other method works for Gitaly Cluster targets. diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index d7d6f6228f9..f471dcd44b0 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -173,7 +173,7 @@ optimal configuration: ## Configuring Puma to listen over SSL -Puma, when deployed with Omnibus GitLab, listens over a Unix socket by +Puma, when deployed with a Linux package installation, listens over a Unix socket by default. To configure Puma to listen over an HTTPS port instead, follow the steps below: @@ -283,7 +283,7 @@ For Helm-based deployments, see the [`webservice` chart documentation](https://docs.gitlab.com/charts/charts/gitlab/webservice/index.html). Starting with GitLab 13.0, Puma is the default web server and Unicorn has been disabled. -In GitLab 14.0, [Unicorn was removed](../../update/removals.md#unicorn-in-gitlab-self-managed) +In GitLab 14.0, [Unicorn was removed](https://docs.gitlab.com/omnibus/update/gitlab_14_changes.html) from the Linux package and is no longer supported. Puma has a multi-thread architecture that uses less memory than a multi-process @@ -386,7 +386,7 @@ downtime. Otherwise, skip to the next section. GDB reports an error if the Puma process terminates before you can run these commands. To buy more time, you can always raise the -Puma worker timeout. For omnibus users, you can edit `/etc/gitlab/gitlab.rb` and +Puma worker timeout. For Linux package installation users, you can edit `/etc/gitlab/gitlab.rb` and increase it from 60 seconds to 600: ```ruby diff --git a/doc/administration/operations/rails_console.md b/doc/administration/operations/rails_console.md index ac550d30566..ac0a7e5870b 100644 --- a/doc/administration/operations/rails_console.md +++ b/doc/administration/operations/rails_console.md @@ -25,28 +25,34 @@ Rails experience is useful but not required. ## Starting a Rails console session -**For Omnibus installations** +The process for starting a Rails console session depends on the type of GitLab installation. + +::Tabs + +:::TabTitle Linux package (Omnibus) ```shell sudo gitlab-rails console ``` -**For Docker installations** +:::TabTitle Docker ```shell docker exec -it <container-id> gitlab-rails console ``` -**For installations from source** +:::TabTitle Self-compiled (source) ```shell sudo -u git -H bundle exec rails console -e production ``` -**For Kubernetes deployments** +:::TabTitle Helm chart (Kubernetes) The console is in the toolbox pod. Refer to our [Kubernetes cheat sheet](https://docs.gitlab.com/charts/troubleshooting/kubernetes_cheat_sheet.html#gitlab-specific-kubernetes-information) for details. +::EndTabs + To exit the console, type: `quit`. ## Enable Active Record logging @@ -130,31 +136,31 @@ environment, you can do so using the [Rails Runner](https://guides.rubyonrails.o When executing a script file, the script must be accessible by the `git` user. When the command or script completes, the Rails Runner process finishes. -It is useful for running within other scripts or cron jobs for example. +It is useful for running in other scripts or cron jobs for example. -**For Omnibus installations** +- For Linux package installations: -```shell -sudo gitlab-rails runner "RAILS_COMMAND" + ```shell + sudo gitlab-rails runner "RAILS_COMMAND" -# Example with a two-line Ruby script -sudo gitlab-rails runner "user = User.first; puts user.username" + # Example with a two-line Ruby script + sudo gitlab-rails runner "user = User.first; puts user.username" -# Example with a ruby script file (make sure to use the full path) -sudo gitlab-rails runner /path/to/script.rb -``` + # Example with a ruby script file (make sure to use the full path) + sudo gitlab-rails runner /path/to/script.rb + ``` -**For installations from source** +- For self-compiled installations: -```shell -sudo -u git -H bundle exec rails runner -e production "RAILS_COMMAND" + ```shell + sudo -u git -H bundle exec rails runner -e production "RAILS_COMMAND" -# Example with a two-line Ruby script -sudo -u git -H bundle exec rails runner -e production "user = User.first; puts user.username" + # Example with a two-line Ruby script + sudo -u git -H bundle exec rails runner -e production "user = User.first; puts user.username" -# Example with a ruby script file (make sure to use the full path) -sudo -u git -H bundle exec rails runner -e production /path/to/script.rb -``` + # Example with a ruby script file (make sure to use the full path) + sudo -u git -H bundle exec rails runner -e production /path/to/script.rb + ``` Rails Runner does not produce the same output as the console. @@ -180,68 +186,6 @@ Some basic knowledge of Ruby is very useful. Try [this 30-minute tutorial](https://try.ruby-lang.org/) for a quick introduction. Rails experience is helpful but not essential. -### Troubleshooting Rails Runner - -The `gitlab-rails` command executes Rails Runner using a non-root account and group, by default: `git:git`. - -If the non-root account cannot find the Ruby script filename passed to `gitlab-rails runner` -you may get a syntax error, not an error that the file couldn't be accessed. - -A common reason for this is that the script has been put in the root account's home directory. - -`runner` tries to parse the path and file parameter as Ruby code. - -For example: - -```plaintext -[root ~]# echo 'puts "hello world"' > ./helloworld.rb -[root ~]# sudo gitlab-rails runner ./helloworld.rb -Please specify a valid ruby command or the path of a script to run. -Run 'rails runner -h' for help. - -/opt/gitlab/..../runner_command.rb:45: syntax error, unexpected '.' -./helloworld.rb -^ -[root ~]# sudo gitlab-rails runner /root/helloworld.rb -Please specify a valid ruby command or the path of a script to run. -Run 'rails runner -h' for help. - -/opt/gitlab/..../runner_command.rb:45: unknown regexp options - hllwrld -[root ~]# mv ~/helloworld.rb /tmp -[root ~]# sudo gitlab-rails runner /tmp/helloworld.rb -hello world -``` - -A meaningful error should be generated if the directory can be accessed, but the file cannot: - -```plaintext -[root ~]# chmod 400 /tmp/helloworld.rb -[root ~]# sudo gitlab-rails runner /tmp/helloworld.rb -Traceback (most recent call last): - [traceback removed] -/opt/gitlab/..../runner_command.rb:42:in `load': cannot load such file -- /tmp/helloworld.rb (LoadError) -``` - -In case you encounter a similar error to this: - -```plaintext -[root ~]# sudo gitlab-rails runner helloworld.rb -Please specify a valid ruby command or the path of a script to run. -Run 'rails runner -h' for help. - -undefined local variable or method `helloworld' for main:Object -``` - -You can either move the file to the `/tmp` directory or create a new directory owned by the user `git` and save the script in that directory as illustrated below: - -```shell -sudo mkdir /scripts -sudo mv /script_path/helloworld.rb /scripts -sudo chown -R git:git /scripts -sudo chmod 700 /scripts -sudo gitlab-rails runner /scripts/helloworld.rb -``` - ## Find specific methods for an object ```ruby @@ -264,7 +208,7 @@ Adding a semicolon(`;`) and a follow-up statement at the end of a statement prev ```ruby puts ActiveRecord::Base.descendants; :ok -Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true +Project.select(&:pages_deployed?).each {|p| puts p.path }; true ``` ## Get or store the result of last operation @@ -756,3 +700,84 @@ project.irb irb(#<Project>)> web_url # => "https://gitlab-example/root/discard" ``` + +## Troubleshooting + +### Rails Runner `syntax error` + +The `gitlab-rails` command executes Rails Runner using a non-root account and group, by default: `git:git`. + +If the non-root account cannot find the Ruby script filename passed to `gitlab-rails runner` +you may get a syntax error, not an error that the file couldn't be accessed. + +A common reason for this is that the script has been put in the root account's home directory. + +`runner` tries to parse the path and file parameter as Ruby code. + +For example: + +```plaintext +[root ~]# echo 'puts "hello world"' > ./helloworld.rb +[root ~]# sudo gitlab-rails runner ./helloworld.rb +Please specify a valid ruby command or the path of a script to run. +Run 'rails runner -h' for help. + +/opt/gitlab/..../runner_command.rb:45: syntax error, unexpected '.' +./helloworld.rb +^ +[root ~]# sudo gitlab-rails runner /root/helloworld.rb +Please specify a valid ruby command or the path of a script to run. +Run 'rails runner -h' for help. + +/opt/gitlab/..../runner_command.rb:45: unknown regexp options - hllwrld +[root ~]# mv ~/helloworld.rb /tmp +[root ~]# sudo gitlab-rails runner /tmp/helloworld.rb +hello world +``` + +A meaningful error should be generated if the directory can be accessed, but the file cannot: + +```plaintext +[root ~]# chmod 400 /tmp/helloworld.rb +[root ~]# sudo gitlab-rails runner /tmp/helloworld.rb +Traceback (most recent call last): + [traceback removed] +/opt/gitlab/..../runner_command.rb:42:in `load': cannot load such file -- /tmp/helloworld.rb (LoadError) +``` + +In case you encounter a similar error to this: + +```plaintext +[root ~]# sudo gitlab-rails runner helloworld.rb +Please specify a valid ruby command or the path of a script to run. +Run 'rails runner -h' for help. + +undefined local variable or method `helloworld' for main:Object +``` + +You can either move the file to the `/tmp` directory or create a new directory owned by the user `git` and save the script in that directory as illustrated below: + +```shell +sudo mkdir /scripts +sudo mv /script_path/helloworld.rb /scripts +sudo chown -R git:git /scripts +sudo chmod 700 /scripts +sudo gitlab-rails runner /scripts/helloworld.rb +``` + +### Filtered console output + +Some output in the console might be filtered by default to prevent leaks of certain values +like variables, logs, or secrets. This output displays as `[FILTERED]`. For example: + +```plain_text +> Plan.default.actual_limits +=> ci_instance_level_variables: "[FILTERED]", +``` + +To work around the filtering, read the values directly from the object. For example: + +```plain_text +> Plan.default.limits.ci_instance_level_variables +=> 25 +``` diff --git a/doc/administration/package_information/defaults.md b/doc/administration/package_information/defaults.md index ac183afdc2f..f85ada3c782 100644 --- a/doc/administration/package_information/defaults.md +++ b/doc/administration/package_information/defaults.md @@ -11,8 +11,7 @@ the package assumes the defaults as noted below. ## Ports -See the table below for the list of ports that the Omnibus GitLab assigns -by default: +See the table below for the list of ports that the Linux package assigns by default: | Component | On by default | Communicates via | Alternative | Connection port | |:--------------------:|:-------------:|:----------------:|:-----------:|:------------------------------------------:| @@ -47,7 +46,7 @@ by default: | Mattermost | No | Port | X | 8065 | | Mattermost | No | Port | X | 80 or 443 | | PgBouncer | No | Port | X | 6432 | -| Consul | No | Port | X | 8300, 8301(UDP), 8500, 8600[^Consul-notes] | +| Consul | No | Port | X | 8300, 8301(TCP and UDP), 8500, 8600[^Consul-notes] | | Patroni | No | Port | X | 8008 | | GitLab KAS | Yes | Port | X | 8150 | | Gitaly | Yes | Socket | Port (8075) | 8075 or 9999 (TLS) | diff --git a/doc/administration/package_information/deprecation_policy.md b/doc/administration/package_information/deprecation_policy.md index d8f4551ca09..f2d23da2b7e 100644 --- a/doc/administration/package_information/deprecation_policy.md +++ b/doc/administration/package_information/deprecation_policy.md @@ -4,9 +4,9 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Omnibus GitLab deprecation policy **(FREE SELF)** +# Linux package deprecation policy **(FREE SELF)** -The Omnibus GitLab packages come with number of different libraries and services which offers users plethora of configuration options. +The Linux packages come with number of different libraries and services which offers users plethora of configuration options. As libraries and services get updated, their configuration options change and become obsolete. To increase maintainability and preserve a working @@ -16,7 +16,7 @@ setup, various configuration requires removal. ### Policy -The Omnibus GitLab package retains configuration for at least **one major** +The Linux package retains configuration for at least **one major** version. We can't guarantee that deprecated configuration is available in the next major release. See [example](#example) for more details. @@ -49,7 +49,8 @@ Deprecation procedure is similar for both `sensitive` and `regular` configuratio Common steps: -1. Create an issue at the [Omnibus GitLab issue tracker](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues) with details on deprecation type and other necessary information. Apply the label `deprecation`. +1. Create an issue at the [`omnibus-gitlab` issue tracker](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues) with + details on deprecation type and other necessary information. Apply the label `deprecation`. 1. Decide on the removal target for the deprecated configuration 1. Formulate deprecation notice for each item as noted in [Notice section](#notice) diff --git a/doc/administration/package_information/index.md b/doc/administration/package_information/index.md index bfa751a051f..503fdfe3ba8 100644 --- a/doc/administration/package_information/index.md +++ b/doc/administration/package_information/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Package information **(FREE SELF)** -The Omnibus GitLab package is bundled with all dependencies required for GitLab +The Linux package is bundled with all dependencies required for GitLab to function correctly. More details can be found at [bundling dependencies document](omnibus_packages.md). @@ -14,11 +14,11 @@ at [bundling dependencies document](omnibus_packages.md). The released package versions are in the format `MAJOR.MINOR.PATCH-EDITION.OMNIBUS_RELEASE` -| Component | Meaning | Example | -|---------------------|---------|---------| -| `MAJOR.MINOR.PATCH` | The GitLab version this corresponds to. | `13.3.0` | -| `EDITION` | The edition of GitLab this corresponds to. | `ee` | -| `OMNIBUS_RELEASE` | The Omnibus GitLab release. Usually, this is 0. This is incremented if we need to build a new package without changing the GitLab version. | `0` | +| Component | Meaning | Example | +|:--------------------|:------------------------------------------------------------------------------------------------------------------------------------------|:---------| +| `MAJOR.MINOR.PATCH` | The GitLab version this corresponds to. | `13.3.0` | +| `EDITION` | The edition of GitLab this corresponds to. | `ee` | +| `OMNIBUS_RELEASE` | The Linux package release. Usually, this is `0`. We increment this if we need to build a new package without changing the GitLab version. | `0` | ## Licenses @@ -26,7 +26,7 @@ See [licensing](licensing.md) ## Defaults -The Omnibus GitLab package requires various configuration to get the components +The Linux package requires various configuration to get the components in working order. If the configuration is not provided, the package uses the default values assumed in the package. @@ -34,10 +34,10 @@ These defaults are noted in the package [defaults document](defaults.md). ## Checking the versions of bundled software -After the Omnibus GitLab package is installed, you can find the version of +After the Linux package is installed, you can find the version of GitLab and all bundled libraries in `/opt/gitlab/version-manifest.txt`. -If you don't have the package installed, you can always check the Omnibus GitLab +If you don't have the package installed, you can always check the Linux package [source repository](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master), specifically the [configuration directory](https://gitlab.com/gitlab-org/omnibus-gitlab/tree/master/config). @@ -51,15 +51,14 @@ Documentation on package signatures can be found at [Signed Packages](signed_pac ## Checking for newer configuration options on upgrade -Configuration file in `/etc/gitlab/gitlab.rb` is created on initial installation -of the Omnibus GitLab package. On subsequent package upgrades, the configuration -file is not updated with new configuration. This is done to avoid -accidental overwrite of user configuration provided in `/etc/gitlab/gitlab.rb`. +The `/etc/gitlab/gitlab.rb` configuration file is created when the Linux package is initially installed. +To avoid accidental overwrites of user configuration, the `/etc/gitlab/gitlab.rb` configuration file is not updated +with new configuration when the Linux package installation is upgraded. New configuration options are noted in the [`gitlab.rb.template` file](https://gitlab.com/gitlab-org/omnibus-gitlab/raw/master/files/gitlab-config-template/gitlab.rb.template). -The Omnibus GitLab package also provides convenience command which +The Linux package also provides a convenience command which compares the existing user configuration with the latest version of the template contained in the package. @@ -76,7 +75,7 @@ characters on each line. ## Init system detection -Omnibus GitLab attempts to query the underlying system to +The Linux package attempts to query the underlying system to check which init system it uses. This manifests itself as a `WARNING` during the `sudo gitlab-ctl reconfigure` run. diff --git a/doc/administration/package_information/licensing.md b/doc/administration/package_information/licensing.md index bbc0e864e95..cad9c0e6075 100644 --- a/doc/administration/package_information/licensing.md +++ b/doc/administration/package_information/licensing.md @@ -8,12 +8,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## License -While GitLab itself is MIT, the Omnibus GitLab sources are licensed under the Apache-2.0. +While GitLab itself is MIT, the Linux package sources are licensed under the Apache-2.0. ## License file location -Starting with version 8.11, the Omnibus GitLab package contains license -information of all software that is bundled within the package. +Starting with version 8.11, the Linux package contains license +information of all software that is bundled in the package. After installing the package, licenses for each individual bundled library can be found in `/opt/gitlab/LICENSES` directory. @@ -21,11 +21,11 @@ can be found in `/opt/gitlab/LICENSES` directory. There is also one `LICENSE` file which contains all licenses compiled together. This compiled license can be found in `/opt/gitlab/LICENSE` file. -Starting with version 9.2, the Omnibus GitLab package ships a +Starting with version 9.2, the Linux package ships with a `dependency_licenses.json` file containing version and license information of all bundled software, including software libraries, Ruby gems that the rails application uses, and JavaScript libraries that is required for the frontend -components. Because it's in JSON format, GitLab can parse this file easily and use it for automated checks or validations. The file may be found at +components. Because it's in JSON format, GitLab can parse this file and use it for automated checks or validations. The file may be found at `/opt/gitlab/dependency_licenses.json`. Starting with version 11.3, we have also made the license information available @@ -33,28 +33,28 @@ online, at: <https://gitlab-org.gitlab.io/omnibus-gitlab/licenses.html> ## Checking licenses -The Omnibus GitLab package is made up of many pieces of software, comprising code +The Linux package is made up of many pieces of software, comprising code that is covered by many different licenses. Those licenses are provided and compiled as stated above. Starting with version 8.13, GitLab has placed an additional step into -Omnibus GitLab. The `license_check` step calls +Linux package installation. The `license_check` step calls `lib/gitlab/tasks/license_check.rake`, which checks the compiled `LICENSE` file against the current list of approved and questionable licenses as denoted in the arrays at the top of the script. This script outputs one of `Good`, `Unknown` or `Check` for each piece of software that is a part of the -Omnibus GitLab package. +Linux package. -- `Good`: denotes a license that is approved for all usage types, within GitLab and - Omnibus GitLab. +- `Good`: denotes a license that is approved for all usage types, in GitLab and + the Linux package. - `Unknown`: denotes a license that is not recognized in the list of 'good' or 'bad', which should be immediately reviewed for implications of use. - `Check`: denotes a license that has the potential be incompatible with GitLab itself, - and thus should be checked for how it is used as a part of the Omnibus GitLab package + and thus should be checked for how it is used as a part of the Linux package to ensure compliance. -This list is currently sourced from the [GitLab development documentation on licensing](https://gitlab.com/gitlab-org/gitlab-foss/blob/master/doc/development/licensing.md). -However, due to the nature of the Omnibus GitLab package the licenses may not apply +This list is sourced from the [GitLab development documentation on licensing](https://gitlab.com/gitlab-org/gitlab-foss/blob/master/doc/development/licensing.md). +However, due to the nature of the Linux package, the licenses may not apply in the same way. Such as with `git` and `rsync`. See the [GNU License FAQ](https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation) ## License acknowledgements diff --git a/doc/administration/package_information/omnibus_packages.md b/doc/administration/package_information/omnibus_packages.md index 222341c7308..d16783034dc 100644 --- a/doc/administration/package_information/omnibus_packages.md +++ b/doc/administration/package_information/omnibus_packages.md @@ -4,7 +4,7 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Omnibus based packages and images **(FREE SELF)** +# Packages and images from the Linux package **(FREE SELF)** Below you can find some basic information on why GitLab provides packages and a Docker image that come with bundled dependencies. @@ -19,7 +19,7 @@ We have a few core goals with these packages: 1. Support for a wide variety of operating systems 1. Wide support of cloud service providers -## Omnibus GitLab Architecture +## Linux package architecture GitLab in its core is a Ruby on Rails project. However, GitLab as a whole application is more complex and has multiple components. If these components are @@ -72,7 +72,7 @@ Some drawbacks of a package with bundled dependencies: 1. Duplication with possibly existing software. 1. Less flexibility in configuration. -## Why would you install an omnibus package when you can use a system package? +## Why would you install a package from the Linux package when you can use a system package? The answer can be simplified to: less maintenance required. Instead of handling multiple packages that *can* break existing functionality if the versions are @@ -83,23 +83,24 @@ Keeping configuration in sync can be error prone. If you have the skill set to maintain all current dependencies and enough time to handle any future dependencies that might get introduced, the above listed -reasons might not be good enough for you to not use the omnibus package. +reasons might not be good enough for you to not use a package from the Linux package. There are two things to keep in mind before going down this route: 1. Getting support for any problems you encounter might be more difficult due to the number of possibilities that exist when using a library version that is not tested by majority of users. -1. Omnibus package also allows shutting off of any services that you do not need, +1. Packages from the Linux package also allow shutting off of any services that you do not need, if you need to run a component independently. For example, you can use a - [non-bundled PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#using-a-non-packaged-postgresql-database-management-server) with the omnibus package. + [non-bundled PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#using-a-non-packaged-postgresql-database-management-server) + with a Linux package installation. -Keep in mind that a non-standard solution like the omnibus package +Keep in mind that a non-standard solution like the Linux package might be a better fit when the application has a number of moving parts. ## Docker image with multiple services -[GitLab Docker image](../../install/docker.md#gitlab-docker-images) is based on the omnibus package. +[GitLab Docker image](../../install/docker.md#gitlab-docker-images) is based on the Linux package. Considering that container spawned from this image contains multiple processes, these types of containers are also referred to as 'fat containers'. diff --git a/doc/administration/package_information/postgresql_versions.md b/doc/administration/package_information/postgresql_versions.md index 44032883eb4..101e1549d19 100644 --- a/doc/administration/package_information/postgresql_versions.md +++ b/doc/administration/package_information/postgresql_versions.md @@ -4,23 +4,24 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# PostgreSQL versions shipped with Omnibus GitLab **(FREE SELF)** +# PostgreSQL versions shipped with the Linux package **(FREE SELF)** NOTE: This table lists only GitLab versions where a significant change happened in the package regarding PostgreSQL versions, not all. Usually, PostgreSQL versions change with major or minor GitLab releases. However, patch versions -of Omnibus GitLab sometimes update the patch level of PostgreSQL. We've established a +of the Linux package sometimes update the patch level of PostgreSQL. We've established a [yearly cadence for PostgreSQL upgrades](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/postgresql-upgrade-cadence.html) and trigger automatic database upgrades in the release before the new version is required. For example: -- Omnibus 12.7.6 shipped with PostgreSQL 9.6.14 and 10.9. -- Omnibus 12.7.7 shipped with PostgreSQL 9.6.17 and 10.12. +- Linux package 12.7.6 shipped with PostgreSQL 9.6.14 and 10.9. +- Linux package 12.7.7 shipped with PostgreSQL 9.6.17 and 10.12. -[Find out which versions of PostgreSQL (and other components) ship with each Omnibus GitLab release](https://gitlab-org.gitlab.io/omnibus-gitlab/licenses.html). +Find out [which versions of PostgreSQL (and other components) ship](https://gitlab-org.gitlab.io/omnibus-gitlab/licenses.html) +with each Linux package release. The lowest supported PostgreSQL versions are listed in the [installation requirements](../../install/requirements.md#postgresql-requirements). @@ -29,13 +30,16 @@ Read more about update policies and warnings in the PostgreSQL [upgrade docs](https://docs.gitlab.com/omnibus/settings/database.html#upgrade-packaged-postgresql-server). | GitLab version | PostgreSQL versions | Default version for fresh installs | Default version for upgrades | Notes | -| -------------- | --------------------- | ---------------------------------- | ---------------------------- | ----- | -| 16.0 | 13.11 | 13.11 | 13.11 | | +| -------------- | ------------------- | ---------------------------------- | ---------------------------- | ----- | +| 16.0.2 | 13.11 | 13.11 | 13.11 | | +| 16.0.0 | 13.8 | 13.8 | 13.8 | | +| 15.11.7 | 13.11 | 13.11 | 12.12 | | +| 15.10.8 | 13.11 | 13.11 | 12.12 | | | 15.6 | 12.12, 13.8 | 13.8 | 12.12 | For upgrades, users can manually upgrade to 13.8 following the [upgrade documentation](https://docs.gitlab.com/omnibus/settings/database.html#gitlab-150-and-later). | | 15.0 | 12.10, 13.6 | 13.6 | 12.10 | For upgrades, users can manually upgrade to 13.6 following the [upgrade documentation](https://docs.gitlab.com/omnibus/settings/database.html#gitlab-150-and-later). | -| 14.1 | 12.7, 13.3 | 12.7 | 12.7 | PostgreSQL 13 available for fresh installations if not using [Geo](../geo/index.md#requirements-for-running-geo) or [Patroni](../postgresql/index.md#postgresql-replication-and-failover-with-omnibus-gitlab). -| 14.0 | 12.7 | 12.7 | 12.7 | HA installations with repmgr are no longer supported and are prevented from upgrading to Omnibus GitLab 14.0 | -| 13.8 | 11.9, 12.4 | 12.4 | 12.4 | Package upgrades automatically performed PostgreSQL upgrade for nodes that are not part of a Geo or HA cluster.). | +| 14.1 | 12.7, 13.3 | 12.7 | 12.7 | PostgreSQL 13 available for fresh installations if not using [Geo](../geo/index.md#requirements-for-running-geo) or [Patroni](../postgresql/index.md#postgresql-replication-and-failover-for-linux-package-installations). +| 14.0 | 12.7 | 12.7 | 12.7 | HA installations with repmgr are no longer supported and are prevented from upgrading to Linux package 14.0 | +| 13.8 | 11.9, 12.4 | 12.4 | 12.4 | Package upgrades automatically performed PostgreSQL upgrade for nodes that are not part of a Geo or HA cluster. | | 13.7 | 11.9, 12.4 | 12.4 | 11.9 | For upgrades users can manually upgrade to 12.4 following the [upgrade documentation](https://docs.gitlab.com/omnibus/settings/database.html#gitlab-133-and-later). | | 13.4 | 11.9, 12.4 | 11.9 | 11.9 | Package upgrades aborted if users not running PostgreSQL 11 already | | 13.3 | 11.7, 12.3 | 11.7 | 11.7 | Package upgrades aborted if users not running PostgreSQL 11 already | diff --git a/doc/administration/package_information/signed_packages.md b/doc/administration/package_information/signed_packages.md index 638dd7820b8..b2aab96e52c 100644 --- a/doc/administration/package_information/signed_packages.md +++ b/doc/administration/package_information/signed_packages.md @@ -6,13 +6,22 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Package Signatures **(FREE SELF)** -Omnibus GitLab packages produced by GitLab are created via the [Omnibus](https://github.com/chef/omnibus) tool, for which GitLab has added DEB signing via `debsigs` in [our own fork](https://gitlab.com/gitlab-org/omnibus). This addition, combined with the existing functionality of RPM signing, allows GitLab to provide signed packages for all supported distributions using DEB or RPM. +Linux packages produced by GitLab are created using [Omnibus](https://github.com/chef/omnibus), for which GitLab +has added DEB signing using `debsigs` in [our own fork](https://gitlab.com/gitlab-org/omnibus). -These packages are produced by the GitLab CI process, as found in the [Omnibus GitLab project](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/.gitlab-ci.yml), prior to their delivery to <https://packages.gitlab.com> to ensure provide assurance that the packages are not altered prior to delivery to our community. +Combined with the existing functionality of RPM signing, this addition allows GitLab to provide signed packages for all +supported distributions using DEB or RPM. + +These packages are produced by the GitLab CI process, as found in the +[`omnibus-gitlab` project](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/.gitlab-ci.yml), +prior to their delivery to <https://packages.gitlab.com> to provide assurance that the packages are not altered prior +to delivery to our community. ## GnuPG Public Keys -All packages are signed with [GnuPG](https://www.gnupg.org/), in a method appropriate for their format. The key used to sign these packages can be found on [MIT PGP Public Key Server](https://pgp.mit.edu) at [0x3cfcf9baf27eab47](https://pgp.mit.edu/pks/lookup?op=vindex&search=0x3CFCF9BAF27EAB47) +All packages are signed with [GnuPG](https://www.gnupg.org/), in a method appropriate for their format. The key used to +sign these packages can be found on [MIT PGP Public Key Server](https://pgp.mit.edu) at +[`0x3cfcf9baf27eab47`](https://pgp.mit.edu/pks/lookup?op=vindex&search=0x3CFCF9BAF27EAB47). ## Verifying Signatures diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md index 8ad3a656e05..16f4c2cdeab 100644 --- a/doc/administration/package_information/supported_os.md +++ b/doc/administration/package_information/supported_os.md @@ -83,7 +83,7 @@ running GitLab on ARM. ## OS Versions that are no longer supported -GitLab provides omnibus packages for operating systems only until their +GitLab provides Linux packages for operating systems only until their EOL (End-Of-Life). After the EOL date of the OS, GitLab stops releasing official packages. The list of deprecated operating systems and the final GitLab release for them can be found below: diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index e3867a25846..007072647a2 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -16,9 +16,11 @@ Registry, see the [user documentation](../../user/packages/container_registry/in ## Enable the Container Registry -**Omnibus GitLab installations** +The process for enabling the Container Registry depends on the type of installation you use. -If you installed GitLab by using the Omnibus installation package, the Container Registry +### Linux package installations + +If you installed GitLab by using the Linux package, the Container Registry may or may not be available by default. The Container Registry is automatically enabled and available on your GitLab domain, port 5050 if @@ -34,9 +36,9 @@ but it's not recommended and is beyond the scope of this document. Read the [insecure Registry documentation](https://docs.docker.com/registry/insecure/) if you want to implement this. -**Installations from source** +### Self-compiled installations -If you have installed GitLab from source: +If you self-compiled your GitLab installation: 1. You must [deploy a registry](https://docs.docker.com/registry/deploying/) using the image corresponding to the version of GitLab you are installing @@ -115,8 +117,8 @@ certificate. If the GitLab domain is `https://gitlab.example.com` and the port to the outside world is `5050`, to configure the Container Registry: -- Edit `gitlab.rb` if you are using Omnibus GitLab. -- Edit `gitlab.yml` if you installed GitLab from source. +- Edit `gitlab.rb` if you are using a Linux package installation. +- Edit `gitlab.yml` if you are using a self-compiled installation. Ensure you choose a port different than the one that Registry listens to (`5000` by default), otherwise conflicts occur. @@ -126,7 +128,9 @@ Host and container firewall rules must be configured to allow traffic in through under the `registry_external_url` line, rather than the port listed under `gitlab_rails['registry_port']` (default `5000`). -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Your `/etc/gitlab/gitlab.rb` should contain the Registry URL as well as the path to the existing TLS certificate and key used by GitLab: @@ -171,7 +175,7 @@ registry_nginx['redirect_http_to_https'] = true registry_nginx['listen_port'] = 5678 ``` -**Installations from source** +:::TabTitle Self-compiled (source) 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and configure it with the following settings: @@ -186,6 +190,8 @@ registry_nginx['listen_port'] = 5678 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. 1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path). +::EndTabs + Users should now be able to sign in to the Container Registry with their GitLab credentials using: @@ -202,12 +208,14 @@ domain. For example, `*.gitlab.example.com`, is a wildcard that matches `registr and is distinct from `*.example.com`. As well as manually generated SSL certificates (explained here), certificates automatically -generated by Let's Encrypt are also [supported in Omnibus installs](https://docs.gitlab.com/omnibus/settings/ssl/index.html). +generated by Let's Encrypt are also [supported in Linux package installations](https://docs.gitlab.com/omnibus/settings/ssl/index.html). Let's assume that you want the container Registry to be accessible at `https://registry.gitlab.example.com`. -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Place your TLS certificate and key in `/etc/gitlab/ssl/registry.gitlab.example.com.crt` and @@ -237,7 +245,7 @@ registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/certificate.pem" registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/certificate.key" ``` -**Installations from source** +:::TabTitle Self-compiled (source) 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and configure it with the following settings: @@ -251,6 +259,8 @@ registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/certificate.key" 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. 1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path). +::EndTabs + Users should now be able to sign in to the Container Registry using their GitLab credentials: @@ -264,7 +274,9 @@ When you disable the Registry by following these steps, you do not remove any existing Docker images. Docker image removal is handled by the Registry application itself. -**Omnibus GitLab** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Open `/etc/gitlab/gitlab.rb` and set `registry['enable']` to `false`: @@ -274,7 +286,7 @@ Registry application itself. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and set `enabled` to `false`: @@ -286,13 +298,17 @@ Registry application itself. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ## Disable Container Registry for new projects site-wide If the Container Registry is enabled, then it should be available on all new projects. To disable this function and let the owners of a project to enable the Container Registry by themselves, follow the steps below. -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: @@ -302,7 +318,7 @@ the Container Registry by themselves, follow the steps below. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `default_projects_features` entry and configure it so that `container_registry` is set to `false`: @@ -320,6 +336,8 @@ the Container Registry by themselves, follow the steps below. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Increase token duration In GitLab, tokens for the Container Registry expire every five minutes. @@ -381,9 +399,11 @@ This path is accessible to: All GitLab, Registry, and web server users must have access to this directory. -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) -The default location where images are stored in Omnibus, is +The default location where images are stored in Linux package installations is `/var/opt/gitlab/gitlab-rails/shared/registry`. To change it: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -394,9 +414,9 @@ The default location where images are stored in Omnibus, is 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) -The default location where images are stored in source installations, is +The default location where images are stored in self-compiled installations is `/home/git/gitlab/shared/registry`. To change it: 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and @@ -409,6 +429,8 @@ The default location where images are stored in source installations, is 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Use object storage If you want to store your images on object storage, you can change the storage @@ -421,9 +443,9 @@ GitLab does not back up Docker images that are not stored on the file system. Enable backups with your object storage provider if desired. -**Omnibus GitLab installations** +#### Linux package installations -To configure the `s3` storage driver in Omnibus: +To configure the `s3` storage driver for a Linux package installation: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -493,7 +515,7 @@ To configure the `s3` storage driver in Omnibus: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +#### Self-compiled installations Configuring the storage driver is done in the registry configuration YAML file created when you [deployed your Docker registry](https://docs.docker.com/registry/deploying/). @@ -602,7 +624,9 @@ When moving from an existing file system or another object storage provider to A Configure it by setting [`trimlegacyrootprefix: true`](https://gitlab.com/gitlab-org/container-registry/-/blob/a3f64464c3ec1c5a599c0a2daa99ebcbc0100b9a/docs-gitlab/README.md#azure-storage-driver) in the Azure storage driver section of the registry configuration. Without this configuration, the Azure storage driver uses `//` instead of `/` as the first section of the root path, rendering the migrated images inaccessible. -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) ```ruby registry['storage'] = { @@ -616,7 +640,7 @@ registry['storage'] = { } ``` -**Installations from source** +:::TabTitle Self-compiled (source) ```yaml storage: @@ -628,6 +652,8 @@ storage: trimlegacyrootprefix: true ``` +::EndTabs + By default, Azure Storage Driver uses the `core.windows.net` realm. You can set another value for `realm` in the `azure` section (for example, `core.usgovcloudapi.net` for Azure Government Cloud). For more information, see the [Docker documentation](https://docs.docker.com/registry/storage-drivers/azure/). ### Disable redirect for storage driver @@ -636,7 +662,9 @@ By default, users accessing a registry configured with a remote backend are redi However, this behavior is undesirable for registries used by internal hosts that usually can't access public servers. To disable redirects and [proxy download](../object_storage.md#proxy-download), set the `disable` flag to true as follows. This makes all traffic always go through the Registry service. This results in improved security (less surface attack as the storage backend is not publicly accessible), but worse performance (all traffic is redirected via the service). -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -657,7 +685,7 @@ However, this behavior is undesirable for registries used by internal hosts that 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Add the `redirect` flag to your registry configuration YAML file: @@ -679,6 +707,8 @@ However, this behavior is undesirable for registries used by internal hosts that 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + #### Encrypted S3 buckets You can use server-side encryption with AWS KMS for S3 buckets that have @@ -689,7 +719,9 @@ encryption keys in every request. For SSE-S3, you must enable the `encrypt` option in the registry settings. How you do this depends on how you installed GitLab. Follow the instructions here that match your installation method. -For Omnibus GitLab installations: +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -709,7 +741,7 @@ For Omnibus GitLab installations: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -For installations from source: +:::TabTitle Self-compiled (source) 1. Edit your registry configuration YAML file: @@ -727,6 +759,8 @@ For installations from source: 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Storage limitations There is no storage limitation, which means a user can upload an @@ -739,7 +773,9 @@ The Registry server listens on localhost at port `5000` by default, which is the address for which the Registry server should accept connections. In the examples below we set the Registry's port to `5010`. -**Omnibus GitLab** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Open `/etc/gitlab/gitlab.rb` and set `registry['registry_http_addr']`: @@ -749,7 +785,7 @@ In the examples below we set the Registry's port to `5010`. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Open the configuration file of your Registry server and edit the [`http:addr`](https://docs.docker.com/registry/configuration/#http) value: @@ -761,6 +797,8 @@ In the examples below we set the Registry's port to `5010`. 1. Save the file and restart the Registry server. +::EndTabs + ## Disable Container Registry per project If Registry is enabled in your GitLab instance, but you don't need it for your @@ -799,7 +837,7 @@ under the project hierarchy, like `registry.example.com/group/project/my/image-name:tag`, and only recognizes `registry.example.com/group/project:tag`. -**Omnibus GitLab** +### Linux package installations You can use GitLab as an auth endpoint with an external container registry. @@ -832,14 +870,14 @@ You can use GitLab as an auth endpoint with an external container registry. # Example: registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" - # Optionally define a custom file for Omnibus GitLab to write the contents + # Optionally define a custom file for a Linux package installation to write the contents # of registry['internal_key'] to. gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" ``` Each time reconfigure is executed, the file specified at `registry_key_path` gets populated with the content specified by `internal_key`. If - no file is specified, Omnibus GitLab defaults it to + no file is specified, Linux package installations default it to `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and populates it. @@ -854,7 +892,7 @@ You can use GitLab as an auth endpoint with an external container registry. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +### Self-compiled installations 1. Open `/home/git/gitlab/config/gitlab.yml`, and edit the configuration settings under `registry`: @@ -885,9 +923,11 @@ Read more about the Container Registry notifications configuration options in th You can configure multiple endpoints for the Container Registry. -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) -To configure a notification endpoint in Omnibus: +To configure a notification endpoint for a Linux package installation: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -908,7 +948,7 @@ To configure a notification endpoint in Omnibus: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) Configuring the notification endpoint is done in your registry configuration YAML file created when you [deployed your Docker registry](https://docs.docker.com/registry/deploying/). @@ -927,6 +967,8 @@ notifications: backoff: 1000 ``` +::EndTabs + ## Run the Cleanup policy now WARNING: @@ -1007,28 +1049,27 @@ NOTE: Retention policies in your object storage provider, such as Amazon S3 Lifecycle, may prevent objects from being properly deleted. -Container Registry can use considerable amounts of disk space. To clear up -some unused layers, the registry includes a garbage collect command. +The container registry can use considerable amounts of storage space, and you might want to +[reduce storage usage](../../user/packages/container_registry/reduce_container_registry_storage.md). +Among the listed options, deleting tags is the most effective option. However, tag deletion +alone does not delete image layers, it only leaves the underlying image manifests untagged. + +To more effectively free up space, the Container Registry has a garbage collector that can +delete unreferenced layers and (optionally) untagged manifests. -GitLab offers a set of APIs to manipulate the Container Registry and aid the process -of removing unused tags. Currently, this is exposed using the API, but in the future, -these controls should migrate to the GitLab interface. +To start the garbage collector, use the `registry-garbage-collect` command provided by `gitlab-ctl`. -Users who have the [Maintainer role](../../user/permissions.md) for the project can -[delete Container Registry tags in bulk](../../api/container_registry.md#delete-registry-repository-tags-in-bulk) -periodically based on their own criteria. However, deleting the tags alone does not recycle data, -it only unlinks tags from manifests and image blobs. To recycle the Container -Registry data in the whole GitLab instance, you can use the built-in garbage collection command -provided by `gitlab-ctl`. +WARNING: +This command shuts down the Container Registry prior to the garbage collection and +only starts it again after garbage collection completes. If you prefer to avoid downtime, +you can manually set the Container Registry to [read-only mode and bypass `gitlab-ctl`](#performing-garbage-collection-without-downtime). + +The time required to perform garbage collection is proportional to the Container Registry data size. Prerequisites: -- You must have installed GitLab by using an Omnibus package or the +- You must have installed GitLab by using a Linux package or the [GitLab Helm chart](https://docs.gitlab.com/charts/charts/registry/#garbage-collection). -- You must set the Registry to [read-only mode](#performing-garbage-collection-without-downtime). - Running garbage collection causes downtime for the Container Registry. When you run this command - on an instance in an environment where another instance is still writing to the Registry storage, - referenced manifests are removed. ### Understanding the content-addressable layers @@ -1053,16 +1094,11 @@ Due to the architecture of registry, this data is still accessible when pulling image `my.registry.com/my.group/my.project@sha256:111111`, though it is no longer directly accessible via the `:latest` tag. -### Recycling unused tags - -Before you run the built-in command, note the following: +### Remove unreferenced layers -- The built-in command stops the registry before it starts the garbage collection. -- The garbage collect command takes some time to complete, depending on the - amount of data that exists. -- If you changed the location of registry configuration file, you must - specify its path. -- After the garbage collection is done, the registry should start automatically. +Image layers are the bulk of the Container Registry storage. A layer is considered +unreferenced when no image manifest references it. Unreferenced layers are the +default target of the Container Registry garbage collector. If you did not change the default location of the configuration file, run: @@ -1070,51 +1106,37 @@ If you did not change the default location of the configuration file, run: sudo gitlab-ctl registry-garbage-collect ``` -This command takes some time to complete, depending on the amount of -layers you have stored. - If you changed the location of the Container Registry `config.yml`: ```shell sudo gitlab-ctl registry-garbage-collect /path/to/config.yml ``` -You may also [remove all untagged manifests and unreferenced layers](#removing-untagged-manifests-and-unreferenced-layers), -although this is a way more destructive operation, and you should first -understand the implications. +You can also [remove all untagged manifests and unreferenced layers](#removing-untagged-manifests-and-unreferenced-layers) +to recover additional space. ### Removing untagged manifests and unreferenced layers -WARNING: -This is a destructive operation. - -The GitLab Container Registry follows the same default workflow as Docker Distribution: -retain untagged manifests and all layers, even ones that are not referenced directly. All content -can be accessed by using context addressable identifiers. +By default the Container Registry garbage collector ignores images that are untagged, +and users can keep pulling untagged images by digest. Users can also re-tag images +in the future, making them visible again in the GitLab UI and API. -However, in most workflows, you don't care about untagged manifests and old layers if they are not directly -referenced by a tagged manifest. The `registry-garbage-collect` command supports the -`-m` switch to allow you to remove all unreferenced manifests and layers that are -not directly accessible via `tag`: +If you do not care about untagged images and the layers exclusively referenced by these images, +you can delete them all. Use the `-m` flag on the `registry-garbage-collect` command: ```shell sudo gitlab-ctl registry-garbage-collect -m ``` -Since this is a way more destructive operation, this behavior is disabled by default. -You are likely expecting this way of operation, but before doing that, ensure -that you have backed up all registry data. - -When the command is used without the `-m` flag, the Container Registry only removes layers that are not referenced by any manifest, tagged or not. +If you are unsure about deleting untagged images, back up your registry data before proceeding. ### Performing garbage collection without downtime -You can perform garbage collection without stopping the Container Registry by putting -it in read-only mode and by not using the built-in command. On large instances -this could require Container Registry to be in read-only mode for a while. -During this time, -you are able to pull from the Container Registry, but you are not able to -push. +To do garbage collection while keeping the Container Registry online, put the registry +in read-only mode and bypass the built-in `gitlab-ctl registry-garbage-collect` command. + +You can pull but not push images while the Container Registry is in read-only mode. The Container +Registry must remain in read-only for the full duration of the garbage collection. By default, the [registry storage path](#configure-storage-for-the-container-registry) is `/var/opt/gitlab/gitlab-rails/shared/registry`. @@ -1146,18 +1168,15 @@ To enable the read-only mode: 1. Next, trigger one of the garbage collect commands: - WARNING: - You must use `/opt/gitlab/embedded/bin/registry` to recycle unused tags. If you use `gitlab-ctl registry-garbage-collect`, **the container registry goes down**. - ```shell - # Recycling unused tags + # Remove unreferenced layers sudo /opt/gitlab/embedded/bin/registry garbage-collect /var/opt/gitlab/registry/config.yml - # Removing unused layers not referenced by manifests + # Remove untagged manifests and unreferenced layers sudo /opt/gitlab/embedded/bin/registry garbage-collect -m /var/opt/gitlab/registry/config.yml ``` - This command starts the garbage collection, which might take some time to complete. + This command starts the garbage collection. The time to complete is proportional to the registry data size. 1. Once done, in `/etc/gitlab/gitlab.rb` change it back to read-write mode: @@ -1212,13 +1231,13 @@ itself on the system so that the `gitlab-ctl` command can bring the registry ser Also, there's no way to save progress or results during the mark phase of the process. Only once blobs start being deleted is anything permanent done. -## Configuring GitLab and Registry to run on separate nodes (Omnibus GitLab) +## Configure GitLab and Registry to run on separate nodes (Linux package installations) By default, package assumes that both services are running on the same node. To get GitLab and Registry to run on a separate nodes, separate configuration is necessary for Registry and GitLab. -### Configuring Registry +### Configure Registry Below you can find configuration options you should set in `/etc/gitlab/gitlab.rb`, for Registry to run separately from GitLab: @@ -1235,7 +1254,7 @@ for Registry to run separately from GitLab: - `registry['health_storagedriver_enabled']`, default [set programmatically](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/10-7-stable/files/gitlab-cookbooks/gitlab/libraries/registry.rb#L88). Configure whether health checks on the configured storage driver are enabled. - `gitlab_rails['registry_issuer']`, [default value](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/10-3-stable/files/gitlab-cookbooks/gitlab/attributes/default.rb#L153). This setting needs to be set the same between Registry and GitLab. -### Configuring GitLab +### Configure GitLab Below you can find configuration options you should set in `/etc/gitlab/gitlab.rb`, for GitLab to run separately from Registry: @@ -1435,7 +1454,9 @@ level=error msg="response completed with error" err.code=unknown err.detail="une To resolve the error specify a `chunksize` value in the Registry configuration. Start with a value between `25000000` (25 MB) and `50000000` (50 MB). -**For Omnibus installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -1452,7 +1473,7 @@ Start with a value between `25000000` (25 MB) and `50000000` (50 MB). 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit `config/gitlab.yml`: @@ -1467,16 +1488,20 @@ Start with a value between `25000000` (25 MB) and `50000000` (50 MB). 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Supporting older Docker clients The Docker container registry shipped with GitLab disables the schema1 manifest by default. If you are still using older Docker clients (1.9 or older), you may experience an error pushing images. See -[omnibus-4145](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4145) for more details. +[issue 4145](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4145) for more details. You can add a configuration option for backwards compatibility. -**For Omnibus installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -1486,7 +1511,7 @@ You can add a configuration option for backwards compatibility. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit the YAML configuration file you created when you [deployed the registry](https://docs.docker.com/registry/deploying/). Add the following snippet: @@ -1498,6 +1523,8 @@ You can add a configuration option for backwards compatibility. 1. Restart the registry for the changes to take affect. +::EndTabs + ### Docker connection error A Docker connection error can occur when there are special characters in either the group, @@ -1522,7 +1549,9 @@ offloaded to a third party reverse proxy. This problem was discussed in a [Docker project issue](https://github.com/docker/distribution/issues/970) and a simple solution would be to enable relative URLs in the Registry. -**For Omnibus installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb`: @@ -1534,7 +1563,7 @@ and a simple solution would be to enable relative URLs in the Registry. 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**For installations from source** +:::TabTitle Self-compiled (source) 1. Edit the YAML configuration file you created when you [deployed the registry](https://docs.docker.com/registry/deploying/). Add the following snippet: @@ -1545,6 +1574,8 @@ and a simple solution would be to enable relative URLs in the Registry. 1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. +::EndTabs + ### Enable the Registry debug server You can use the Container Registry debug server to diagnose problems. The debug endpoint can monitor metrics and health, as well as do profiling. @@ -1602,7 +1633,9 @@ for more information. The following sections provide additional details about each installation method. -#### Helm chart installations +::Tabs + +:::TabTitle Helm chart (Kubernetes) For Helm chart installations: @@ -1614,9 +1647,9 @@ For Helm chart installations: No other registry configuration changes are required. -#### Omnibus installations +:::TabTitle Linux package (Omnibus) -For Omnibus installations: +For Linux package installations: 1. Temporarily replace the registry binary that ships with GitLab 13.9+ for one prior to `v3.0.0-gitlab`. To do so, pull a previous version of the Docker image for the GitLab Container @@ -1629,20 +1662,21 @@ For Omnibus installations: docker rm $id ``` -1. Replace the binary embedded in the Omnibus install, located at +1. Replace the binary embedded in the Linux package installation located at `/opt/gitlab/embedded/bin/registry`, with `registry-2.13.1-gitlab`. Make sure to start by backing - up the original binary embedded in Omnibus, and restore it after performing the - [image upgrade](#images-upgrade)) steps. You should [stop](https://docs.gitlab.com/omnibus/maintenance/#starting-and-stopping) + up the original binary embedded in the Linux package, and restore it after performing the + [image upgrade](#images-upgrade) steps. You should [stop](https://docs.gitlab.com/omnibus/maintenance/#starting-and-stopping) the registry service before replacing its binary and start it right after. No registry configuration changes are required. -#### Source installations +:::TabTitle Self-compiled (source) For source installations, locate your `registry` binary and temporarily replace it with the one -obtained from `v3.0.0-gitlab`, as explained for [Omnibus installations](#omnibus-installations). +obtained from `v3.0.0-gitlab`, as explained for Linux package installations. Make sure to start by backing up the original registry binary, and restore it after performing the -[images upgrade](#images-upgrade)) -steps. +[images upgrade](#images-upgrade) steps. + +::EndTabs #### Images upgrade diff --git a/doc/administration/packages/dependency_proxy.md b/doc/administration/packages/dependency_proxy.md index 5e770c2464b..ee352a82058 100644 --- a/doc/administration/packages/dependency_proxy.md +++ b/doc/administration/packages/dependency_proxy.md @@ -25,13 +25,11 @@ The GitLab Dependency Proxy: The Dependency Proxy is enabled by default. If you are an administrator, you can turn off the Dependency Proxy. To turn off the Dependency Proxy, follow the instructions that -correspond to your GitLab installation: +correspond to your GitLab installation. -- [Omnibus GitLab installations](#omnibus-gitlab-installations) -- [Helm chart installations](#helm-chart-installations) -- [Installations from source](#installations-from-source) +::Tabs -### Omnibus GitLab installations +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: @@ -42,7 +40,7 @@ correspond to your GitLab installation: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -### Helm chart installations +:::TabTitle Helm chart (Kubernetes) After the installation is complete, update the global `appConfig` to turn off the Dependency Proxy: @@ -59,7 +57,7 @@ global: For more information, see [Configure Charts using Globals](https://docs.gitlab.com/charts/charts/globals.html#configure-appconfig-settings). -### Installations from source +:::TabTitle Self-compiled (source) 1. After the installation is complete, configure the `dependency_proxy` section in `config/gitlab.yml`. Set `enabled` to `false` to turn off the Dependency Proxy: @@ -69,13 +67,13 @@ For more information, see [Configure Charts using Globals](https://docs.gitlab.c enabled: false ``` -1. [Restart GitLab](../restart_gitlab.md#installations-from-source "How to restart GitLab") - for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. + +::EndTabs ### Multi-node GitLab installations -Follow the steps for [Omnibus GitLab installations](#omnibus-gitlab-installations) -for each Web and Sidekiq node. +Follow the steps for Linux package installations for each Web and Sidekiq node. ## Turn on the Dependency Proxy @@ -91,12 +89,13 @@ local location or even use object storage. ### Changing the local storage path -The Dependency Proxy files for Omnibus GitLab installations are stored under +The Dependency Proxy files for Linux package installations are stored under `/var/opt/gitlab/gitlab-rails/shared/dependency_proxy/` and for source installations under `shared/dependency_proxy/` (relative to the Git home directory). -To change the local storage path: -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: @@ -106,7 +105,7 @@ To change the local storage path: 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Edit the `dependency_proxy` section in `config/gitlab.yml`: @@ -116,7 +115,9 @@ To change the local storage path: storage_path: shared/dependency_proxy ``` -1. [Restart GitLab](../restart_gitlab.md#installations-from-source "How to restart GitLab") for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. + +::EndTabs ### Using object storage @@ -127,7 +128,9 @@ This section describes the earlier configuration format. [Migration steps still [Read more about using object storage with GitLab](../object_storage.md). -**Omnibus GitLab installations** +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Edit `/etc/gitlab/gitlab.rb` and add the following lines (uncomment where necessary): @@ -158,7 +161,7 @@ This section describes the earlier configuration format. [Migration steps still 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -**Installations from source** +:::TabTitle Self-compiled (source) 1. Edit the `dependency_proxy` section in `config/gitlab.yml` (uncomment where necessary): @@ -190,7 +193,9 @@ This section describes the earlier configuration format. [Migration steps still # path_style: false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'. ``` -1. [Restart GitLab](../restart_gitlab.md#installations-from-source "How to restart GitLab") for the changes to take effect. +1. [Restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect. + +::EndTabs #### Migrate local Dependency Proxy blobs and manifests to object storage @@ -200,24 +205,24 @@ After [configuring object storage](#using-object-storage), use the following task to migrate existing Dependency Proxy blobs and manifests from local storage to remote storage. The processing is done in a background worker and requires no downtime. -For Omnibus GitLab: +- For Linux package installations: -```shell -sudo gitlab-rake "gitlab:dependency_proxy:migrate" -``` + ```shell + sudo gitlab-rake "gitlab:dependency_proxy:migrate" + ``` -For installations from source: +- For installations from source: -```shell -RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:dependency_proxy:migrate -``` + ```shell + RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:dependency_proxy:migrate + ``` You can optionally track progress and verify that all Dependency Proxy blobs and manifests migrated successfully using the [PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database): -- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier. -- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later. -- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. +- `sudo gitlab-rails dbconsole` for Linux package installations running version 14.1 and earlier. +- `sudo gitlab-rails dbconsole --database main` for Linux package installations running version 14.2 and later. +- `sudo -u git -H psql -d gitlabhq_production` for self-compiled instances. Verify that `objectstg` (where `file_store = '2'`) has the count of all Dependency Proxy blobs and manifests for each respective query: diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index b735204c323..77730384623 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -55,7 +55,7 @@ guides you through the process. When downloading packages as dependencies in downstream projects, many requests are made through the Packages API. You may therefore reach enforced user and IP rate limits. To address this issue, you -can define specific rate limits for the Packages API. For more details, see [Package Registry Rate Limits](../../user/admin_area/settings/package_registry_rate_limits.md). +can define specific rate limits for the Packages API. For more details, see [Package Registry Rate Limits](../settings/package_registry_rate_limits.md). ## Enable or disable the Package Registry diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index e86726524d0..726307229d6 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -10,9 +10,8 @@ GitLab Pages allows for hosting of static sites. It must be configured by an administrator. Separate [user documentation](../../user/project/pages/index.md) is available. NOTE: -This guide is for Omnibus GitLab installations. If you have installed -GitLab from source, see -[GitLab Pages administration for source installations](source.md). +This guide is for Linux package installations. If you have a self-compiled GitLab installation, see +[GitLab Pages administration for self-compiled installations](source.md). ## The GitLab Pages daemon @@ -225,88 +224,89 @@ This setup is primarily intended to be used when [installing a GitLab POC on Ama ### Global settings -Below is a table of all configuration settings known to Pages in Omnibus GitLab, +Below is a table of all configuration settings known to Pages in a Linux package installation, and what they do. These options can be adjusted in `/etc/gitlab/gitlab.rb`, and take effect after you [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation). Most of these settings don't have to be configured manually unless you need more granular control over how the Pages daemon runs and serves content in your environment. -| Setting | Description | -|-----------------------------------------|-------------| +| Setting | Description | +|-----------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `pages_external_url` | The URL where GitLab Pages is accessible, including protocol (HTTP / HTTPS). If `https://` is used, additional configuration is required. See [Wildcard domains with TLS support](#wildcard-domains-with-tls-support) and [Custom domains with TLS support](#custom-domains-with-tls-support) for details. | -| **`gitlab_pages[]`** | | -| `access_control` | Whether to enable [access control](index.md#access-control). | -| `api_secret_key` | Full path to file with secret key used to authenticate with the GitLab API. Auto-generated when left unset. | -| `artifacts_server` | Enable viewing [artifacts](../job_artifacts.md) in GitLab Pages. | -| `artifacts_server_timeout` | Timeout (in seconds) for a proxied request to the artifacts server. | -| `artifacts_server_url` | API URL to proxy artifact requests to. Defaults to GitLab `external URL` + `/api/v4`, for example `https://gitlab.com/api/v4`. When running a [separate Pages server](#running-gitlab-pages-on-a-separate-server), this URL must point to the main GitLab server's API. | -| `auth_redirect_uri` | Callback URL for authenticating with GitLab. Defaults to project's subdomain of `pages_external_url` + `/auth`. | -| `auth_secret` | Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration. | -| `dir` | Working directory for configuration and secrets files. | -| `enable` | Enable or disable GitLab Pages on the current system. | -| `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`. | -| `external_https` | Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_https`. | -| `server_shutdown_timeout` | GitLab Pages server shutdown timeout in seconds (default: `30s`). | -| `gitlab_client_http_timeout` | GitLab API HTTP client connection timeout in seconds (default: `10s`). | -| `gitlab_client_jwt_expiry` | JWT Token expiry time in seconds (default: `30s`). | -| `gitlab_cache_expiry` | The maximum time a domain's configuration is stored in the cache (default: `600s`). | -| `gitlab_cache_refresh` | The interval at which a domain's configuration is set to be due to refresh (default: `60s`). | -| `gitlab_cache_cleanup` | The interval at which expired items are removed from the cache (default: `60s`). | -| `gitlab_retrieval_timeout` | The maximum time to wait for a response from the GitLab API per request (default: `30s`). | -| `gitlab_retrieval_interval` | The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: `1s`). | -| `gitlab_retrieval_retries` | The maximum number of times to retry to resolve a domain's configuration via the API (default: 3). | -| `domain_config_source` | This parameter was removed in 14.0, on earlier versions it can be used to enable and test API domain configuration source | -| `gitlab_id` | The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. | -| `gitlab_secret` | The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. | -| `auth_scope` | The OAuth application scope to use for authentication. Must match GitLab Pages OAuth application settings. Leave blank to use `api` scope by default. | -| `auth_cookie_session_timeout` | Authentication cookie session timeout in seconds (default: `10m`). A value of `0` means the cookie is deleted after the browser session ends. | -| `gitlab_server` | Server to use for authentication when access control is enabled; defaults to GitLab `external_url`. | -| `headers` | Specify any additional http headers that should be sent to the client with each response. Multiple headers can be given as an array, header and value as one string, for example `['my-header: myvalue', 'my-other-header: my-other-value']` | -| `enable_disk` | Allows the GitLab Pages daemon to serve content from disk. Shall be disabled if shared disk storage isn't available. | -| `insecure_ciphers` | Use default list of cipher suites, may contain insecure ones like 3DES and RC4. | -| `internal_gitlab_server` | Internal GitLab server address used exclusively for API requests. Useful if you want to send that traffic over an internal load balancer. Defaults to GitLab `external_url`. | -| `listen_proxy` | The addresses to listen on for reverse-proxy requests. Pages binds to these addresses' network sockets and receives incoming requests from them. Sets the value of `proxy_pass` in `$nginx-dir/conf/gitlab-pages.conf`. | -| `log_directory` | Absolute path to a log directory. | -| `log_format` | The log output format: `text` or `json`. | -| `log_verbose` | Verbose logging, true/false. | -| `propagate_correlation_id` | Set to true (false by default) to re-use existing Correlation ID from the incoming request header `X-Request-ID` if present. If a reverse proxy sets this header, the value is propagated in the request chain. | -| `max_connections` | Limit on the number of concurrent connections to the HTTP, HTTPS or proxy listeners. | +| **`gitlab_pages[]`** | | +| `access_control` | Whether to enable [access control](index.md#access-control). | +| `api_secret_key` | Full path to file with secret key used to authenticate with the GitLab API. Auto-generated when left unset. | +| `artifacts_server` | Enable viewing [artifacts](../job_artifacts.md) in GitLab Pages. | +| `artifacts_server_timeout` | Timeout (in seconds) for a proxied request to the artifacts server. | +| `artifacts_server_url` | API URL to proxy artifact requests to. Defaults to GitLab `external URL` + `/api/v4`, for example `https://gitlab.com/api/v4`. When running a [separate Pages server](#running-gitlab-pages-on-a-separate-server), this URL must point to the main GitLab server's API. | +| `auth_redirect_uri` | Callback URL for authenticating with GitLab. Defaults to project's subdomain of `pages_external_url` + `/auth`. | +| `auth_secret` | Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration. | +| `dir` | Working directory for configuration and secrets files. | +| `enable` | Enable or disable GitLab Pages on the current system. | +| `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`. | +| `external_https` | Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_https`. | +| `server_shutdown_timeout` | GitLab Pages server shutdown timeout in seconds (default: `30s`). | +| `gitlab_client_http_timeout` | GitLab API HTTP client connection timeout in seconds (default: `10s`). | +| `gitlab_client_jwt_expiry` | JWT Token expiry time in seconds (default: `30s`). | +| `gitlab_cache_expiry` | The maximum time a domain's configuration is stored in the cache (default: `600s`). | +| `gitlab_cache_refresh` | The interval at which a domain's configuration is set to be due to refresh (default: `60s`). | +| `gitlab_cache_cleanup` | The interval at which expired items are removed from the cache (default: `60s`). | +| `gitlab_retrieval_timeout` | The maximum time to wait for a response from the GitLab API per request (default: `30s`). | +| `gitlab_retrieval_interval` | The interval to wait before retrying to resolve a domain's configuration via the GitLab API (default: `1s`). | +| `gitlab_retrieval_retries` | The maximum number of times to retry to resolve a domain's configuration via the API (default: 3). | +| `domain_config_source` | This parameter was removed in 14.0, on earlier versions it can be used to enable and test API domain configuration source | +| `gitlab_id` | The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. | +| `gitlab_secret` | The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. | +| `auth_scope` | The OAuth application scope to use for authentication. Must match GitLab Pages OAuth application settings. Leave blank to use `api` scope by default. | +| `auth_cookie_session_timeout` | Authentication cookie session timeout in seconds (default: `10m`). A value of `0` means the cookie is deleted after the browser session ends. | +| `gitlab_server` | Server to use for authentication when access control is enabled; defaults to GitLab `external_url`. | +| `headers` | Specify any additional http headers that should be sent to the client with each response. Multiple headers can be given as an array, header and value as one string, for example `['my-header: myvalue', 'my-other-header: my-other-value']` | +| `enable_disk` | Allows the GitLab Pages daemon to serve content from disk. Shall be disabled if shared disk storage isn't available. | +| `insecure_ciphers` | Use default list of cipher suites, may contain insecure ones like 3DES and RC4. | +| `internal_gitlab_server` | Internal GitLab server address used exclusively for API requests. Useful if you want to send that traffic over an internal load balancer. Defaults to GitLab `external_url`. | +| `listen_proxy` | The addresses to listen on for reverse-proxy requests. Pages binds to these addresses' network sockets and receives incoming requests from them. Sets the value of `proxy_pass` in `$nginx-dir/conf/gitlab-pages.conf`. | +| `log_directory` | Absolute path to a log directory. | +| `log_format` | The log output format: `text` or `json`. | +| `log_verbose` | Verbose logging, true/false. | +| `propagate_correlation_id` | Set to true (false by default) to re-use existing Correlation ID from the incoming request header `X-Request-ID` if present. If a reverse proxy sets this header, the value is propagated in the request chain. | +| `max_connections` | Limit on the number of concurrent connections to the HTTP, HTTPS or proxy listeners. | | `max_uri_length` | The maximum length of URIs accepted by GitLab Pages. Set to 0 for unlimited length. [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5729) in GitLab 14.5. -| `metrics_address` | The address to listen on for metrics requests. | -| `redirect_http` | Redirect pages from HTTP to HTTPS, true/false. | -| `redirects_max_config_size` | The maximum size of the `_redirects` file, in bytes (default: 65536). | -| `redirects_max_path_segments` | The maximum number of path segments allowed in `_redirects` rules URLs (default: 25). | -| `redirects_max_rule_count` | The maximum number of rules allowed in `_redirects` (default: 1000). | -| `sentry_dsn` | The address for sending Sentry crash reporting to. | -| `sentry_enabled` | Enable reporting and logging with Sentry, true/false. | -| `sentry_environment` | The environment for Sentry crash reporting. | -| `status_uri` | The URL path for a status page, for example, `/@status`. | -| `tls_max_version` | Specifies the maximum TLS version ("tls1.2" or "tls1.3"). | -| `tls_min_version` | Specifies the minimum TLS version ("tls1.2" or "tls1.3"). | -| `use_http2` | Enable HTTP2 support. | -| **`gitlab_pages['env'][]`** | | -| `http_proxy` | Configure GitLab Pages to use an HTTP Proxy to mediate traffic between Pages and GitLab. Sets an environment variable `http_proxy` when starting Pages daemon. | -| **`gitlab_rails[]`** | | -| `pages_domain_verification_cron_worker` | Schedule for verifying custom GitLab Pages domains. | -| `pages_domain_ssl_renewal_cron_worker` | Schedule for obtaining and renewing SSL certificates through Let's Encrypt for GitLab Pages domains. | -| `pages_domain_removal_cron_worker` | Schedule for removing unverified custom GitLab Pages domains. | -| `pages_path` | The directory on disk where pages are stored, defaults to `GITLAB-RAILS/shared/pages`. | -| **`pages_nginx[]`** | | -| `enable` | Include a virtual host `server{}` block for Pages inside NGINX. Needed for NGINX to proxy traffic back to the Pages daemon. Set to `false` if the Pages daemon should directly receive all requests, for example, when using [custom domains](index.md#custom-domains). | -| `FF_ENABLE_PLACEHOLDERS` | Feature flag for rewrites (enabled by default). See [Rewrites](../../user/project/pages/redirects.md#rewrites) for more information. | -| `use_legacy_storage` | Temporarily-introduced parameter allowing to use legacy domain configuration source and storage. [Removed in 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6166). | -| `rate_limit_source_ip` | Rate limit per source IP in number of requests per second. Set to `0` to disable this feature. | -| `rate_limit_source_ip_burst` | Rate limit per source IP maximum burst allowed per second. | -| `rate_limit_domain` | Rate limit per domain in number of requests per second. Set to `0` to disable this feature. | -| `rate_limit_domain_burst` | Rate limit per domain maximum burst allowed per second. | -| `rate_limit_tls_source_ip` | Rate limit per source IP in number of TLS connections per second. Set to `0` to disable this feature. | -| `rate_limit_tls_source_ip_burst` | Rate limit per source IP maximum TLS connections burst allowed per second. | -| `rate_limit_tls_domain` | Rate limit per domain in number of TLS connections per second. Set to `0` to disable this feature. | -| `rate_limit_tls_domain_burst` | Rate limit per domain maximum TLS connections burst allowed per second. | -| `server_read_timeout` | Maximum duration to read the request headers and body. For no timeout, set to `0` or a negative value. Default: `5s` | -| `server_read_header_timeout` | Maximum duration to read the request headers. For no timeout, set to `0` or a negative value. Default: `1s` | -| `server_write_timeout` | Maximum duration to write all files in the response. Larger files require more time. For no timeout, set to `0` or a negative value. Default: `0` | -| `server_keep_alive` | The `Keep-Alive` period for network connections accepted by this listener. If `0`, `Keep-Alive` is enabled if supported by the protocol and operating system. If negative, `Keep-Alive` is disabled. Default: `15s` | +| `metrics_address` | The address to listen on for metrics requests. | +| `redirect_http` | Redirect pages from HTTP to HTTPS, true/false. | +| `redirects_max_config_size` | The maximum size of the `_redirects` file, in bytes (default: 65536). | +| `redirects_max_path_segments` | The maximum number of path segments allowed in `_redirects` rules URLs (default: 25). | +| `redirects_max_rule_count` | The maximum number of rules allowed in `_redirects` (default: 1000). | +| `sentry_dsn` | The address for sending Sentry crash reporting to. | +| `sentry_enabled` | Enable reporting and logging with Sentry, true/false. | +| `sentry_environment` | The environment for Sentry crash reporting. | +| `status_uri` | The URL path for a status page, for example, `/@status`. | +| `tls_max_version` | Specifies the maximum TLS version ("tls1.2" or "tls1.3"). | +| `tls_min_version` | Specifies the minimum TLS version ("tls1.2" or "tls1.3"). | +| `use_http2` | Enable HTTP2 support. | +| **`gitlab_pages['env'][]`** | | +| `http_proxy` | Configure GitLab Pages to use an HTTP Proxy to mediate traffic between Pages and GitLab. Sets an environment variable `http_proxy` when starting Pages daemon. | +| **`gitlab_rails[]`** | | +| `pages_domain_verification_cron_worker` | Schedule for verifying custom GitLab Pages domains. | +| `pages_domain_ssl_renewal_cron_worker` | Schedule for obtaining and renewing SSL certificates through Let's Encrypt for GitLab Pages domains. | +| `pages_domain_removal_cron_worker` | Schedule for removing unverified custom GitLab Pages domains. | +| `pages_path` | The directory on disk where pages are stored, defaults to `GITLAB-RAILS/shared/pages`. | +| **`pages_nginx[]`** | | +| `enable` | Include a virtual host `server{}` block for Pages inside NGINX. Needed for NGINX to proxy traffic back to the Pages daemon. Set to `false` if the Pages daemon should directly receive all requests, for example, when using [custom domains](index.md#custom-domains). | +| `FF_CONFIGURABLE_ROOT_DIR` | Feature flag to [customize the default folder](../../user/project/pages/introduction.md#customize-the-default-folder) (enabled by default). | +| `FF_ENABLE_PLACEHOLDERS` | Feature flag for rewrites (enabled by default). See [Rewrites](../../user/project/pages/redirects.md#rewrites) for more information. | +| `use_legacy_storage` | Temporarily-introduced parameter allowing to use legacy domain configuration source and storage. [Removed in 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6166). | +| `rate_limit_source_ip` | Rate limit per source IP in number of requests per second. Set to `0` to disable this feature. | +| `rate_limit_source_ip_burst` | Rate limit per source IP maximum burst allowed per second. | +| `rate_limit_domain` | Rate limit per domain in number of requests per second. Set to `0` to disable this feature. | +| `rate_limit_domain_burst` | Rate limit per domain maximum burst allowed per second. | +| `rate_limit_tls_source_ip` | Rate limit per source IP in number of TLS connections per second. Set to `0` to disable this feature. | +| `rate_limit_tls_source_ip_burst` | Rate limit per source IP maximum TLS connections burst allowed per second. | +| `rate_limit_tls_domain` | Rate limit per domain in number of TLS connections per second. Set to `0` to disable this feature. | +| `rate_limit_tls_domain_burst` | Rate limit per domain maximum TLS connections burst allowed per second. | +| `server_read_timeout` | Maximum duration to read the request headers and body. For no timeout, set to `0` or a negative value. Default: `5s` | +| `server_read_header_timeout` | Maximum duration to read the request headers. For no timeout, set to `0` or a negative value. Default: `1s` | +| `server_write_timeout` | Maximum duration to write all files in the response. Larger files require more time. For no timeout, set to `0` or a negative value. Default: `0` | +| `server_keep_alive` | The `Keep-Alive` period for network connections accepted by this listener. If `0`, `Keep-Alive` is enabled if supported by the protocol and operating system. If negative, `Keep-Alive` is disabled. Default: `15s` | ## Advanced configuration @@ -530,7 +530,7 @@ This usually results in this error: For installation from source, this can be fixed by installing the custom Certificate Authority (CA) in the system certificate store. -For Omnibus, this is fixed by [installing a custom CA in Omnibus GitLab](https://docs.gitlab.com/omnibus/settings/ssl/index.html#install-custom-public-certificates). +For Linux package installations, this is fixed by [installing a custom CA](https://docs.gitlab.com/omnibus/settings/ssl/index.html#install-custom-public-certificates). ### ZIP serving and cache configuration @@ -590,6 +590,27 @@ gitlab_pages['redirects_max_path_segments'] = 50 gitlab_pages['redirects_max_rule_count'] = 2000 ``` +## Use environment variables + +You can pass an environment variable to the Pages daemon (for example, +to enable or disable a feature flag). + +To disable the configurable directory feature: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['env'] = { + 'FF_CONFIGURABLE_ROOT_DIR' => "false" + } + ``` + +1. Save the file and reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + ## Activate verbose logging for daemon Follow the steps below to configure verbose logging of GitLab Pages daemon. @@ -698,10 +719,7 @@ Prerequisite: To set the maximum size of GitLab Pages site in a project, overriding the inherited setting: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. On the left sidebar, select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../../user/project/pages/index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. In **Maximum size of pages**, enter the size in MB. 1. Select **Save changes**. @@ -758,7 +776,7 @@ database encryption. Proceed with caution. gitlab_pages['access_control'] = true ``` -1. Configure [the object storage and migrate pages data to it](#using-object-storage). +1. Configure [the object storage and migrate pages data to it](#object-storage-settings). 1. [Reconfigure the **GitLab server**](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. The `gitlab-secrets.json` file is now updated with the @@ -766,7 +784,7 @@ database encryption. Proceed with caution. 1. Set up a new server. This becomes the **Pages server**. -1. On the **Pages server**, install Omnibus GitLab and modify `/etc/gitlab/gitlab.rb` +1. On the **Pages server**, install GitLab by using the Linux package and modify `/etc/gitlab/gitlab.rb` to include: ```ruby @@ -832,45 +850,12 @@ This approach had several disadvantages and was replaced with GitLab Pages using every time a new domain is requested. The domain information is also cached by the Pages daemon to speed up subsequent requests. -From [GitLab 13.3 to GitLab 13.12](#domain-source-configuration-before-140) GitLab Pages supported both ways of obtaining domain information. - Starting from [GitLab 14.0](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5993) GitLab Pages uses API by default and fails to start if it can't connect to it. For common issues, see [troubleshooting](troubleshooting.md#failed-to-connect-to-the-internal-gitlab-api). For more details see this [blog post](https://about.gitlab.com/blog/2020/08/03/how-gitlab-pages-uses-the-gitlab-api-to-serve-content/). -### Domain source configuration before 14.0 - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217912) in GitLab 13.3. - -WARNING: -`domain_config_source` parameter is removed and has no effect starting from [GitLab 14.0](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5993) - -From [GitLab 13.3](https://gitlab.com/gitlab-org/gitlab/-/issues/217912) to [GitLab 13.12](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5993) GitLab Pages can either use `disk` or `gitlab` domain configuration source. - -We highly advise you to use `gitlab` configuration source as it makes transitions to newer versions easier. - -To explicitly enable API source: - -1. Add the following to your `/etc/gitlab/gitlab.rb` file: - - ```ruby - gitlab_pages['domain_config_source'] = "gitlab" - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. - -Or if you want to use legacy configuration source you can: - -1. Add the following to your `/etc/gitlab/gitlab.rb` file: - - ```ruby - gitlab_pages['domain_config_source'] = "disk" - ``` - -1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. - ### GitLab API cache configuration > [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/520) in GitLab 13.10. @@ -909,18 +894,12 @@ only when there is an error response from the API, for example a connection time - Decreasing `gitlab_retrieval_retries` reduces the number of times a domain's configuration is tried to be resolved automatically before reporting an error. -## Using object storage - -> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5577) in GitLab 13.6. +## Object storage settings -For more information, see [object storage](../object_storage.md). +The following [object storage](../object_storage.md) settings are: -### Object storage settings - -The following settings are: - -- Nested under `pages:` and then `object_store:` on source installations. -- Prefixed by `pages_object_store_` on Omnibus GitLab installations. +- Nested under `pages:` and then `object_store:` on self-compiled installations. +- Prefixed by `pages_object_store_` on Linux package installations. | Setting | Description | Default | |---------|-------------|---------| @@ -932,7 +911,7 @@ NOTE: If you want to stop using and disconnect the NFS server, you need to [explicitly disable local storage](#disable-pages-local-storage), and it's only possible after upgrading to GitLab 13.11. -#### S3-compatible connection settings +### S3-compatible connection settings In GitLab 13.2 and later, you should use the [consolidated object storage settings](../object_storage.md#configure-a-single-storage-connection-for-all-object-types-consolidated-form). @@ -940,7 +919,9 @@ This section describes the earlier configuration format. See [the available connection settings for different providers](../object_storage.md#configure-the-connection-settings). -In Omnibus installations: +::Tabs + +:::TabTitle Linux package (Omnibus) 1. Add the following lines to `/etc/gitlab/gitlab.rb` and replace the values with the ones you want: @@ -971,7 +952,7 @@ In Omnibus installations: 1. [Migrate existing Pages deployments to object storage.](#migrate-pages-deployments-to-object-storage) -In installations from source: +:::TabTitle Self-compiled (source) 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: @@ -992,75 +973,14 @@ In installations from source: 1. [Migrate existing Pages deployments to object storage.](#migrate-pages-deployments-to-object-storage) -## ZIP storage - -In GitLab 14.0 the underlying storage format of GitLab Pages is changing from -files stored directly in disk to a single ZIP archive per project. - -These ZIP archives can be stored either locally on disk storage or on [object storage](#using-object-storage) if it is configured. - -[Starting from GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/245308) ZIP archives are stored every time pages site is updated. - -### Migrate legacy storage to ZIP storage - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59003) in GitLab 13.11. - -GitLab tries to -[automatically migrate](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/54578) -the old storage format to the new ZIP-based one when you upgrade to GitLab 13.11 or further. -However, some projects may fail to be migrated for different reasons. -To verify that all projects have been migrated successfully, you can manually run the migration: - -```shell -sudo gitlab-rake gitlab:pages:migrate_legacy_storage -``` - -It's safe to interrupt this task and run it multiple times. - -There are two most common problems this task can report: - -- `Missing public directory` error: - - ```txt - E, [2021-04-09T13:11:52.534768 #911919] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated in 0.07 seconds: Archive not created. Missing public directory in /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test - ``` - - In this case, you should verify that these projects don't have pages deployed, and re-run the migration with an additional flag to mark those projects as not deployed with GitLab Pages: - - ```shell - sudo PAGES_MIGRATION_MARK_PROJECTS_AS_NOT_DEPLOYED=true gitlab-rake gitlab:pages:migrate_legacy_storage - ``` - -- File `is invalid` error: - - ```txt - E, [2021-04-09T14:43:05.821767 #923322] ERROR -- : project_id: 1 /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test failed to be migrated: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test/public/link is invalid, input_dir: /home/vlad/gdk/gitlab/shared/pages/gitlab-org/gitlab-test - ``` - - This error indicates invalid files on disk storage, most commonly symlinks leading outside of the `public` directory. - You can manually remove these files, or just ignore them during migration: - - ```shell - sudo PAGES_MIGRATION_IGNORE_INVALID_ENTRIES=true gitlab-rake gitlab:pages:migrate_legacy_storage - ``` - -### Rolling back ZIP migration - -If you find that migrated data is invalid, you can remove all migrated data by running: - -```shell -sudo gitlab-rake gitlab:pages:clean_migrated_zip_storage -``` - -This does not remove any data from the legacy disk storage and the GitLab Pages daemon automatically falls back -to using that. +::EndTabs ### Migrate Pages deployments to object storage -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325285) in GitLab 13.11. +Existing Pages deployment objects (zip archives) can be stored in either: -Existing Pages deployment objects (which store [ZIP archives](#zip-storage)) can similarly be -migrated to [object storage](#using-object-storage). +- Local storage +- [Object storage](../object_storage.md) Migrate your existing Pages deployments from local storage to object storage: @@ -1071,9 +991,9 @@ sudo gitlab-rake gitlab:pages:deployments:migrate_to_object_storage You can track progress and verify that all Pages deployments migrated successfully using the [PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database): -- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier. -- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later. -- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. +- `sudo gitlab-rails dbconsole` for Linux package installations running GitLab 14.1 and earlier. +- `sudo gitlab-rails dbconsole --database main` for Linux package installations running 14.2 and later. +- `sudo -u git -H psql -d gitlabhq_production` for self-compiled installations. Verify `objectstg` below (where `store=2`) has count of all Pages deployments: @@ -1100,7 +1020,7 @@ sudo gitlab-rake gitlab:pages:deployments:migrate_to_local > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301159) in GitLab 13.11. -If you use [object storage](#using-object-storage), you can disable local storage to avoid unnecessary disk usage/writes: +If you use [object storage](#object-storage-settings), you can disable local storage to avoid unnecessary disk usage/writes: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -1110,34 +1030,18 @@ If you use [object storage](#using-object-storage), you can disable local storag 1. [Reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux-package-installation) for the changes to take effect. -Starting from GitLab 13.12, this setting also disables the [legacy storage](#migrate-legacy-storage-to-zip-storage), so if you were using NFS to serve Pages, you can completely disconnect from it. - -## Prepare GitLab Pages for 14.0 - -In GitLab 14.0 a number of breaking changes were introduced which may require some user intervention. -The steps below describe the best way to migrate without causing any downtime for your GitLab instance. +## ZIP storage -A GitLab instance running on a single server typically upgrades to 14.0 smoothly, and there should be minimal issues after the upgrade is complete. -Regardless, you should follow the migration steps to ensure a successful upgrade. -For common issues, see [troubleshooting](troubleshooting.md). +In GitLab 14.0 the underlying storage format of GitLab Pages changed from +files stored directly in disk to a single ZIP archive per project. -If your current GitLab version is lower than 13.12, then you must first update to 13.12. -Updating directly to 14.0 is [not supported](../../update/index.md#upgrade-paths) -and may cause downtime for some web-sites hosted on GitLab Pages. After you update to 13.12, -migrate GitLab Pages to prepare them for GitLab 14.0: +These ZIP archives can be stored either locally on disk storage or on [object storage](#object-storage-settings) if it is configured. -1. Set [`domain_config_source` to `gitlab`](#domain-source-configuration-before-140), which -is the default starting from GitLab 14.0. Skip this step if you're already running GitLab 14.0 or above. -1. If you want to store your pages content in [object storage](#using-object-storage), make sure to configure it. -If you want to store the pages content locally or continue using an NFS server, skip this step. -1. [Migrate legacy storage to ZIP storage.](#migrate-legacy-storage-to-zip-storage) -1. If you have configured GitLab to store your pages content in [object storage](#using-object-storage), - [migrate Pages deployments to object storage](#migrate-pages-deployments-to-object-storage) -1. Upgrade GitLab to 14.0. +[Starting from GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/245308) ZIP archives are stored every time pages site is updated. ## Backup -GitLab Pages are part of the [regular backup](../../raketasks/backup_restore.md), so there is no separate backup to configure. +GitLab Pages are part of the [regular backup](../../administration/backup_restore/index.md), so there is no separate backup to configure. ## Security diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index 2ee9dd653f0..26dedd47473 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -4,21 +4,17 @@ group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab Pages administration for source installations **(FREE SELF)** +# GitLab Pages administration for self-compiled installations **(FREE SELF)** NOTE: Before attempting to enable GitLab Pages, first make sure you have [installed GitLab](../../install/installation.md) successfully. -This document explains how to configure GitLab Pages when you have installed -GitLab from source and not the Omnibus packages. +This document explains how to configure GitLab Pages for self-compiled GitLab installations. -You are encouraged to read the [Omnibus documentation](index.md) as it provides -invaluable information about the configuration of GitLab Pages. +For more information about configuring GitLab Pages for Linux Package installations (recommended), see the [Linux package documentation](index.md). -We also highly recommend that you use the Omnibus GitLab packages. We -optimize them specifically for GitLab, and we take care of upgrading GitLab -Pages to the latest supported version. +The advantage of using the Linux package installation is that it contains the latest supported version of GitLab Pages. ## How GitLab Pages works @@ -491,7 +487,7 @@ To change this value: ## Backup -Pages are part of the [regular backup](../../raketasks/backup_restore.md) so there is nothing to configure. +Pages are part of the [regular backup](../../administration/backup_restore/index.md) so there is nothing to configure. ## Security diff --git a/doc/administration/pages/troubleshooting.md b/doc/administration/pages/troubleshooting.md index e829943f270..7e184631515 100644 --- a/doc/administration/pages/troubleshooting.md +++ b/doc/administration/pages/troubleshooting.md @@ -250,7 +250,7 @@ the shared pages directory is mounted on a different path on the main GitLab ser GitLab Pages server. In that case, it's highly recommended you to configure -[object storage and migrate any existing pages data to it](index.md#using-object-storage). +[object storage and migrate any existing pages data to it](index.md#object-storage-settings). Alternatively, you can mount the GitLab Pages shared directory to the same path on both servers. @@ -259,7 +259,7 @@ both servers. GitLab 14.0 introduces a number of changes to GitLab Pages which may require manual intervention. -1. Firstly [follow the migration guide](index.md#prepare-gitlab-pages-for-140). +1. Firstly [follow the migration guide](https://archives.docs.gitlab.com/14.10/ee/administration/pages/#prepare-gitlab-pages-for-140). 1. Try to upgrade to GitLab 14.3 or above. Some of the issues were fixed in GitLab 14.1, 14.2 and 14.3. 1. If it doesn't work, see [GitLab Pages logs](#how-to-see-gitlab-pages-logs), and if you see any errors there then search them on this page. diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md index cc550dbe389..f8b6be1fb21 100644 --- a/doc/administration/postgresql/database_load_balancing.md +++ b/doc/administration/postgresql/database_load_balancing.md @@ -64,7 +64,7 @@ To enable Database Load Balancing, make sure that: - The HA PostgreSQL setup has one or more secondary nodes replicating the primary. - Each PostgreSQL node is connected with the same credentials and on the same port. -For Omnibus GitLab, you also need PgBouncer configured on each PostgreSQL node to pool +For Linux package installations, you also need PgBouncer configured on each PostgreSQL node to pool all load-balanced connections when [configuring a multi-node setup](replication_and_failover.md). ## Configuring Database Load Balancing @@ -112,7 +112,7 @@ checks a DNS `A` record, using the IPs returned by this record as the addresses for the secondaries. For service discovery to work, all you need is a DNS server and an `A` record containing the IP addresses of your secondaries. -When using Omnibus GitLab the provided [Consul](../consul.md) service works as +When using a Linux package installation, the provided [Consul](../consul.md) service works as a DNS server and returns PostgreSQL addresses via the `postgresql-ha.service.consul` record. For example: diff --git a/doc/administration/postgresql/external.md b/doc/administration/postgresql/external.md index 8f664f9809e..9c44d53213b 100644 --- a/doc/administration/postgresql/external.md +++ b/doc/administration/postgresql/external.md @@ -11,7 +11,7 @@ managed service for PostgreSQL. For example, AWS offers a managed Relational Database Service (RDS) that runs PostgreSQL. Alternatively, you may opt to manage your own PostgreSQL instance or cluster -separate from the Omnibus GitLab package. +separate from the Linux package. If you use a cloud-managed service, or provide your own PostgreSQL instance: diff --git a/doc/administration/postgresql/index.md b/doc/administration/postgresql/index.md index e829397abc1..af0a86c3d72 100644 --- a/doc/administration/postgresql/index.md +++ b/doc/administration/postgresql/index.md @@ -13,32 +13,32 @@ be used with GitLab in one of our [reference architectures](../reference_archite Choose one of the following PostgreSQL configuration options: -### Standalone PostgreSQL using Omnibus GitLab +### Standalone PostgreSQL for Linux package installations -This setup is for when you have installed the -[Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), +This setup is for when you have installed GitLab by using the +[Linux package](https://about.gitlab.com/install/) (CE or EE), to use the bundled PostgreSQL having only its service enabled. -Read how to [set up a standalone PostgreSQL instance](standalone.md) using Omnibus GitLab. +Read how to [set up a standalone PostgreSQL instance](standalone.md) for Linux package installations. ### Provide your own PostgreSQL instance This setup is for when you have installed GitLab using the -[Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), -or installed it [from source](../../install/installation.md), but you want to use +[Linux package](https://about.gitlab.com/install/) (CE or EE), +or [self-compiled](../../install/installation.md) your installation, but you want to use your own external PostgreSQL server. Read how to [set up an external PostgreSQL instance](external.md). -### PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** +### PostgreSQL replication and failover for Linux package installations **(PREMIUM SELF)** This setup is for when you have installed GitLab using the -[Omnibus GitLab **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). +[Linux **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). All the tools that are needed like PostgreSQL, PgBouncer, and Patroni are bundled in the package, so you can use it to set up the whole PostgreSQL infrastructure (primary, replica). -Read how to [set up PostgreSQL replication and failover](replication_and_failover.md) using Omnibus GitLab. +Read how to [set up PostgreSQL replication and failover](replication_and_failover.md) for Linux package installations. ## Related topics diff --git a/doc/administration/postgresql/multiple_databases.md b/doc/administration/postgresql/multiple_databases.md index f03781d0ee2..5dcb080d707 100644 --- a/doc/administration/postgresql/multiple_databases.md +++ b/doc/administration/postgresql/multiple_databases.md @@ -65,7 +65,7 @@ copy the database across. 1. Configure GitLab to [use multiple databases](#set-up-multiple-databases). -### Existing Omnibus installation +### Existing Linux package installations 1. Stop GitLab, except for PostgreSQL: @@ -103,7 +103,7 @@ the other way around. 1. For existing installations, [migrate the data](#migrate-existing-installations) first. -1. [Back up GitLab](../../raketasks/backup_restore.md) +1. [Back up GitLab](../../administration/backup_restore/index.md) in case of unforeseen issues. 1. Stop GitLab: @@ -152,12 +152,12 @@ the other way around. sudo service gitlab restart ``` -### Omnibus GitLab installations +### Linux package installations 1. For existing installations, [migrate the data](#migrate-existing-installations) first. -1. [Back up GitLab](../../raketasks/backup_restore.md) +1. [Back up GitLab](../../administration/backup_restore/index.md) in case of unforeseen issues. 1. Stop GitLab: diff --git a/doc/administration/postgresql/pgbouncer.md b/doc/administration/postgresql/pgbouncer.md index 59aac9141a4..7e46933113b 100644 --- a/doc/administration/postgresql/pgbouncer.md +++ b/doc/administration/postgresql/pgbouncer.md @@ -25,9 +25,9 @@ This content has been moved to a [new location](replication_and_failover.md#conf ## PgBouncer as part of a non-fault-tolerant GitLab installation -1. Generate PGBOUNCER_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 pgbouncer` +1. Generate `PGBOUNCER_USER_PASSWORD_HASH` with the command `gitlab-ctl pg-password-md5 pgbouncer` -1. Generate SQL_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 gitlab`. Enter the plaintext SQL_USER_PASSWORD later. +1. Generate `SQL_USER_PASSWORD_HASH` with the command `gitlab-ctl pg-password-md5 gitlab`. Enter the plaintext SQL_USER_PASSWORD later. 1. On your database node, ensure the following is set in your `/etc/gitlab/gitlab.rb` @@ -88,7 +88,7 @@ This content has been moved to a [new location](replication_and_failover.md#conf Do not backup or restore GitLab through a PgBouncer connection: it causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer). ## Enable Monitoring @@ -119,7 +119,7 @@ If you enable Monitoring, it must be enabled on **all** PgBouncer servers. ## Administrative console -As part of Omnibus GitLab, a command is provided to automatically connect to the +In Linux package installations, a command is provided to automatically connect to the PgBouncer administrative console. See the [PgBouncer documentation](https://www.pgbouncer.org/usage.html#admin-console) for detailed instructions on how to interact with the console. @@ -174,11 +174,11 @@ ote_pid | tls ## Procedure for bypassing PgBouncer -### Omnibus installations +### Linux package installations Some database changes have to be done directly, and not through PgBouncer. -The main affected tasks are [database restores](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) +The main affected tasks are [database restores](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer) and [GitLab upgrades with database migrations](../../update/zero_downtime.md#postgresql). 1. To find the primary node, run the following on a database node: @@ -208,8 +208,8 @@ After you've performed the tasks or procedure, switch back to using PgBouncer: ### Helm chart installations -High-availability deployments also need to bypass PgBouncer for the same reasons as Omnibus-based ones. -For this type of installation: +High-availability deployments also need to bypass PgBouncer for the same reasons as Linux package-based ones. +For Helm chart installations: - Database backup and restore tasks are performed by the toolbox container. - Migration tasks are performed by the migrations container. @@ -226,7 +226,7 @@ In specific cases you may want to change the performance-specific and resource-s throughput or to limit resource utilization that could cause memory exhaustion on the database. You can find the parameters and respective documentation on the [official PgBouncer documentation](https://www.pgbouncer.org/config.html). -Listed below are the most relevant ones and their defaults on an Omnibus GitLab installation: +Listed below are the most relevant ones and their defaults on a Linux package installation: - `pgbouncer['max_client_conn']` (default: `2048`, depends on server file descriptor limits) This is the "frontend" pool in PgBouncer: connections from Rails to PgBouncer. diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md index 392f9f2b89c..05ff6fa8a4a 100644 --- a/doc/administration/postgresql/replication_and_failover.md +++ b/doc/administration/postgresql/replication_and_failover.md @@ -4,21 +4,21 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** +# PostgreSQL replication and failover for Linux package installations **(PREMIUM SELF)** If you're a Free user of GitLab self-managed, consider using a cloud-hosted solution. This document doesn't cover installations from source. If a setup with replication and failover isn't what you were looking for, see the [database configuration document](https://docs.gitlab.com/omnibus/settings/database.html) -for the Omnibus GitLab packages. +for the Linux packages. It's recommended to read this document fully before attempting to configure PostgreSQL with replication and failover for GitLab. ## Architecture -The Omnibus GitLab recommended configuration for a PostgreSQL cluster with +The Linux pacakage-recommended configuration for a PostgreSQL cluster with replication failover requires: - A minimum of three PostgreSQL nodes. @@ -73,9 +73,9 @@ sure you have redundant connectivity between all Database and GitLab instances to avoid the network becoming a single point of failure. NOTE: -As of GitLab 13.3, PostgreSQL 12 is shipped with Omnibus GitLab. Clustering for PostgreSQL 12 is supported only with +As of GitLab 13.3, PostgreSQL 12 is shipped with Linux package installations. Clustering for PostgreSQL 12 is supported only with Patroni. See the [Patroni](#patroni) section for further details. Starting with GitLab 14.0, only PostgreSQL 12 is -shipped with Omnibus GitLab, and thus Patroni becomes mandatory for replication and failover. +shipped with Linux package installations, and thus Patroni becomes mandatory for replication and failover. ### Database node @@ -152,7 +152,7 @@ This is why you need: When using default setup, minimum configuration requires: -- `CONSUL_USERNAME`. The default user for Omnibus GitLab is `gitlab-consul` +- `CONSUL_USERNAME`. The default user for Linux package installations is `gitlab-consul` - `CONSUL_DATABASE_PASSWORD`. Password for the database user. - `CONSUL_PASSWORD_HASH`. This is a hash generated out of Consul username/password pair. It can be generated with: @@ -192,7 +192,7 @@ server nodes on hand. You need the following password information for the application's database user: -- `POSTGRESQL_USERNAME`. The default user for Omnibus GitLab is `gitlab` +- `POSTGRESQL_USERNAME`. The default user for Linux package installations is `gitlab` - `POSTGRESQL_USER_PASSWORD`. The password for the database user - `POSTGRESQL_PASSWORD_HASH`. This is a hash generated out of the username/password pair. It can be generated with: @@ -212,7 +212,7 @@ You need the following password information for the Patroni API: When using a default setup, the minimum configuration requires: -- `PGBOUNCER_USERNAME`. The default user for Omnibus GitLab is `pgbouncer` +- `PGBOUNCER_USERNAME`. The default user for Linux package installations is `pgbouncer` - `PGBOUNCER_PASSWORD`. This is a password for PgBouncer service. - `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of PgBouncer username/password pair. It can be generated with: @@ -230,10 +230,9 @@ Few things to remember about the service itself: - `/etc/gitlab/gitlab.rb`: hashed, and in plain text - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed -### Installing Omnibus GitLab +### Installing the Linux package -First, make sure to [download/install](https://about.gitlab.com/install/) -Omnibus GitLab **on each node**. +First, make sure to [download and install](https://about.gitlab.com/install/) the Linux package **on each node**. Make sure you install the necessary dependencies from step 1, add GitLab package repository from step 2. @@ -373,7 +372,7 @@ patroni['tls_key_password'] = 'private-key-password' # This is the plain-text pa ``` If you are using a self-signed certificate or an internal CA, you need to either disable the TLS verification or pass the certificate of the -internal CA, otherwise you may run into an unexpected error when using the `gitlab-ctl patroni ....` commands. Omnibus ensures that Patroni API +internal CA, otherwise you may run into an unexpected error when using the `gitlab-ctl patroni ....` commands. The Linux package ensures that Patroni API clients honor this configuration. TLS certificate verification is enabled by default. To disable it: @@ -415,7 +414,7 @@ authentication mode (`patroni['tls_client_mode']`), must each have the same valu 1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul-information), [`CONSUL_PASSWORD_HASH`](#consul-information), and [`PGBOUNCER_PASSWORD_HASH`](#pgbouncer-information) before executing the next step. -1. One each node, edit the `/etc/gitlab/gitlab.rb` configuration file and replace values noted in the `# START user configuration` section as below: +1. On each node, edit the `/etc/gitlab/gitlab.rb` configuration file and replace values noted in the `# START user configuration` section as below: ```ruby # Disable all components except PgBouncer and Consul agent @@ -571,7 +570,7 @@ in the Troubleshooting section before proceeding. Do not backup or restore GitLab through a PgBouncer connection: this causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../administration/backup_restore/backup_gitlab.md#back-up-and-restore-for-installations-using-pgbouncer). ### Ensure GitLab is running @@ -837,7 +836,7 @@ Read more about the data returned by the replica ### Selecting the appropriate Patroni replication method -[Review the Patroni documentation carefully](https://patroni.readthedocs.io/en/latest/SETTINGS.html#postgresql) +[Review the Patroni documentation carefully](https://patroni.readthedocs.io/en/latest/yaml_configuration.html#postgresql) before making changes as **_some of the options carry a risk of potential data loss if not fully understood_**. The [replication mode](https://patroni.readthedocs.io/en/latest/replication_modes.html) configured determines the amount of tolerable data loss. @@ -845,7 +844,7 @@ configured determines the amount of tolerable data loss. WARNING: Replication is not a backup strategy! There is no replacement for a well-considered and tested backup solution. -Omnibus GitLab defaults [`synchronous_commit`](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT) to `on`. +Linux package installations default [`synchronous_commit`](https://www.postgresql.org/docs/11/runtime-config-wal.html#GUC-SYNCHRONOUS-COMMIT) to `on`. ```ruby postgresql['synchronous_commit'] = 'on' @@ -854,7 +853,7 @@ gitlab['geo-postgresql']['synchronous_commit'] = 'on' #### Customizing Patroni failover behavior -Omnibus GitLab exposes several options allowing more control over the [Patroni restoration process](#recovering-the-patroni-cluster). +Linux package installations expose several options allowing more control over the [Patroni restoration process](#recovering-the-patroni-cluster). Each option is shown below with its default value in `/etc/gitlab/gitlab.rb`. @@ -864,7 +863,7 @@ patroni['remove_data_directory_on_rewind_failure'] = false patroni['remove_data_directory_on_diverged_timelines'] = false ``` -[The upstream documentation is always more up to date](https://patroni.readthedocs.io/en/latest/SETTINGS.html#postgresql), but the table below should provide a minimal overview of functionality. +[The upstream documentation is always more up to date](https://patroni.readthedocs.io/en/latest/patroni_configuration.html), but the table below should provide a minimal overview of functionality. |Setting|Overview| |-|-| @@ -996,7 +995,9 @@ You can switch an exiting database cluster to use Patroni instead of repmgr with ### Upgrading PostgreSQL major version in a Patroni cluster -As of GitLab 14.1, PostgreSQL 12.6 and 13.3 are both shipped with Omnibus GitLab by default. As of GitLab 15.0, PostgreSQL 13 is the default. If you want to upgrade to PostgreSQL 13 in versions prior to GitLab 15.0, you must ask for it explicitly. +As of GitLab 14.1, PostgreSQL 12.6 and 13.3 are both shipped with the Linux package by default. As of GitLab 15.0, +PostgreSQL 13 is the default. If you want to upgrade to PostgreSQL 13 in versions prior to GitLab 15.0, you must ask for +it explicitly. WARNING: The procedure for upgrading PostgreSQL in a Patroni cluster is different than when upgrading using repmgr. @@ -1466,9 +1467,9 @@ Workarounds: - If set to enforcing, SELinux may also prevent these operations. Verify the issue is fixed by setting SELinux to permissive. -Patroni first shipped in Omnibus GitLab 13.1, along with a build of Python 3.7. +Patroni first shipped in the Linux package for GitLab 13.1, along with a build of Python 3.7. The code which causes this was removed in Python 3.8: this fix shipped in -[Omnibus GitLab 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5547) +[the Linux package for GitLab 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5547) and later, removing the need for a workaround. ### Errors running `gitlab-ctl` diff --git a/doc/administration/postgresql/standalone.md b/doc/administration/postgresql/standalone.md index d00310ecee0..d6f3460e255 100644 --- a/doc/administration/postgresql/standalone.md +++ b/doc/administration/postgresql/standalone.md @@ -4,19 +4,19 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Standalone PostgreSQL using Omnibus GitLab **(FREE SELF)** +# Standalone PostgreSQL for Linux package installations **(FREE SELF)** If you wish to have your database service hosted separately from your GitLab application servers, you can do this using the PostgreSQL binaries packaged -together with Omnibus GitLab. This is recommended as part of our +together with the Linux package. This is recommended as part of our [reference architecture for up to 2,000 users](../reference_architectures/2k_users.md). ## Setting it up 1. SSH in to the PostgreSQL server. -1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab - package you want using *steps 1 and 2* from the GitLab downloads page. - - Do not complete any other steps on the download page. +1. [Download and install](https://about.gitlab.com/install/) the Linux + package you want using *steps 1 and 2* from the GitLab downloads page. Do not complete any other steps on the + download page. 1. Generate a password hash for PostgreSQL. This assumes you are using the default username of `gitlab` (recommended). The command requests a password and confirmation. Use the value that is output by this command in the next diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index e55a0f1c8a7..9ced19b53b7 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -42,17 +42,17 @@ This task loops through the project code repositories and runs the integrity che described previously. If a project uses a pool repository, that is also checked. Other types of Git repositories [are not checked](https://gitlab.com/gitlab-org/gitaly/-/issues/3643). -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:git:fsck -``` + ```shell + sudo gitlab-rake gitlab:git:fsck + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production + ``` ## Checksum of repository refs @@ -73,17 +73,17 @@ checksums in the format `<PROJECT ID>,<CHECKSUM>`. - If a repository exists but is empty, the output checksum is `0000000000000000000000000000000000000000`. - Projects which don't exist are skipped. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:git:checksum_projects -``` + ```shell + sudo gitlab-rake gitlab:git:checksum_projects + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake gitlab:git:checksum_projects RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake gitlab:git:checksum_projects RAILS_ENV=production + ``` For example, if: @@ -124,23 +124,23 @@ Integrity checks are supported for the following types of file: - Project-level Secure Files (introduced in GitLab 16.1.0) - User uploads (introduced in GitLab 10.6.0) -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:artifacts:check -sudo gitlab-rake gitlab:ci_secure_files:check -sudo gitlab-rake gitlab:lfs:check -sudo gitlab-rake gitlab:uploads:check -``` + ```shell + sudo gitlab-rake gitlab:artifacts:check + sudo gitlab-rake gitlab:ci_secure_files:check + sudo gitlab-rake gitlab:lfs:check + sudo gitlab-rake gitlab:uploads:check + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake gitlab:artifacts:check RAILS_ENV=production -sudo -u git -H bundle exec rake gitlab:ci_secure_files:check RAILS_ENV=production -sudo -u git -H bundle exec rake gitlab:lfs:check RAILS_ENV=production -sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake gitlab:artifacts:check RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:ci_secure_files:check RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:lfs:check RAILS_ENV=production + sudo -u git -H bundle exec rake gitlab:uploads:check RAILS_ENV=production + ``` These tasks also accept some environment variables which you can use to override certain values: @@ -215,22 +215,22 @@ secrets file (`gitlab-secrets.json`). Automatic resolution is not yet implemented. If you have values that cannot be decrypted, you can follow steps to reset them, see our -documentation on what to do [when the secrets file is lost](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost). +documentation on what to do [when the secrets file is lost](../../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost). This can take a very long time, depending on the size of your database, as it checks all rows in all tables. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:doctor:secrets -``` + ```shell + sudo gitlab-rake gitlab:doctor:secrets + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:doctor:secrets RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:doctor:secrets RAILS_ENV=production + ``` **Example output** @@ -251,17 +251,17 @@ I, [2020-06-11T17:18:15.575711 #27148] INFO -- : Done! To get more detailed information about which rows and columns can't be decrypted, you can pass a `VERBOSE` environment variable: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1 -``` + ```shell + sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1 + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:doctor:secrets RAILS_ENV=production VERBOSE=1 -``` + ```shell + bundle exec rake gitlab:doctor:secrets RAILS_ENV=production VERBOSE=1 + ``` **Example verbose output** diff --git a/doc/administration/raketasks/geo.md b/doc/administration/raketasks/geo.md index 03b09e00f1c..c6bc891f529 100644 --- a/doc/administration/raketasks/geo.md +++ b/doc/administration/raketasks/geo.md @@ -18,17 +18,17 @@ next repository sync in a **secondary** node: This is equivalent of running `git repack -d` on a _bare_ repository. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake geo:git:housekeeping:incremental_repack -``` + ```shell + sudo gitlab-rake geo:git:housekeeping:incremental_repack + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake geo:git:housekeeping:incremental_repack RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake geo:git:housekeeping:incremental_repack RAILS_ENV=production + ``` ### Full Repack @@ -36,48 +36,48 @@ This is equivalent of running `git repack -d -A --pack-kept-objects` on a _bare_ repository which optionally, writes a reachability bitmap index when this is enabled in GitLab. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake geo:git:housekeeping:full_repack -``` + ```shell + sudo gitlab-rake geo:git:housekeeping:full_repack + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake geo:git:housekeeping:full_repack RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake geo:git:housekeeping:full_repack RAILS_ENV=production + ``` ### GC This is equivalent of running `git gc` on a _bare_ repository, optionally writing a reachability bitmap index when this is enabled in GitLab. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake geo:git:housekeeping:gc -``` + ```shell + sudo gitlab-rake geo:git:housekeeping:gc + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake geo:git:housekeeping:gc RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake geo:git:housekeeping:gc RAILS_ENV=production + ``` ## Remove orphaned project registries Under certain conditions your project registry can contain obsolete records, you can remove them using the Rake task `geo:run_orphaned_project_registry_cleaner`: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake geo:run_orphaned_project_registry_cleaner -``` + ```shell + sudo gitlab-rake geo:run_orphaned_project_registry_cleaner + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake geo:run_orphaned_project_registry_cleaner RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake geo:run_orphaned_project_registry_cleaner RAILS_ENV=production + ``` diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md index f6c5f84c500..4a657b04bdc 100644 --- a/doc/administration/raketasks/ldap.md +++ b/doc/administration/raketasks/ldap.md @@ -15,17 +15,17 @@ The LDAP check Rake task tests the `bind_dn` and `password` credentials executed as part of the `gitlab:check` task, but can run independently using the command below. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:check -``` + ```shell + sudo gitlab-rake gitlab:ldap:check + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake gitlab:ldap:check RAILS_ENV=production + ``` By default, the task returns a sample of 100 LDAP users. Change this limit by passing a number to the check task: @@ -47,17 +47,17 @@ If you'd like to change the frequency at which a group sync is performed, [adjust the cron schedule](../auth/ldap/ldap_synchronization.md#adjust-ldap-group-sync-schedule) instead. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:group_sync -``` + ```shell + sudo gitlab-rake gitlab:ldap:group_sync + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:ldap:group_sync -``` + ```shell + bundle exec rake gitlab:ldap:group_sync + ``` ## Rename a provider @@ -86,17 +86,17 @@ If you input an incorrect new provider, users cannot sign in. If this happens, run the task again with the incorrect provider as the `old_provider` and the correct provider as the `new_provider`. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:rename_provider[old_provider,new_provider] -``` + ```shell + sudo gitlab-rake gitlab:ldap:rename_provider[old_provider,new_provider] + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:ldap:rename_provider[old_provider,new_provider] RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:ldap:rename_provider[old_provider,new_provider] RAILS_ENV=production + ``` ### Example @@ -123,17 +123,17 @@ User identities were successfully updated If you do not specify an `old_provider` and `new_provider` the task prompts you for them: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:rename_provider -``` + ```shell + sudo gitlab-rake gitlab:ldap:rename_provider + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:ldap:rename_provider RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:ldap:rename_provider RAILS_ENV=production + ``` **Example output:** @@ -158,17 +158,17 @@ The following Rake tasks are provided for updating the contents of the encrypted Show the contents of the current LDAP secrets. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:secret:show -``` + ```shell + sudo gitlab-rake gitlab:ldap:secret:show + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:ldap:secret:show RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:ldap:secret:show RAILS_ENV=production + ``` **Example output:** @@ -182,33 +182,33 @@ main: Opens the secret contents in your editor, and writes the resulting content to the encrypted secret file when you exit. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:ldap:secret:edit EDITOR=vim -``` + ```shell + sudo gitlab-rake gitlab:ldap:secret:edit EDITOR=vim + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:ldap:secret:edit RAILS_ENV=production EDITOR=vim -``` + ```shell + bundle exec rake gitlab:ldap:secret:edit RAILS_ENV=production EDITOR=vim + ``` ### Write raw secret Write new secret content by providing it on STDIN. -**Omnibus Installation** +- Linux package installations: -```shell -echo -e "main:\n password: '123'" | sudo gitlab-rake gitlab:ldap:secret:write -``` + ```shell + echo -e "main:\n password: '123'" | sudo gitlab-rake gitlab:ldap:secret:write + ``` -**Source Installation** +- Self-compiled installations: -```shell -echo -e "main:\n password: '123'" | bundle exec rake gitlab:ldap:secret:write RAILS_ENV=production -``` + ```shell + echo -e "main:\n password: '123'" | bundle exec rake gitlab:ldap:secret:write RAILS_ENV=production + ``` ### Secrets examples diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index 50c4b004f9c..c2c6429ba8b 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -13,17 +13,17 @@ GitLab provides Rake tasks for general maintenance. This command gathers information about your GitLab installation and the system it runs on. These may be useful when asking for help or reporting issues. In a multi-node environment, run this command on nodes running GitLab Rails to avoid PostgreSQL socket errors. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:env:info -``` + ```shell + sudo gitlab-rake gitlab:env:info + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:env:info RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:env:info RAILS_ENV=production + ``` Example output: @@ -69,24 +69,24 @@ GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20501) in GitLab 12.6. > - Moved to GitLab Premium in 13.9. -This command shows information about your [GitLab license](../../user/admin_area/license.md) and +This command shows information about your [GitLab license](../../administration/license.md) and how many seats are used. It is only available on GitLab Enterprise installations: a license cannot be installed into GitLab Community Edition. These may be useful when raising tickets with Support, or for programmatically checking your license parameters. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:license:info -``` + ```shell + sudo gitlab-rake gitlab:license:info + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:license:info RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:license:info RAILS_ENV=production + ``` Example output: @@ -117,24 +117,24 @@ If you're running Geo, see also the [Geo Health check Rake task](../geo/replicat You may also have a look at our troubleshooting guides for: -- [GitLab](../troubleshooting/index.md) -- [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html#troubleshooting) +- [GitLab](../troubleshooting/index.md). +- [Linux package installations](https://docs.gitlab.com/omnibus/index.html#troubleshooting). Additionally you should also [verify database values can be decrypted using the current secrets](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). To run `gitlab:check`, run: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:check -``` + ```shell + sudo gitlab-rake gitlab:check + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:check RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:check RAILS_ENV=production + ``` Use `SANITIZE=true` for `gitlab:check` if you want to omit project names from the output. @@ -190,18 +190,18 @@ for example, if after an upgrade you receive `Permission denied (publickey)` whe and find `404 Key Not Found` errors in [the `gitlab-shell.log` file](../logs/index.md#gitlab-shelllog). To rebuild `authorized_keys`, run: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:shell:setup -``` + ```shell + sudo gitlab-rake gitlab:shell:setup + ``` -**Source Installation** +- Self-compiled installations: -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production -``` + ```shell + cd /home/git/gitlab + sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production + ``` Example output: @@ -216,18 +216,18 @@ Do you want to continue (yes/no)? yes If for some reason the dashboard displays the wrong information, you might want to clear Redis' cache. To do this, run: -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake cache:clear -``` + ```shell + sudo gitlab-rake cache:clear + ``` -**Source Installation** +- Self-compiled installations: -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production -``` + ```shell + cd /home/git/gitlab + sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production + ``` ## Precompile the assets @@ -235,24 +235,24 @@ Sometimes during version upgrades you might end up with some wrong CSS or missing some icons. In that case, try to precompile the assets again. This Rake task only applies to source installations. [Read more](../../update/package/index.md#missing-asset-files) -about troubleshooting this problem when running the Omnibus GitLab package. -The guidance for Omnibus GitLab might be applicable for Kubernetes and Docker Omnibus +about troubleshooting this problem when running the Linux package. +The guidance for Linux package might be applicable for Kubernetes and Docker deployments of GitLab, though in general, container-based installations don't have issues with missing assets. -**Source Installation** +- Self-compiled installations: -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production -``` + ```shell + cd /home/git/gitlab + sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production + ``` -For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at -the release of upstream GitLab. The omnibus version includes optimized versions +For Linux package installations, the unoptimized assets (JavaScript, CSS) are frozen at +the release of upstream GitLab. The Linux package installation includes optimized versions of those assets. Unless you are modifying the JavaScript / CSS code on your production machine after installing the package, there should be no reason to redo `rake gitlab:assets:compile` on the production machine. If you suspect that assets -have been corrupted, you should reinstall the omnibus package. +have been corrupted, you should reinstall the Linux package. ## Check TCP connectivity to a remote site @@ -261,18 +261,18 @@ service on another machine (for example a PostgreSQL or web server) to troubleshoot proxy issues. A Rake task is included to help you with this. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:tcp_check[example.com,80] -``` + ```shell + sudo gitlab-rake gitlab:tcp_check[example.com,80] + ``` -**Source Installation** +- Self-compiled installations: -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:tcp_check[example.com,80] RAILS_ENV=production -``` + ```shell + cd /home/git/gitlab + sudo -u git -H bundle exec rake gitlab:tcp_check[example.com,80] RAILS_ENV=production + ``` ## Clear exclusive lease (DANGER) @@ -410,3 +410,16 @@ To re-import the metrics you can run: ```shell sudo gitlab-rake metrics:setup_common_metrics ``` + +## Troubleshooting + +### Advisory lock connection information + +After running the `db:migrate` Rake task, you may see output like the following: + +```shell +main: == [advisory_lock_connection] object_id: 173580, pg_backend_pid: 5532 +main: == [advisory_lock_connection] object_id: 173580, pg_backend_pid: 5532 +``` + +The messages returned are informational and can be ignored. diff --git a/doc/administration/raketasks/praefect.md b/doc/administration/raketasks/praefect.md index 1f9eb06f567..ba2a135cb01 100644 --- a/doc/administration/raketasks/praefect.md +++ b/doc/administration/raketasks/praefect.md @@ -20,14 +20,14 @@ Rake tasks are available for projects that have been created on Praefect storage Run this Rake task on the node that GitLab is installed and not on the node that Praefect is installed. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake "gitlab:praefect:replicas[project_id]" -``` + ```shell + sudo gitlab-rake "gitlab:praefect:replicas[project_id]" + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo -u git -H bundle exec rake "gitlab:praefect:replicas[project_id]" RAILS_ENV=production -``` + ```shell + sudo -u git -H bundle exec rake "gitlab:praefect:replicas[project_id]" RAILS_ENV=production + ``` diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index 17a0eb46a30..36013803999 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -33,7 +33,7 @@ Parameters: bundle exec rake "gitlab:import_export:import[root, group/subgroup, testingprojectimport, /path/to/file.tar.gz]" ``` -If you're running Omnibus, run the following Rake task: +If you're running a Linux package installation, run the following Rake task: ```shell gitlab-rake "gitlab:import_export:import[root, group/subgroup, testingprojectimport, /path/to/file.tar.gz]" @@ -99,10 +99,10 @@ To fix the issue: 1. Change the file permissions to `0400`. 1. Move the file to a public folder (for example `/tmp/`). -### `Name can contain only letters, digits, emojis ...` +### `Name can contain only letters, digits, emoji ...` ```plaintext -Name can contain only letters, digits, emojis, '_', '.', '+', dashes, or spaces. It must start with a letter, +Name can contain only letters, digits, emoji, '_', '.', '+', dashes, or spaces. It must start with a letter, digit, emoji, or '_', and Path can contain only letters, digits, '_', '-', or '.'. It cannot start with '-', end in '.git', or end in '.atom'. ``` diff --git a/doc/administration/raketasks/service_desk_email.md b/doc/administration/raketasks/service_desk_email.md index 1cbdec35171..9bf0846fef2 100644 --- a/doc/administration/raketasks/service_desk_email.md +++ b/doc/administration/raketasks/service_desk_email.md @@ -12,7 +12,7 @@ The following are Service Desk email-related Rake tasks. ## Secrets -GitLab can use [Service Desk email](../../user/project/service_desk.md#configure-a-custom-mailbox) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. +GitLab can use [Service Desk email](../../user/project/service_desk.md#configure-service-desk-alias-email) secrets read from an encrypted file instead of storing them in plaintext in the file system. The following Rake tasks are provided for updating the contents of the encrypted file. ### Show secret diff --git a/doc/administration/raketasks/smtp.md b/doc/administration/raketasks/smtp.md index 5e9e3544902..3cb161345cb 100644 --- a/doc/administration/raketasks/smtp.md +++ b/doc/administration/raketasks/smtp.md @@ -18,17 +18,17 @@ GitLab can use SMTP configuration secrets to read from an encrypted file. The fo Show the contents of the current SMTP secrets. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:smtp:secret:show -``` + ```shell + sudo gitlab-rake gitlab:smtp:secret:show + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:smtp:secret:show RAILS_ENV=production -``` + ```shell + bundle exec rake gitlab:smtp:secret:show RAILS_ENV=production + ``` **Example output:** @@ -41,33 +41,33 @@ user_name: 'gitlab-inst' Opens the secret contents in your editor, and writes the resulting content to the encrypted secret file when you exit. -**Omnibus Installation** +- Linux package installations: -```shell -sudo gitlab-rake gitlab:smtp:secret:edit EDITOR=vim -``` + ```shell + sudo gitlab-rake gitlab:smtp:secret:edit EDITOR=vim + ``` -**Source Installation** +- Self-compiled installations: -```shell -bundle exec rake gitlab:smtp:secret:edit RAILS_ENV=production EDITOR=vim -``` + ```shell + bundle exec rake gitlab:smtp:secret:edit RAILS_ENV=production EDITOR=vim + ``` ### Write raw secret Write new secret content by providing it on `STDIN`. -**Omnibus Installation** +- Linux package installations: -```shell -echo -e "password: '123'" | sudo gitlab-rake gitlab:smtp:secret:write -``` + ```shell + echo -e "password: '123'" | sudo gitlab-rake gitlab:smtp:secret:write + ``` -**Source Installation** +- Self-compiled installations: -```shell -echo -e "password: '123'" | bundle exec rake gitlab:smtp:secret:write RAILS_ENV=production -``` + ```shell + echo -e "password: '123'" | bundle exec rake gitlab:smtp:secret:write RAILS_ENV=production + ``` ### Secrets examples diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index 3664a79bf43..9e0a89fa7cb 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -20,7 +20,7 @@ available on legacy and hashed storage. To have a summary and then a list of projects and their attachments using legacy storage: -- **Omnibus installation** +- Linux package installations: ```shell # Projects @@ -32,7 +32,7 @@ To have a summary and then a list of projects and their attachments using legacy sudo gitlab-rake gitlab:storage:list_legacy_attachments ``` -- **Source installation** +- Self-compiled installations: ```shell # Projects @@ -48,7 +48,7 @@ To have a summary and then a list of projects and their attachments using legacy To have a summary and then a list of projects and their attachments using hashed storage: -- **Omnibus installation** +- Linux package installations: ```shell # Projects @@ -60,7 +60,7 @@ To have a summary and then a list of projects and their attachments using hashed sudo gitlab-rake gitlab:storage:list_hashed_attachments ``` -- **Source installation** +- Self-compiled installations: ```shell # Projects @@ -86,13 +86,13 @@ This task must be run on any machine that has Rails/Sidekiq configured, and the schedules all your existing projects and attachments associated with it to be migrated to the **Hashed** storage type: -- **Omnibus installation** +- Linux package installations: ```shell sudo gitlab-rake gitlab:storage:migrate_to_hashed ``` -- **Source installation** +- Self-compiled installations: ```shell sudo -u git -H bundle exec rake gitlab:storage:migrate_to_hashed RAILS_ENV=production @@ -101,7 +101,7 @@ migrated to the **Hashed** storage type: If you have any existing integration, you may want to do a small rollout first, to validate. You can do so by specifying an ID range with the operation by using the environment variables `ID_FROM` and `ID_TO`. For example, to limit the rollout -to project IDs 50 to 100 in an Omnibus GitLab installation: +to project IDs 50 to 100 in a Linux package installation: ```shell sudo gitlab-rake gitlab:storage:migrate_to_hashed ID_FROM=50 ID_TO=100 @@ -139,13 +139,13 @@ been disabled. This task schedules all your existing projects and associated attachments to be rolled back to the legacy storage type. -- **Omnibus installation** +- Linux package installations: ```shell sudo gitlab-rake gitlab:storage:rollback_to_legacy ``` -- **Source installation** +- Self-compiled installations: ```shell sudo -u git -H bundle exec rake gitlab:storage:rollback_to_legacy RAILS_ENV=production @@ -154,7 +154,7 @@ legacy storage type. If you have any existing integration, you may want to do a small rollback first, to validate. You can do so by specifying an ID range with the operation by using the environment variables `ID_FROM` and `ID_TO`. For example, to limit the rollout -to project IDs 50 to 100 in an Omnibus GitLab installation: +to project IDs 50 to 100 in a Linux package installation: ```shell sudo gitlab-rake gitlab:storage:rollback_to_legacy ID_FROM=50 ID_TO=100 @@ -244,9 +244,9 @@ If destroying the project generates a stack trace relating to encryption or the 1. [Verify your GitLab secrets](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). 1. If the affected projects have secrets that cannot be decrypted it will be necessary to remove those specific secrets. - [Our documentation for dealing with lost secrets](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost) + [Our documentation for dealing with lost secrets](../../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost) is for loss of all secrets, but it's possible for specific projects to be affected. For example, - to [reset specific runner registration tokens](../../raketasks/backup_restore.md#reset-runner-registration-tokens) + to [reset specific runner registration tokens](../../administration/backup_restore/backup_gitlab.md#reset-runner-registration-tokens) for a specific project ID: ```sql diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 1f6e7fda082..a1826d3e5df 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -30,24 +30,24 @@ These [individual Rake tasks](#individual-rake-tasks) are described in the next To migrate all uploads from local storage to object storage, run: -**Omnibus Installation** +- Linux package installations: -```shell -gitlab-rake "gitlab:uploads:migrate:all" -``` + ```shell + gitlab-rake "gitlab:uploads:migrate:all" + ``` -**Source Installation** +- Self-compiled installations: -```shell -sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all -``` + ```shell + sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all + ``` You can optionally track progress and verify that all uploads migrated successfully using the [PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database): -- `sudo gitlab-rails dbconsole` for Omnibus GitLab 14.1 and earlier. -- `sudo gitlab-rails dbconsole --database main` for Omnibus GitLab 14.2 and later. -- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. +- `sudo gitlab-rails dbconsole` for Linux package installations running GitLab 14.1 and earlier. +- `sudo gitlab-rails dbconsole --database main` for Linux package installations running GitLab 14.2 and later. +- `sudo -u git -H psql -d gitlabhq_production` for self-compiled installations. Verify `objectstg` below (where `store=2`) has count of all artifacts: @@ -92,7 +92,9 @@ the default batch size: The following shows how to run `gitlab:uploads:migrate` for individual types of uploads. -**Omnibus Installation** +::Tabs + +:::TabTitle Linux package (Omnibus) ```shell # gitlab-rake gitlab:uploads:migrate[uploader_class, model_class, mount_point] @@ -120,7 +122,7 @@ gitlab-rake "gitlab:uploads:migrate[FileUploader, MergeRequest]" gitlab-rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action, :image_v432x230]" ``` -**Source Installation** +:::TabTitle Self-compiled (source) Use `RAILS_ENV=production` for every task. @@ -150,6 +152,8 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, MergeReque sudo -u git -H bundle exec rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action]" ``` +::EndTabs + ## Migrate to local storage If you need to disable [object storage](../../object_storage.md) for any reason, you must first @@ -172,17 +176,17 @@ Keep in mind the task name in this case is `gitlab:uploads:migrate_to_local`. To migrate uploads from object storage to local storage, run the following Rake task: -**Omnibus GitLab installation** +- Linux package installations: -```shell -gitlab-rake "gitlab:uploads:migrate_to_local:all" -``` + ```shell + gitlab-rake "gitlab:uploads:migrate_to_local:all" + ``` -**Source installation** +- Self-compiled installations: -```shell -sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all -``` + ```shell + sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all + ``` After running the Rake task, you can disable object storage by undoing the changes described in the instructions to [configure object storage](../../uploads.md#using-object-storage). diff --git a/doc/administration/raketasks/uploads/sanitize.md b/doc/administration/raketasks/uploads/sanitize.md index 567a20a37f3..f86b4406a9e 100644 --- a/doc/administration/raketasks/uploads/sanitize.md +++ b/doc/administration/raketasks/uploads/sanitize.md @@ -15,8 +15,8 @@ can remove EXIF data from existing images that were uploaded to an earlier versi To run this Rake task, you need `exiftool` installed on your system. If you installed GitLab: -- Using the Omnibus package, you're all set. -- From source, make sure `exiftool` is installed: +- By using the Linux package, you're all set. +- By using the self-compiled installation, make sure `exiftool` is installed: ```shell # Debian/Ubuntu diff --git a/doc/administration/read_only_gitlab.md b/doc/administration/read_only_gitlab.md index 3842cf0846b..adc6f42271c 100644 --- a/doc/administration/read_only_gitlab.md +++ b/doc/administration/read_only_gitlab.md @@ -66,7 +66,7 @@ sudo gitlab-ctl start puma If you want to allow users to use the GitLab UI, ensure that the database is read-only: -1. Take a [GitLab backup](../raketasks/backup_restore.md) +1. Take a [GitLab backup](../administration/backup_restore/index.md) in case things don't go as expected. 1. Enter PostgreSQL on the console as an administrator user: diff --git a/doc/administration/redis/replication_and_failover.md b/doc/administration/redis/replication_and_failover.md index 4a6f58a8d6a..1db5b82e7dc 100644 --- a/doc/administration/redis/replication_and_failover.md +++ b/doc/administration/redis/replication_and_failover.md @@ -659,7 +659,7 @@ persistence classes. | `shared_state` | Store session-related and other persistent data. | | `actioncable` | Pub/Sub queue backend for ActionCable. | | `trace_chunks` | Store [CI trace chunks](../job_logs.md#enable-or-disable-incremental-logging) data. | -| `rate_limiting` | Store [rate limiting](../../user/admin_area/settings/user_and_ip_rate_limits.md) state. | +| `rate_limiting` | Store [rate limiting](../settings/user_and_ip_rate_limits.md) state. | | `sessions` | Store [sessions](../../../ee/development/session.md#gitlabsession). | | `repository_cache` | Store cache data specific to repositories. | diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index c3cf7c599a3..e9a77d15a6c 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -232,7 +232,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -2265,7 +2265,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 37571ed5771..0e608a953b8 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -243,7 +243,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -2283,7 +2283,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 455b0fbafd1..e3e361db133 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -134,7 +134,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -963,7 +963,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 6a7d9864376..4b7563d9d8d 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -243,7 +243,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -2272,7 +2272,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index 9a2c354f27c..6dc7f7affab 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -241,7 +241,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -2282,7 +2282,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index b0bc70aaf00..754a844df3f 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -240,7 +240,7 @@ spread connections equally in practice. ### Readiness checks Ensure the external load balancer only routes to working services with built -in monitoring endpoints. The [readiness checks](../../user/admin_area/monitoring/health_check.md) +in monitoring endpoints. The [readiness checks](../monitoring/health_check.md) all require [additional configuration](../monitoring/ip_allowlist.md) on the nodes being checked, otherwise, the external load balancer will not be able to connect. @@ -2240,7 +2240,7 @@ Refer to [epic 6127](https://gitlab.com/groups/gitlab-org/-/epics/6127) for more ### Cluster topology The following tables and diagram detail the hybrid environment using the same formats -as the normal environment above. +as the typical environment above. First are the components that run in Kubernetes. These run across several node groups, although you can change the overall makeup as desired as long as the minimum CPU and Memory requirements are observed. diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 08aeb149454..8fc9fbce119 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -63,7 +63,7 @@ This section explains the designs you can choose from. It begins with the least ### Standalone (non-HA) -For environments serving 2,000 or fewer users, we generally recommend a standalone approach by deploying a non-highly available single or multi-node environment. With this approach, you can employ strategies such as [automated backups](../../raketasks/backup_gitlab.md#configuring-cron-to-make-daily-backups) for recovery to provide a good level of RPO / RTO while avoiding the complexities that come with HA. +For environments serving 2,000 or fewer users, we generally recommend a standalone approach by deploying a non-highly available single or multi-node environment. With this approach, you can employ strategies such as [automated backups](../../administration/backup_restore/backup_gitlab.md#configuring-cron-to-make-daily-backups) for recovery to provide a good level of RPO / RTO while avoiding the complexities that come with HA. *[RTO]: Recovery time objective *[RPO]: Recovery point objective @@ -312,7 +312,7 @@ Additionally, the following cloud provider services are validated and supported </tr> <tr> <td>Redis</td> - <td></td> + <td>🟢 <a href="https://cloud.google.com/memorystore" target="_blank" rel="noopener noreferrer">Memorystore</a></td> <td>🟢 <a href="https://aws.amazon.com/elasticache/" target="_blank" rel="noopener noreferrer">ElastiCache</a></td> <td></td> </tr> @@ -633,8 +633,8 @@ Most setups would only need vertical scaling, but there are some specific areas - Gitaly to Gitaly Cluster w/ Praefect - From 10k users and higher, Redis is recommended to be split into multiple HA servers as it's single threaded. -Conversely, if you have robust metrics in place that show the environment is over-provisioned you can apply the same process for -scaling downloads. It's recommended to take an iterative approach when scaling downwards however to ensure there are no issues. +Conversely, if you have robust metrics in place that show the environment is over-provisioned, you can apply the same process for +scaling downwards. You should take an iterative approach when scaling downwards, however, to ensure there are no issues. ### How to monitor your environment diff --git a/doc/administration/reporting/git_abuse_rate_limit.md b/doc/administration/reporting/git_abuse_rate_limit.md new file mode 100644 index 00000000000..270a7cb4800 --- /dev/null +++ b/doc/administration/reporting/git_abuse_rate_limit.md @@ -0,0 +1,49 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Git abuse rate limit (administration) **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/394996) in GitLab 15.11. Feature flag `git_abuse_rate_limit_feature_flag` removed. + +This is the administration documentation. For information about Git abuse rate limiting at the group level, see the [group-level documentation](../../user/group/reporting/git_abuse_rate_limit.md). + +Git abuse rate limiting is a feature to automatically [ban users](../../administration/moderate_users.md#ban-and-unban-users) who download, clone, or fork more than a specified number of repositories in any project in the instance in a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../user/profile/personal_access_tokens.md) or [group access token](../../user/group/settings/group_access_tokens.md). + +Git abuse rate limiting does not apply to instance administrators, [deploy tokens](../../user/project/deploy_tokens/index.md), or [deploy keys](../../user/project/deploy_keys/index.md). + +How GitLab determines a user's rate limit is under development. +GitLab team members can view more information in this confidential epic: +`https://gitlab.com/groups/gitlab-org/modelops/anti-abuse/-/epics/14`. + +## Configure Git abuse rate limiting + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Reporting**. +1. Expand **Git abuse rate limit**. +1. Update the Git abuse rate limit settings: + 1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled. + 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled. + 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned. + 1. Add up to `100` users to the **Send notifications to** field. You must select at least one user. All application administrators are selected by default. + 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. +1. Select **Save changes**. + +## Automatic ban notifications + +If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent to the users listed under **Send notifications to**. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. + +If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the GitLab instance. + +## Unban a user + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Banned** tab and search for the account you want to unban. +1. From the **User administration** dropdown list select **Unban user**. +1. On the confirmation dialog, select **Unban user**. diff --git a/doc/administration/reporting/ip_addr_restrictions.md b/doc/administration/reporting/ip_addr_restrictions.md new file mode 100644 index 00000000000..5b749c62c30 --- /dev/null +++ b/doc/administration/reporting/ip_addr_restrictions.md @@ -0,0 +1,33 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# IP address restrictions **(FREE SELF)** + +IP address restrictions help prevent malicious users hiding their activities behind multiple IP addresses. + +GitLab maintains a list of the unique IP addresses used by a user to make requests over a specified period. When the +specified limit is reached, any requests made by the user from a new IP address are rejected with a `403 Forbidden` error. + +IP addresses are cleared from the list when no further requests have been made by the user from the IP address in the specified time period. + +NOTE: +When a runner runs a CI/CD job as a particular user, the runner IP address is also stored against the user's list of +unique IP addresses. Therefore, the IP addresses per user limit should take into account the number of configured active runners. + +## Configure IP address restrictions + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Reporting**. +1. Expand **Spam and Anti-bot Protection**. +1. Update the IP address restrictions settings: + 1. Select the **Limit sign in from multiple IP addresses** checkbox to enable IP address restrictions. + 1. Enter a number in the **IP addresses per user** field, greater than or equal to `1`. This number specifies the + maximum number of unique IP addresses a user can access GitLab from in the specified time period before requests + from a new IP address are rejected. + 1. Enter a number in the **IP address expiration time** field, greater than or equal to `0`. This number specifies the + time in seconds an IP address counts towards the limit for a user, taken from the time the last request was made. +1. Select **Save changes**. diff --git a/doc/administration/reporting/spamcheck.md b/doc/administration/reporting/spamcheck.md new file mode 100644 index 00000000000..8e478729299 --- /dev/null +++ b/doc/administration/reporting/spamcheck.md @@ -0,0 +1,69 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Spamcheck anti-spam service **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259) in GitLab 14.8. + +WARNING: +Spamcheck is available to all tiers, but only on instances using GitLab Enterprise Edition (EE). For [licensing reasons](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259#note_726605397), it is not included in the GitLab Community Edition (CE) package. You can [migrate from CE to EE](../../update/package/convert_to_ee.md). + +[Spamcheck](https://gitlab.com/gitlab-org/spamcheck) is an anti-spam engine +developed by GitLab originally to combat rising amount of spam in GitLab.com, +and later made public to be used in self-managed GitLab instances. + +## Enable Spamcheck + +Spamcheck is only available for package-based installations: + +1. Edit `/etc/gitlab/gitlab.rb` and enable Spamcheck: + + ```ruby + spamcheck['enable'] = true + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Verify that the new services `spamcheck` and `spam-classifier` are + up and running: + + ```shell + sudo gitlab-ctl status + ``` + +## Configure GitLab to use Spamcheck + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Reporting**. +1. Expand **Spam and Anti-bot Protection**. +1. Update the Spam Check settings: + 1. Check the "Enable Spam Check via external API endpoint" checkbox. + 1. For **URL of the external Spam Check endpoint** use `grpc://localhost:8001`. + 1. Leave **Spam Check API key** blank. +1. Select **Save changes**. + +NOTE: +In single-node instances, Spamcheck runs over `localhost`, and hence is running +in an unauthenticated mode. If on multi-node instances where GitLab runs on one +server and Spamcheck runs on another server listening over a public endpoint, it +is recommended to enforce some sort of authentication using a reverse proxy in +front of the Spamcheck service that can be used along with an API key. One +example would be to use `JWT` authentication for this and specifying a bearer +token as the API key. +[Native authentication for Spamcheck is in the works](https://gitlab.com/gitlab-com/gl-security/engineering-and-research/automation-team/spam/spamcheck/-/issues/171). + +## Running Spamcheck over TLS + +Spamcheck service on its own cannot communicate directly over TLS with GitLab. +However, Spamcheck can be deployed behind a reverse proxy which performs TLS +termination. In such a scenario, GitLab can be made to communicate with +Spamcheck over TLS by specifying `tls://` scheme for the external Spamcheck URL +instead of `grpc://` in the Admin Area settings. diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 1a83a05c3dd..9967b623773 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -45,8 +45,8 @@ For more information on: WARNING: The following information is for configuring GitLab to directly access repositories. This -configuration option is deprecated in favor of using [Gitaly](gitaly/index.md). Gitaly issue -[1690](https://gitlab.com/gitlab-org/gitaly/-/issues/1690) proposes to remove this configuration option. +configuration option is deprecated in favor of using [Gitaly](gitaly/index.md). +[Issue 403318](https://gitlab.com/gitlab-org/gitlab/-/issues/403318) proposes to remove this configuration option. To configure repository storage paths: @@ -73,7 +73,7 @@ For repository storage paths: ### Configure for backups -For [backups](../raketasks/backup_restore.md) to work correctly: +For [backups](../administration/backup_restore/index.md) to work correctly: - The repository storage path cannot be a mount point. - The GitLab user must have correct permissions for the parent directory of the path. diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index 3bd73b4df94..e1512038286 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -73,7 +73,7 @@ translate between the human-readable project name and the hashed storage path. Y Administrators can look up a project's hashed path from its name or ID using: -- The [Admin Area](../user/admin_area/index.md#administering-projects). +- The [Admin Area](../administration/admin_area.md#administering-projects). - A Rails console. To look up a project's hash path in the Admin Area: diff --git a/doc/administration/review_abuse_reports.md b/doc/administration/review_abuse_reports.md new file mode 100644 index 00000000000..e3891cbe68a --- /dev/null +++ b/doc/administration/review_abuse_reports.md @@ -0,0 +1,96 @@ +--- +stage: Anti-Abuse +group: Anti-Abuse +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference, howto +--- + +# Review abuse reports **(FREE SELF)** + +View and resolve abuse reports from GitLab users. + +GitLab administrators can view and [resolve](#resolving-abuse-reports) abuse +reports in the Admin Area. + +## Receive notification of abuse reports by email + +To receive notifications of new abuse reports by email: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Reporting**. +1. Expand the **Abuse reports** section. +1. Provide an email address and select **Save changes**. + +The notification email address can also be set and retrieved +[using the API](../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). + +## Reporting abuse + +To find out more about reporting abuse, see +[abuse reports user documentation](../user/report_abuse.md). + +## Resolving abuse reports + +To access abuse reports: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Abuse Reports**. + +There are 3 ways to resolve an abuse report, with a button for each method: + +- Remove user & report. This: + - [Deletes the reported user](../user/profile/account/delete_account.md) from the + instance. + - Removes the abuse report from the list. +- [Block user](#blocking-users). +- Remove report. This: + - Removes the abuse report from the list. + - Removes access restrictions for the reported user. + +The following is an example of the **Abuse Reports** page: + + + +### Blocking users + +A blocked user cannot sign in or access any repositories, but all of their data +remains. + +Blocking a user: + +- Leaves them in the abuse report list. +- Changes the **Block user** button to a disabled **Already blocked** button. + +The user is notified with the following message: + +```plaintext +Your account has been blocked. If you believe this is in error, contact a staff member. +``` + +After blocking, you can still either: + +- Remove the user and report if necessary. +- Remove the report. + +The following is an example of a blocked user listed on the **Abuse Reports** +page: + + + +NOTE: +Users can be [blocked](../api/users.md#block-user) and +[unblocked](../api/users.md#unblock-user) using the GitLab API. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/account_and_limit_settings.md b/doc/administration/settings/account_and_limit_settings.md new file mode 100644 index 00000000000..ca56a322237 --- /dev/null +++ b/doc/administration/settings/account_and_limit_settings.md @@ -0,0 +1,399 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +type: reference +--- + +# Account and limit settings **(FREE SELF)** + +## Default projects limit + +You can configure the default maximum number of projects new users can create in their +personal namespace. This limit affects only new user accounts created after you change +the setting. This setting is not retroactive for existing users, but you can separately edit +the [project limits for existing users](#projects-limit-for-a-user). + +To configure the maximum number of projects in personal namespaces for new users: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Increase or decrease that **Default projects limit** value. + +If you set **Default projects limit** to 0, users are not allowed to create projects +in their users personal namespace. However, projects can still be created in a group. + +### Projects limit for a user + +You can edit a specific user, and change the maximum number of projects this user +can create in their personal namespace: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview** > **Users**. +1. From the list of users, select a user. +1. Select **Edit**. +1. Increase or decrease the **Projects limit** value. + +## Max attachment size + +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/20061) from 10 MB to 100 MB in GitLab 15.7. + +The maximum file size for attachments in GitLab comments and replies is 100 MB. +To change the maximum attachment size: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Increase or decrease by changing the value in **Maximum attachment size (MB)**. + +If you choose a size larger than the configured value for the web server, +you may receive errors. Read the [troubleshooting section](#troubleshooting) for more +details. + +For GitLab.com repository size limits, read [accounts and limit settings](../../user/gitlab_com/index.md#account-and-limit-settings). + +## Max push size + +You can change the maximum push size for your instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Increase or decrease by changing the value in **Maximum push size (MB)**. + +For GitLab.com push size limits, read [accounts and limit settings](../../user/gitlab_com/index.md#account-and-limit-settings). + +NOTE: +When you [add files to a repository](../../user/project/repository/web_editor.md#create-a-file) +through the web UI, the maximum **attachment** size is the limiting factor, +because the [web server](../../development/architecture.md#components) +must receive the file before GitLab can generate the commit. +Use [Git LFS](../../topics/git/lfs/index.md) to add large files to a repository. + +## Max export size + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86124) in GitLab 15.0. + +To modify the maximum file size for exports in GitLab: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**, then expand **Account and limit**. +1. Increase or decrease by changing the value in **Maximum export size (MB)**. + +## Max import size + +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. + +To modify the maximum file size for imports in GitLab: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Increase or decrease by changing the value in **Maximum import size (MB)**. + +This setting applies only to repositories +[imported from a GitLab export file](../../user/project/settings/import_export.md#import-a-project-and-its-data). + +If you choose a size larger than the configured value for the web server, +you may receive errors. See the [troubleshooting section](#troubleshooting) for more +details. + +For GitLab.com repository size limits, read [accounts and limit settings](../../user/gitlab_com/index.md#account-and-limit-settings). + +## Personal access token prefix + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20968) in GitLab 13.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342327) in GitLab 14.5, a default prefix. + +You can specify a prefix for personal access tokens. You might use a prefix +to find tokens more quickly, or for use with automation tools. + +The default prefix is `glpat-` but administrators can change it. + +[Project access tokens](../../user/project/settings/project_access_tokens.md) and +[group access tokens](../../user/group/settings/group_access_tokens.md) also inherit this prefix. + +### Set a prefix + +To change the default global prefix: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. +1. Fill in the **Personal Access Token prefix** field. +1. Select **Save changes**. + +You can also configure the prefix by using the +[settings API](../../api/settings.md). + +## Repository size limit **(PREMIUM SELF)** + +Repositories in your GitLab instance can grow quickly, especially if you are +using LFS. Their size can grow exponentially, rapidly consuming available storage. +To prevent this from happening, you can set a hard limit for your repositories' size. +This limit can be set globally, per group, or per project, with per project limits +taking the highest priority. + +There are numerous use cases where you might set up a limit for repository size. +For instance, consider the following workflow: + +1. Your team develops apps which require large files to be stored in + the application repository. +1. Although you have enabled [Git LFS](../../topics/git/lfs/index.md#git-large-file-storage-lfs) + to your project, your storage has grown significantly. +1. Before you exceed available storage, you set up a limit of 10 GB + per repository. + +NOTE: +For GitLab.com repository size limits, read [accounts and limit settings](../../user/gitlab_com/index.md#account-and-limit-settings). + +### How it works + +Only a GitLab administrator can set those limits. Setting the limit to `0` means +there are no restrictions. + +These settings can be found in: + +- Each project's settings: + 1. From the Project's homepage, navigate to **Settings > General**. + 1. Fill in the **Repository size limit (MB)** field in the **Naming, topics, avatar** section. + 1. Select **Save changes**. +- Each group's settings: + 1. From the Group's homepage, navigate to **Settings > General**. + 1. Fill in the **Repository size limit (MB)** field in the **Naming, visibility** section. + 1. Select **Save changes**. +- GitLab global settings: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. Select **Settings > General**. + 1. Expand the **Account and limit** section. + 1. Fill in the **Size limit per repository (MB)** field. + 1. Select **Save changes**. + +The first push of a new project, including LFS objects, is checked for size. +If the sum of their sizes exceeds the maximum allowed repository size, the push +is rejected. + +NOTE: +The repository size limit includes repository files and LFS, but does not include artifacts, uploads, +wiki, packages, or snippets. The repository size limit applies to both private and public projects. + +For details on manually purging files, see [reducing the repository size using Git](../../user/project/repository/reducing_the_repo_size_using_git.md). + +## Session duration + +### Customize the default session duration + +You can change how long users can remain signed in without activity. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. The set duration is in **Session duration (minutes)**. + +If [Remember me](#turn-remember-me-on-or-off) is enabled, users' sessions can remain active for an indefinite period of time. + +For details, see [cookies used for sign-in](../../user/profile/index.md#cookies-used-for-sign-in). + +### Turn **Remember me** on or off + +> Ability to turn the **Remember me** setting on and off [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/369133) in GitLab 16.0. + +Users can select the **Remember me** checkbox on sign-in, and their session will remain active for an indefinite period of time when accessed from that specific browser. You can turn off this setting if you need sessions to expire for security or compliance purposes. Turning off this setting will ensure users' sessions expire after the number of minutes of inactivity set when you [customize your session duration](#customize-the-default-session-duration). + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Select or clear the **Remember me** checkbox to turn this setting on or off. + +### Customize session duration for Git Operations when 2FA is enabled **(PREMIUM SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296669) in GitLab 13.9. +> - It's deployed behind a feature flag, disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. This feature is not ready for production use. This feature flag also affects [2FA for Git over SSH operations](../../security/two_factor_authentication.md#2fa-for-git-over-ssh-operations). + +GitLab administrators can choose to customize the session duration (in minutes) for Git operations when 2FA is enabled. The default is 15 and this can be set to a value between 1 and 10080. + +To set a limit on how long these sessions are valid: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. +1. Fill in the **Session duration for Git operations when 2FA is enabled (minutes)** field. +1. Select **Save changes**. + +## Limit the lifetime of SSH keys **(ULTIMATE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1007) in GitLab 14.6 [with a flag](../../administration/feature_flags.md) named `ff_limit_ssh_key_lifetime`. Disabled by default. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/346753) in GitLab 14.6. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/1007) in GitLab 14.7. [Feature flag `ff_limit_ssh_key_lifetime`](https://gitlab.com/gitlab-org/gitlab/-/issues/347408) removed. + +Users can optionally specify a lifetime for +[SSH keys](../../user/ssh.md). +This lifetime is not a requirement, and can be set to any arbitrary number of days. + +SSH keys are user credentials to access GitLab. +However, organizations with security requirements may want to enforce more protection by +requiring the regular rotation of these keys. + +### Set a lifetime + +Only a GitLab administrator can set a lifetime. Leaving it empty means +there are no restrictions. + +To set a lifetime on how long SSH keys are valid: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. +1. Fill in the **Maximum allowable lifetime for SSH keys (days)** field. +1. Select **Save changes**. + +Once a lifetime for SSH keys is set, GitLab: + +- Requires users to set an expiration date that is no later than the allowed lifetime on new + SSH keys. +- Applies the lifetime restriction to existing SSH keys. Keys with no expiry or a lifetime + greater than the maximum immediately become invalid. + +NOTE: +When a user's SSH key becomes invalid they can delete and re-add the same key again. + +## Limit the lifetime of access tokens **(ULTIMATE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) in GitLab 12.6. + +Users can optionally specify a maximum lifetime in days for +access tokens, this includes [personal](../../user/profile/personal_access_tokens.md), +[group](../../user/group/settings/group_access_tokens.md), and [project](../../user/project/settings/project_access_tokens.md) access tokens. +This lifetime is not a requirement, and can be set to any value greater than 0 and less than or equal to 365. If this setting is left blank, the default allowable lifetime of access tokens is 365 days. + +Access tokens are the only tokens needed for programmatic access to GitLab. +However, organizations with security requirements may want to enforce more protection by +requiring the regular rotation of these tokens. + +### Set a lifetime + +Only a GitLab administrator can set a lifetime. Leaving it empty means +there are no restrictions. + +To set a lifetime on how long access tokens are valid: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Account and limit** section. +1. Fill in the **Maximum allowable lifetime for access tokens (days)** field. +1. Select **Save changes**. + +Once a lifetime for access tokens is set, GitLab: + +- Applies the lifetime for new personal access tokens, and require users to set an expiration date + and a date no later than the allowed lifetime. +- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the + allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, + or remove it, before revocation takes place. + +## Disable user profile name changes **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24605) in GitLab 12.7. + +To maintain integrity of user details in [Audit Events](../../administration/audit_events.md), GitLab administrators can choose to disable a user's ability to change their profile name. + +To do this: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Select the **Prevent users from changing their profile name** checkbox. + +NOTE: +When this ability is disabled, GitLab administrators can still use the +[Admin Area](../../administration/admin_area.md#administering-users) or the +[API](../../api/users.md#user-modification) to update usernames. + +## Prevent new users from creating top-level groups + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367754) in GitLab 15.5. + +By default, new users can create top-level groups. GitLab administrators can prevent new users from creating top-level groups: + +- In GitLab 15.5 and later, using either: + - The GitLab UI using the steps in this section. + - The [application setting API](../../api/settings.md#change-application-settings). +- In GitLab 15.4 and earlier, a [configuration file](../../administration/user_settings.md#use-configuration-files-to-prevent-new-users-from-creating-top-level-groups). + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Clear the **Allow new users to create top-level groups** checkbox. + +## Set profiles of new users to private by default + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231301) in GitLab 15.8. + +By default, newly created users have a public profile. GitLab administrators can set new users to have a private profile by default: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Select the **Make new users' profiles private by default** checkbox. + +## Prevent users from deleting their accounts **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26053) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `deleting_account_disabled_for_users`. Enabled by default. + +By default, users can delete their own accounts. GitLab administrators can prevent +users from deleting their own accounts: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Account and limit**. +1. Clear the **Allows users to delete their own accounts** checkbox. + +## Troubleshooting + +### 413 Request Entity Too Large + +When attaching a file to a comment or reply in GitLab displays a `413 Request Entity Too Large` +error, the [max attachment size](#max-attachment-size) +is probably larger than the web server's allowed value. + +To increase the max attachment size to 200 MB in a +[Linux package](https://docs.gitlab.com/omnibus/) install, you may need to +add the line below to `/etc/gitlab/gitlab.rb` before increasing the max attachment size: + +```ruby +nginx['client_max_body_size'] = "200m" +``` + +### This repository has exceeded its size limit + +If you receive intermittent push errors in your [Rails exceptions log](../../administration/logs/index.md#exceptions_jsonlog), like this: + +```plaintext +Your push has been rejected, because this repository has exceeded its size limit. +``` + +[Housekeeping](../../administration/housekeeping.md) tasks may be causing your repository size to grow. +To resolve this problem, either of these options helps in the short- to middle-term: + +- Increase the [repository size limit](#repository-size-limit). +- [Reduce the repository size](../../user/project/repository/reducing_the_repo_size_using_git.md). diff --git a/doc/administration/settings/continuous_integration.md b/doc/administration/settings/continuous_integration.md new file mode 100644 index 00000000000..eaa240d4c96 --- /dev/null +++ b/doc/administration/settings/continuous_integration.md @@ -0,0 +1,429 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Continuous Integration and Deployment Admin Area settings **(FREE SELF)** + +The [Admin Area](index.md) has the instance settings for Auto DevOps, runners, and +job artifacts. + +## Auto DevOps + +To enable (or disable) [Auto DevOps](../../topics/autodevops/index.md) +for all projects: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Check (or uncheck to disable) the box that says **Default to Auto DevOps pipeline for all projects**. +1. Optionally, set up the [Auto DevOps base domain](../../topics/autodevops/requirements.md#auto-devops-base-domain) + which is used for Auto Deploy and Auto Review Apps. +1. Select **Save changes** for the changes to take effect. + +From now on, every existing project and newly created ones that don't have a +`.gitlab-ci.yml` use the Auto DevOps pipelines. + +If you want to disable it for a specific project, you can do so in +[its settings](../../topics/autodevops/index.md#enable-or-disable-auto-devops). + +## Enable shared runners for new projects + +You can set all new projects to have the instance's shared runners available by default. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. Select the **Enable shared runners for new projects** checkbox. + +Any time a new project is created, the shared runners are available. + +## Shared runners compute quota + +As an administrator you can set either a global or namespace-specific +limit on the number of [compute minutes](../../ci/pipelines/cicd_minutes.md) you can use. + +## Enable a project runner for multiple projects + +If you have already registered a [project runner](../../ci/runners/runners_scope.md#project-runners) +you can assign that runner to other projects. + +To enable a project runner for more than one project: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. From the left sidebar, select **CI/CD > Runners**. +1. Select the runner you want to edit. +1. In the upper-right corner, select **Edit** (**{pencil}**). +1. Under **Restrict projects for this runner**, search for a project. +1. To the left of the project, select **Enable**. +1. Repeat this process for each additional project. + +## Add a message for shared runners + +To display details about the instance's shared runners in all projects' +runner settings: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. Enter text, including Markdown if you want, in the **Shared runner details** field. For example: + +  + +To view the rendered details: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. + + + +## Maximum artifacts size + +The maximum size of the [job artifacts](../../administration/job_artifacts.md) +can be set at: + +- The instance level. +- [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/21688), the project and group level. + +For the setting on GitLab.com, see [Artifacts maximum size](../../user/gitlab_com/index.md#gitlab-cicd). + +The value is in MB and the default is 100 MB per job. To change it at the: + +- Instance level: + + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Settings > CI/CD > Continuous Integration and Deployment**. + 1. Change the value of **Maximum artifacts size (MB)**. + 1. Select **Save changes** for the changes to take effect. + +- Group level (this overrides the instance setting): + + 1. Go to the group's **Settings > CI/CD > General Pipelines**. + 1. Change the value of **Maximum artifacts size** (in MB). + 1. Select **Save changes** for the changes to take effect. + +- Project level (this overrides the instance and group settings): + + 1. Go to the project's **Settings > CI/CD > General Pipelines**. + 1. Change the value of **maximum artifacts size** (in MB). + 1. Select **Save changes** for the changes to take effect. + +NOTE: +The setting at all levels is only available to GitLab administrators. + +## Default artifacts expiration + +The default expiration time of the [job artifacts](../../administration/job_artifacts.md) +can be set in the Admin Area of your GitLab instance. The syntax of duration is +described in [`artifacts:expire_in`](../../ci/yaml/index.md#artifactsexpire_in) +and the default value is `30 days`. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Change the value of default expiration time. +1. Select **Save changes** for the changes to take effect. + +This setting is set per job and can be overridden in +[`.gitlab-ci.yml`](../../ci/yaml/index.md#artifactsexpire_in). +To disable the expiration, set it to `0`. The default unit is in seconds. + +NOTE: +Any changes to this setting applies to new artifacts only. The expiration time is not +be updated for artifacts created before this setting was changed. +The administrator may need to manually search for and expire previously-created +artifacts, as described in the [troubleshooting documentation](../../administration/job_artifacts.md#delete-job-artifacts-from-jobs-completed-before-a-specific-date). + +## Keep the latest artifacts for all jobs in the latest successful pipelines + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50889) in GitLab 13.9. + +When enabled (default), the artifacts of the most recent pipeline for each Git ref +([branches and tags](https://git-scm.com/book/en/v2/Git-Internals-Git-References)) +are locked against deletion and kept regardless of the expiry time. + +When disabled, the latest artifacts for any **new** successful or fixed pipelines +are allowed to expire. + +This setting takes precedence over the [project level setting](../../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). +If disabled at the instance level, you cannot enable this per-project. + +To disable the setting: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand **Continuous Integration and Deployment**. +1. Clear the **Keep the latest artifacts for all jobs in the latest successful pipelines** checkbox. +1. Select **Save changes** + +When you disable the feature, the latest artifacts do not immediately expire. +A new pipeline must run before the latest artifacts can expire and be deleted. + +NOTE: +All application settings have a [customizable cache expiry interval](../../administration/application_settings_cache.md) which can delay the settings affect. + +## Archive jobs + +Archiving jobs is useful for reducing the CI/CD footprint on the system by removing some +of the capabilities of the jobs (metadata stored in the database needed to run the job), +but persisting the traces and artifacts for auditing purposes. + +To set the duration for which the jobs are considered as old and expired: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Continuous Integration and Deployment** section. +1. Set the value of **Archive jobs**. +1. Select **Save changes** for the changes to take effect. + +After that time passes, the jobs are archived in the background and no longer able to be +retried. Make it empty to never expire jobs. It has to be no less than 1 day, +for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. + +For the value set for GitLab.com, see [Scheduled job archiving](../../user/gitlab_com/index.md#gitlab-cicd). + +## Protect CI/CD variables by default + +To set all new [CI/CD variables](../../ci/variables/index.md) as +[protected](../../ci/variables/index.md#protect-a-cicd-variable) by default: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Select **Protect CI/CD variables by default**. + +## Maximum includes + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207270) in GitLab 16.0. + +The maximum number of [includes](../../ci/yaml/includes.md) per pipeline can be set at the instance level. +The default is `150`. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Change the value of **Maximum includes**. +1. Select **Save changes** for the changes to take effect. + +## Default CI/CD configuration file + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18073) in GitLab 12.5. + +The default CI/CD configuration file and path for new projects can be set in the Admin Area +of your GitLab instance (`.gitlab-ci.yml` if not set): + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Input the new file and path in the **Default CI/CD configuration file** field. +1. Select **Save changes** for the changes to take effect. + +It is also possible to specify a [custom CI/CD configuration file for a specific project](../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file). + +## Set CI/CD limits + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352175) in GitLab 14.10. + +You can configure some [CI/CD limits](../../administration/instance_limits.md#cicd-limits) +from the Admin Area: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Continuous Integration and Deployment** section. +1. In the **CI/CD limits** section, you can set the following limits: + - **Maximum number of jobs in a single pipeline** + - **Total number of jobs in currently active pipelines** + - **Maximum number of active pipelines per project** + - **Maximum number of pipeline subscriptions to and from a project** + - **Maximum number of pipeline schedules** + - **Maximum number of DAG dependencies that a job can have** + - **Maximum number of runners registered per group** + - **Maximum number of runners registered per project** + - **Maximum number of downstream pipelines in a pipeline's hierarchy tree** + +## Enable or disable the pipeline suggestion banner + +By default, a banner displays in merge requests with no pipeline suggesting a +walkthrough on how to add one. + + + +To enable or disable the banner: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Select or clear the **Enable pipeline suggestion banner** checkbox. +1. Select **Save changes**. + +## Required pipeline configuration **(ULTIMATE SELF)** + +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9. + +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9 +and is planned for removal in 17.0. Use [compliance pipelines](../../user/group/compliance_frameworks.md#compliance-pipelines) +instead. This change is a breaking change. + +You can set a [CI/CD template](../../ci/examples/index.md#cicd-templates) +as a required pipeline configuration for all projects on a GitLab instance. You can +use a template from: + +- The default CI/CD templates. +- A custom template stored in an [instance template repository](instance_template_repository.md). + + NOTE: + When you use a configuration defined in an instance template repository, + nested [`include:`](../../ci/yaml/index.md#include) keywords + (including `include:file`, `include:local`, `include:remote`, and `include:template`) + [do not work](https://gitlab.com/gitlab-org/gitlab/-/issues/35345). + +The project CI/CD configuration merges into the required pipeline configuration when +a pipeline runs. The merged configuration is the same as if the required pipeline configuration +added the project configuration with the [`include` keyword](../../ci/yaml/index.md#include). +To view a project's full merged configuration, [View full configuration](../../ci/pipeline_editor/index.md#view-full-configuration) +in the pipeline editor. + +To select a CI/CD template for the required pipeline configuration: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Required pipeline configuration** section. +1. Select a CI/CD template from the dropdown list. +1. Select **Save changes**. + +## Package Registry configuration + +### Maven Forwarding **(PREMIUM SELF)** + +GitLab administrators can disable the forwarding of Maven requests to [Maven Central](https://search.maven.org/). + +To disable forwarding Maven requests: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Clear the checkbox **Forward Maven package requests to the Maven Registry if the packages are not found in the GitLab Package Registry**. +1. Select **Save changes**. + +### npm Forwarding **(PREMIUM SELF)** + +GitLab administrators can disable the forwarding of npm requests to [npmjs.com](https://www.npmjs.com/). + +To disable it: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Clear the checkbox **Forward npm package requests to the npm Registry if the packages are not found in the GitLab Package Registry**. +1. Select **Save changes**. + +### PyPI Forwarding **(PREMIUM SELF)** + +GitLab administrators can disable the forwarding of PyPI requests to [pypi.org](https://pypi.org/). + +To disable it: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Clear the checkbox **Forward PyPI package requests to the PyPI Registry if the packages are not found in the GitLab Package Registry**. +1. Select **Save changes**. + +### Package file size limits + +GitLab administrators can adjust the maximum allowed file size for each package type. + +To set the maximum file size: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand the **Package Registry** section. +1. Find the package type you would like to adjust. +1. Enter the maximum file size, in bytes. +1. Select **Save size limits**. + +## Restrict runner registration by all users in an instance + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22225) in GitLab 14.1. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/368008) in GitLab 15.5. + +GitLab administrators can adjust who is allowed to register runners, by showing and hiding areas of the UI. + +When the registration sections are hidden in the UI, members of the project or group must contact administrators to enable runner registration in the group or project. If you plan to prevent registration, ensure users have access to the runners they need to run jobs. + +By default, all members of a project and group are able to register runners. + +To restrict all users in an instance from registering runners: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Runner registration** section, clear the **Members of the project can register runners** and + **Members of the group can register runners** checkboxes to remove runner registration from the UI. +1. Select **Save changes**. + +NOTE: +After you disable runner registration by members of a project, the registration +token automatically rotates. The token is no longer valid and you must +use the new registration token for the project. + +## Restrict runner registration by all members in a group + +Prerequisites: + +- Runner registration must be enabled for [all users in the instance](#restrict-runner-registration-by-all-users-in-an-instance). + +GitLab administrators can adjust group permissions to restrict runner registration by group members. + +To restrict runner registration by members in a specific group: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Groups** and find your group. +1. Select **Edit**. +1. Clear the **New group runners can be registered** checkbox if you want to disable runner registration by all members in the group. If the setting is read-only, you must enable runner registration for the [instance](#restrict-runner-registration-by-all-users-in-an-instance). +1. Select **Save changes**. + +## Disable runner version management + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114041) in GitLab 15.10. + +By default, GitLab instances periodically fetch official runner version data from GitLab.com to [determine whether the runners need upgrades](../../ci/runners/runners_scope.md#determine-which-runners-need-to-be-upgraded). + +To disable your instance fetching this data: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Runner version management** section, clear the **Fetch GitLab Runner release version data from GitLab.com** checkbox. +1. Select **Save changes**. + +## Troubleshooting + +### 413 Request Entity Too Large + +When build jobs fail with the following error, +increase the [maximum artifacts size](#maximum-artifacts-size). + +```plaintext +Uploading artifacts as "archive" to coordinator... too large archive <job-id> responseStatus=413 Request Entity Too Large status=413" at end of a build job on pipeline when trying to store artifacts to <object-storage>. +``` diff --git a/doc/administration/settings/deprecated_api_rate_limits.md b/doc/administration/settings/deprecated_api_rate_limits.md new file mode 100644 index 00000000000..f8db0810af5 --- /dev/null +++ b/doc/administration/settings/deprecated_api_rate_limits.md @@ -0,0 +1,54 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Deprecated API rate limits **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68645) in GitLab 14.4. + +Deprecated API endpoints are those which have been replaced with alternative +functionality, but cannot be removed without breaking backward compatibility. +Setting a restrictive rate limit on these endpoints can encourage users to +switch to the alternatives. + +## Deprecated API endpoints + +Not all deprecated API endpoints are included in this rate limit - just those +that might have a performance impact: + +- [`GET /groups/:id`](../../api/groups.md#details-of-a-group) **without** the `with_projects=0` query parameter. + +## Define Deprecated API rate limits + +Rate limits for deprecated API endpoints are disabled by default. When enabled, they supersede +the general user and IP rate limits for requests to deprecated endpoints. You can keep any general user +and IP rate limits already in place, and increase or decrease the rate limits +for deprecated API endpoints. No other new features are provided by this override. + +Prerequisite: + +- You must have administrator access to the instance. + +To override the general user and IP rate limits for requests to deprecated API endpoints: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Deprecated API Rate Limits**. +1. Select the checkboxes for the types of rate limits you want to enable: + - **Unauthenticated API request rate limit** + - **Authenticated API request rate limit** +1. If you selected **unauthenticated**: + 1. Select the **Maximum unauthenticated API requests per period per IP**. + 1. Select the **Unauthenticated API rate limit period in seconds**. +1. If you selected **authenticated**: + 1. Select the **Maximum authenticated API requests per period per user**. + 1. Select the **Authenticated API rate limit period in seconds**. + +## Related topics + +- [Rate limits](../../security/rate_limits.md) +- [User and IP rate limits](../settings/user_and_ip_rate_limits.md) diff --git a/doc/administration/settings/email.md b/doc/administration/settings/email.md new file mode 100644 index 00000000000..e4972897aab --- /dev/null +++ b/doc/administration/settings/email.md @@ -0,0 +1,125 @@ +--- +type: reference +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Email **(FREE SELF)** + +You can customize some of the content in emails sent from your GitLab instance. + +## Custom logo + +The logo in the header of some emails can be customized, see the [logo customization section](../../administration/appearance.md#navigation-bar). + +## Include author name in email notification email body **(PREMIUM SELF)** + +By default, GitLab overrides the email address in notification emails with the email address +of the issue, merge request, or comment author. Enable this setting to include the author's email +address in the body of the email instead. + +To include the author's email address in the email body: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Select the **Include author name in email notification email body** checkbox. +1. Select **Save changes**. + +## Enable multipart email **(PREMIUM SELF)** + +GitLab can send email in multipart format (HTML and plain text) or plain text only. + +To enable multipart email: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Select **Enable multipart email**. +1. Select **Save changes**. + +## Custom hostname for private commit emails **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22560) in GitLab 11.5. + +This configuration option sets the email hostname for [private commit emails](../../user/profile/index.md#use-an-automatically-generated-private-commit-email). + By default it is set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. + +To change the hostname used in private commit emails: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Enter the desired hostname in the **Custom hostname (for private commit emails)** field. +1. Select **Save changes**. + +NOTE: +After the hostname is configured, every private commit email using the previous hostname is not +recognized by GitLab. This can directly conflict with certain [Push rules](../../user/project/repository/push_rules.md) such as +`Check whether author is a GitLab user` and `Check whether committer is the current authenticated user`. + +## Custom additional text **(PREMIUM SELF)** + +You can add additional text at the bottom of any email that GitLab sends. This additional text +can be used for legal, auditing, or compliance reasons, for example. + +To add additional text to emails: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Enter your text in the **Additional text** field. +1. Select **Save changes**. + +## User deactivation emails + +GitLab sends email notifications to users when their account has been deactivated. + +To disable these notifications: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Clear the **Enable user deactivation emails** checkbox. +1. Select **Save changes**. + +### Custom additional text in deactivation emails **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. +> - [Enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111882) in GitLab 15.9. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an +administrator to [disable the feature flag](../../administration/feature_flags.md) named +`deactivation_email_additional_text`. + +You can add additional text at the bottom of the email that GitLab sends to users when their account +is deactivated. This email text is separate from the [custom additional text](#custom-additional-text) +setting. + +To add additional text to deactivation emails: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Email**. +1. Enter your text in the **Additional text for deactivation email** field. +1. Select **Save changes**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/external_authorization.md b/doc/administration/settings/external_authorization.md new file mode 100644 index 00000000000..45887fdccb8 --- /dev/null +++ b/doc/administration/settings/external_authorization.md @@ -0,0 +1,144 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# External authorization control **(FREE SELF)** + +> [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27056) from GitLab Premium to GitLab Free in 11.10. + +In highly controlled environments, it may be necessary for access policy to be +controlled by an external service that permits access based on project +classification and user access. GitLab provides a way to check project +authorization with your own defined service. + +After the external service is configured and enabled, when a project is +accessed, a request is made to the external service with the user information +and project classification label assigned to the project. When the service +replies with a known response, the result is cached for six hours. + +If the external authorization is enabled, GitLab further blocks pages and +functionality that render cross-project data. That includes: + +- Most pages under Dashboard (Activity, Milestones, Snippets, Assigned merge + requests, Assigned issues, To-Do List). +- Under a specific group (Activity, Contribution analytics, Issues, Issue boards, + Labels, Milestones, Merge requests). +- Global and Group search are disabled. + +This is to prevent performing too many requests at once to the external +authorization service. + +Whenever access is granted or denied this is logged in a log file called +`external-policy-access-control.log`. Read more about the logs GitLab keeps in +the [Linux package documentation](https://docs.gitlab.com/omnibus/settings/logs.html). + +When using TLS Authentication with a self signed certificate, the CA certificate +needs to be trusted by the OpenSSL installation. When using GitLab installed +using the Linux package, learn to install a custom CA in the +[Linux package documentation](https://docs.gitlab.com/omnibus/settings/ssl/index.html). +Alternatively, learn where to install custom certificates by using +`openssl version -d`. + +## Configuration + +The external authorization service can be enabled by an administrator: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **External authorization**. +1. Complete the fields. +1. Select **Save changes**. + +### Allow external authorization with deploy tokens and deploy keys + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386656) in GitLab 15.9. +> - Deploy tokens no longer being able to access container or package registries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387721) in GitLab 16.0. + +You can set your instance to allow external authorization for Git operations with +[deploy tokens](../../user/project/deploy_tokens/index.md) or [deploy keys](../../user/project/deploy_keys/index.md). + +Prerequisites: + +- You must be using classification labels without a service URL for external authorization. + +To allow authorization with deploy tokens and keys: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **External authorization**, and: + - Leave the service URL field empty. + - Select **Allow deploy tokens and deploy keys to be used with external authorization**. +1. Select **Save changes**. + +WARNING: +If you enable external authorization, deploy tokens cannot access container or package registries. If you use deploy tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use tokens with container or package registries. + +## How it works + +When GitLab requests access, it sends a JSON POST request to the external +service with this body: + +```json +{ + "user_identifier": "jane@acme.org", + "project_classification_label": "project-label", + "user_ldap_dn": "CN=Jane Doe,CN=admin,DC=acme", + "identities": [ + { "provider": "ldap", "extern_uid": "CN=Jane Doe,CN=admin,DC=acme" }, + { "provider": "bitbucket", "extern_uid": "2435223452345" } + ] +} +``` + +The `user_ldap_dn` is optional and is only sent when the user is signed in +through LDAP. + +`identities` contains the details of all the identities associated with the +user. This is an empty array if there are no identities associated with the +user. + +When the external authorization service responds with a status code 200, the +user is granted access. When the external service responds with a status code +401 or 403, the user is denied access. In any case, the request is cached for +six hours. + +When denying access, a `reason` can be optionally specified in the JSON body: + +```json +{ + "reason": "You are not allowed access to this project." +} +``` + +Any other status code than 200, 401 or 403 also deny access to the user, but the +response isn't cached. + +If the service times out (after 500 ms), a message "External Policy Server did +not respond" is displayed. + +## Classification labels + +You can use your own classification label in the project's +**Settings > General > General project settings** page in the "Classification +label" box. When no classification label is specified on a project, the default +label defined in the [global settings](#configuration) is used. + +On all project pages, in the upper-right corner, the label appears. + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/files_api_rate_limits.md b/doc/administration/settings/files_api_rate_limits.md new file mode 100644 index 00000000000..cb5442c957f --- /dev/null +++ b/doc/administration/settings/files_api_rate_limits.md @@ -0,0 +1,51 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Rate limits on Repository files API **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68561) in GitLab 14.3. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75918) in GitLab 14.6. [Feature flag `files_api_throttling`](https://gitlab.com/gitlab-org/gitlab/-/issues/338903) removed. + +The [Repository files API](../../api/repository_files.md) enables you to +fetch, create, update, and delete files in your repository. To improve the security +and durability of your web application, you can enforce +[rate limits](../../security/rate_limits.md) on this API. Any rate limits you +create for the Files API override the [general user and IP rate limits](user_and_ip_rate_limits.md). + +## Define Files API rate limits + +Rate limits for the Files API are disabled by default. When enabled, they supersede +the general user and IP rate limits for requests to the +[Repository files API](../../api/repository_files.md). You can keep any general user +and IP rate limits already in place, and increase or decrease the rate limits +for the Files API. No other new features are provided by this override. + +Prerequisite: + +- You must have administrator access to the instance. + +To override the general user and IP rate limits for requests to the Repository files API: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Files API Rate Limits**. +1. Select the checkboxes for the types of rate limits you want to enable: + - **Unauthenticated API request rate limit** + - **Authenticated API request rate limit** +1. If you selected **unauthenticated**: + 1. Select the **Max unauthenticated API requests per period per IP**. + 1. Select the **Unauthenticated API rate limit period in seconds**. +1. If you selected **authenticated**: + 1. Select the **Max authenticated API requests per period per user**. + 1. Select the **Authenticated API rate limit period in seconds**. + +## Related topics + +- [Rate limits](../../security/rate_limits.md) +- [Repository files API](../../api/repository_files.md) +- [User and IP rate limits](user_and_ip_rate_limits.md) diff --git a/doc/administration/settings/floc.md b/doc/administration/settings/floc.md new file mode 100644 index 00000000000..6bd5a6dfed4 --- /dev/null +++ b/doc/administration/settings/floc.md @@ -0,0 +1,42 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Federated Learning of Cohorts (FLoC) **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60933) in GitLab 13.12. + +Federated Learning of Cohorts (FLoC) is a new feature of the Chrome browser. +It works by categorizing users into different cohorts, so that +advertisers can use this data to uniquely target and track users. For more +information, see the [FLoC repository](https://github.com/WICG/floc). + +To avoid users being tracked and categorized in any GitLab instance, FLoC is +disabled by default by sending the following header: + +```plaintext +Permissions-Policy: interest-cohort=() +``` + +To enable it: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Federated Learning of Cohorts (FLoC)**. +1. Select the **Participate in FLoC** checkbox. +1. Select **Save changes**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/git_lfs_rate_limits.md b/doc/administration/settings/git_lfs_rate_limits.md new file mode 100644 index 00000000000..cb2cc80e397 --- /dev/null +++ b/doc/administration/settings/git_lfs_rate_limits.md @@ -0,0 +1,36 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Rate limits on Git LFS **(FREE SELF)** + +[Git LFS (Large File Storage)](../../topics/git/lfs/index.md) is a Git extension +for handling large files. If you use Git LFS in your repository, common Git operations +can generate many Git LFS requests. You can enforce +[general user and IP rate limits](../settings/user_and_ip_rate_limits.md), but you can also +override the general setting to enforce additional limits on Git LFS requests. This +override can improve the security and durability of your web application. Aside from +precedence, this configuration provides the same features as the general user and IP +rate limits. + +## Configure Git LFS rate limits + +Git LFS rate limits are disabled by default. If enabled and configured, these limits +supersede the [general user and IP rate limits](../settings/user_and_ip_rate_limits.md): + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Git LFS Rate Limits**. +1. Select **Enable authenticated Git LFS request rate limit**. +1. Enter a value for **Max authenticated Git LFS requests per period per user**. +1. Enter a value for **Authenticated Git LFS rate limit period in seconds**. +1. Select **Save changes**. + +## Related topics + +- [Rate limiting](../../security/rate_limits.md) +- [User and IP rate limits](../settings/user_and_ip_rate_limits.md) diff --git a/doc/administration/settings/gitaly_timeouts.md b/doc/administration/settings/gitaly_timeouts.md new file mode 100644 index 00000000000..49dc7763cd0 --- /dev/null +++ b/doc/administration/settings/gitaly_timeouts.md @@ -0,0 +1,27 @@ +--- +stage: Systems +group: Gitaly +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Gitaly timeouts **(FREE SELF)** + +[Gitaly](../gitaly/index.md) timeouts are configurable. The timeouts can be +configured to make sure that long-running Gitaly calls don't needlessly take up resources. + +To access Gitaly timeout settings: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand the **Gitaly timeouts** section. + +## Available timeouts + +The following timeouts are available. + +| Timeout | Default | Description | +|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../operations/puma.md#change-the-worker-timeout) that can be configured for [Puma](../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | +| Fast | 10 seconds | Timeout for fast Gitaly operations used within requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | +| Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly within requests) but preferably not used multiple times within a request. For example, loading blobs. Timeout that should be set between Default and Fast. | diff --git a/doc/administration/settings/help_page.md b/doc/administration/settings/help_page.md new file mode 100644 index 00000000000..46c2c395102 --- /dev/null +++ b/doc/administration/settings/help_page.md @@ -0,0 +1,111 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Customize the Help and sign-in page messages **(FREE SELF)** + +In large organizations, it is useful to have information about who to contact or where +to go for help. You can customize and display this information on the GitLab `/help` page and on +the GitLab sign-in page. + +## Add a help message to the Help page + +You can add a help message, which is shown at the top of the GitLab `/help` page (for example, +<https://gitlab.com/help>): + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sign-in and Help page**. +1. In **Additional text to show on the Help page**, enter the information you want to display on `/help`. +1. Select **Save changes**. + +You can now see the message on `/help`. + +NOTE: +By default, `/help` is visible to unauthenticated users. However, if the +[**Public** visibility level](visibility_and_access_controls.md#restrict-visibility-levels) +is restricted, `/help` is visible only to authenticated users. + +## Add a help message to the sign-in page + +You can add a help message, which is shown on the GitLab sign-in page. The message appears on the sign-in page: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sign-in and Help page**. +1. In **Additional text to show on the sign-in page**, enter the information you want to + display on the sign-in page. +1. Select **Save changes**. + +You can now see the message on the sign-in page. + +## Hide marketing-related entries from the Help page + +GitLab marketing-related entries are occasionally shown on the Help page. To hide these entries: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sign-in and Help page**. +1. Select the **Hide marketing-related entries from the Help page** checkbox. +1. Select **Save changes**. + +## Set a custom Support page URL + +You can specify a custom URL to which users are directed when they: + +- Select **Support** from the Help dropdown list. +- Select **See our website for help** on the Help page. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sign-in and Help page**. +1. In the **Support page URL** field, enter the URL. +1. Select **Save changes**. + +## Redirect `/help` pages + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43157) in GitLab 13.5. +> - [Feature flag `help_page_documentation_redirect`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) removed in GitLab 14.4. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) in GitLab 14.4. + +You can redirect all `/help` links to a destination that meets the [necessary requirements](#destination-requirements). + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sign-in and Help page**. +1. In the **Documentation pages URL** field, enter the URL. +1. Select **Save changes**. + +If the "Documentation pages URL" field is empty, the GitLab instance displays a basic version of the documentation sourced from the [`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. + +### Destination requirements + +When redirecting `/help`, GitLab: + +- Redirects requests to the specified URL. +- Appends `ee` and the documentation path, which includes the version number, to the URL. +- Appends `.html` to the URL, and removes `.md` if necessary. + +For example, if the URL is set to `https://docs.gitlab.com`, requests for +`/help/administration/settings/help_page.md` redirect to: +`https://docs.gitlab.com/${VERSION}/ee/administration/settings/help_page.html`. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/img/bulk_push_event_v12_4.png b/doc/administration/settings/img/bulk_push_event_v12_4.png Binary files differnew file mode 100644 index 00000000000..114e87a61f1 --- /dev/null +++ b/doc/administration/settings/img/bulk_push_event_v12_4.png diff --git a/doc/administration/settings/img/classification_label_on_project_page_v14_8.png b/doc/administration/settings/img/classification_label_on_project_page_v14_8.png Binary files differnew file mode 100644 index 00000000000..4bd2e7d389b --- /dev/null +++ b/doc/administration/settings/img/classification_label_on_project_page_v14_8.png diff --git a/doc/user/admin_area/settings/img/continuous_integration_shared_runner_details_input_v14_10.png b/doc/administration/settings/img/continuous_integration_shared_runner_details_input_v14_10.png Binary files differindex 08451f36962..08451f36962 100644 --- a/doc/user/admin_area/settings/img/continuous_integration_shared_runner_details_input_v14_10.png +++ b/doc/administration/settings/img/continuous_integration_shared_runner_details_input_v14_10.png diff --git a/doc/user/admin_area/settings/img/continuous_integration_shared_runner_details_v14_10.png b/doc/administration/settings/img/continuous_integration_shared_runner_details_v14_10.png Binary files differindex 64bd9cf6911..64bd9cf6911 100644 --- a/doc/user/admin_area/settings/img/continuous_integration_shared_runner_details_v14_10.png +++ b/doc/administration/settings/img/continuous_integration_shared_runner_details_v14_10.png diff --git a/doc/administration/settings/img/custom_git_clone_url_for_https_v12_4.png b/doc/administration/settings/img/custom_git_clone_url_for_https_v12_4.png Binary files differnew file mode 100644 index 00000000000..22cdd15cc0c --- /dev/null +++ b/doc/administration/settings/img/custom_git_clone_url_for_https_v12_4.png diff --git a/doc/administration/settings/img/domain_denylist_v14_1.png b/doc/administration/settings/img/domain_denylist_v14_1.png Binary files differnew file mode 100644 index 00000000000..e27997fc2c2 --- /dev/null +++ b/doc/administration/settings/img/domain_denylist_v14_1.png diff --git a/doc/administration/settings/img/email_notification_for_unknown_sign_ins_v13_2.png b/doc/administration/settings/img/email_notification_for_unknown_sign_ins_v13_2.png Binary files differnew file mode 100644 index 00000000000..fdcc542c4d7 --- /dev/null +++ b/doc/administration/settings/img/email_notification_for_unknown_sign_ins_v13_2.png diff --git a/doc/administration/settings/img/file_template_user_dropdown.png b/doc/administration/settings/img/file_template_user_dropdown.png Binary files differnew file mode 100644 index 00000000000..8c9eb49f6c9 --- /dev/null +++ b/doc/administration/settings/img/file_template_user_dropdown.png diff --git a/doc/administration/settings/img/mirror_settings_v15_7.png b/doc/administration/settings/img/mirror_settings_v15_7.png Binary files differnew file mode 100644 index 00000000000..5da41dbeceb --- /dev/null +++ b/doc/administration/settings/img/mirror_settings_v15_7.png diff --git a/doc/administration/settings/img/protected_paths.png b/doc/administration/settings/img/protected_paths.png Binary files differnew file mode 100644 index 00000000000..2233a71a139 --- /dev/null +++ b/doc/administration/settings/img/protected_paths.png diff --git a/doc/administration/settings/img/push_event_activities_limit_v12_4.png b/doc/administration/settings/img/push_event_activities_limit_v12_4.png Binary files differnew file mode 100644 index 00000000000..ea618ad4c50 --- /dev/null +++ b/doc/administration/settings/img/push_event_activities_limit_v12_4.png diff --git a/doc/administration/settings/img/rate_limit_on_issues_creation_v14_2.png b/doc/administration/settings/img/rate_limit_on_issues_creation_v14_2.png Binary files differnew file mode 100644 index 00000000000..1a0a7548a91 --- /dev/null +++ b/doc/administration/settings/img/rate_limit_on_issues_creation_v14_2.png diff --git a/doc/administration/settings/img/rate_limits_on_raw_endpoints.png b/doc/administration/settings/img/rate_limits_on_raw_endpoints.png Binary files differnew file mode 100644 index 00000000000..c59f67df1ce --- /dev/null +++ b/doc/administration/settings/img/rate_limits_on_raw_endpoints.png diff --git a/doc/administration/settings/img/restricted_url.png b/doc/administration/settings/img/restricted_url.png Binary files differnew file mode 100644 index 00000000000..c71abf0a226 --- /dev/null +++ b/doc/administration/settings/img/restricted_url.png diff --git a/doc/administration/settings/img/sign_up_terms.png b/doc/administration/settings/img/sign_up_terms.png Binary files differnew file mode 100644 index 00000000000..4cac9d426c9 --- /dev/null +++ b/doc/administration/settings/img/sign_up_terms.png diff --git a/doc/user/admin_area/settings/img/suggest_pipeline_banner_v14_5.png b/doc/administration/settings/img/suggest_pipeline_banner_v14_5.png Binary files differindex 0d9bfa4a173..0d9bfa4a173 100644 --- a/doc/user/admin_area/settings/img/suggest_pipeline_banner_v14_5.png +++ b/doc/administration/settings/img/suggest_pipeline_banner_v14_5.png diff --git a/doc/administration/settings/img/two_factor_grace_period.png b/doc/administration/settings/img/two_factor_grace_period.png Binary files differnew file mode 100644 index 00000000000..e7fb52969aa --- /dev/null +++ b/doc/administration/settings/img/two_factor_grace_period.png diff --git a/doc/administration/settings/img/update-available.png b/doc/administration/settings/img/update-available.png Binary files differnew file mode 100644 index 00000000000..9887e06c7dc --- /dev/null +++ b/doc/administration/settings/img/update-available.png diff --git a/doc/administration/settings/import_export_rate_limits.md b/doc/administration/settings/import_export_rate_limits.md new file mode 100644 index 00000000000..99385c77cdf --- /dev/null +++ b/doc/administration/settings/import_export_rate_limits.md @@ -0,0 +1,31 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limits for imports and exports of project and groups **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35728) in GitLab 13.2. + +You can configure the rate limits for imports and exports of projects and groups: + +To change a rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Import and export rate limits**. +1. Change the value of any rate limit. The rate limits are per minute per user, not per IP address. + Set to `0` to disable a rate limit. + +| Limit | Default | +|-------------------------|---------| +| Project Import | 6 | +| Project Export | 6 | +| Project Export Download | 1 | +| Group Import | 6 | +| Group Export | 6 | +| Group Export Download | 1 | + +When a user exceeds a rate limit, it is logged in `auth.log`. diff --git a/doc/administration/settings/incident_management_rate_limits.md b/doc/administration/settings/incident_management_rate_limits.md new file mode 100644 index 00000000000..2a74c843107 --- /dev/null +++ b/doc/administration/settings/incident_management_rate_limits.md @@ -0,0 +1,39 @@ +--- +type: reference +stage: Monitor +group: Respond +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Incident management rate limits **(ULTIMATE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17859) in GitLab 12.5. + +You can limit the number of inbound alerts for [incidents](../../operations/incident_management/incidents.md) +that can be created in a period of time. The inbound [incident management](../../operations/incident_management/index.md) +alert limit can help prevent overloading your incident responders by reducing the +number of alerts or duplicate issues. + +As an example, if you set a limit of `10` requests every `60` seconds, +and `11` requests are sent to an [alert integration endpoint](../../operations/incident_management/integrations.md) within one minute, +the eleventh request is blocked. Access to the endpoint is allowed again after one minute. + +This limit is: + +- Applied independently per project. +- Not applied per IP address. +- Disabled by default. + +Requests that exceed the limit are logged into `auth.log`. + +## Set a limit on inbound alerts + +To set inbound incident management alert limits: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Incident Management Limits**. +1. Select the **Enable Incident Management inbound alert limit** checkbox. +1. Optional. Input a custom value for **Maximum requests per project per rate limit period**. Default is 3600. +1. Optional. Input a custom value for **Rate limit period**. Default is 3600 seconds. diff --git a/doc/administration/settings/index.md b/doc/administration/settings/index.md new file mode 100644 index 00000000000..5fc0c029c30 --- /dev/null +++ b/doc/administration/settings/index.md @@ -0,0 +1,215 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index +--- + +# Admin Area settings **(FREE SELF)** + +As an administrator of a GitLab self-managed instance, you can manage the behavior of your +deployment. + +The **Admin Area** is not accessible on GitLab.com, and settings can only be changed by the +GitLab.com administrators. For the settings and limits on the GitLab.com instance, +read [GitLab.com settings](../../user/gitlab_com/index.md). + +## Access the Admin Area + +To access the **Admin Area**: + +1. Sign in to your GitLab instance as an administrator. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings**, and the group of settings to view: + - [General](#general) + - [Geo](#geo) + - [CI/CD](#cicd) + - [Integrations](#integrations) + - [Metrics and profiling](#metrics-and-profiling) + - [Network](#network) + - [Preferences](#preferences) + - [Reporting](#reporting) + - [Repository](#repository) + - [Templates](#templates) + +### General + +The **General** settings contain: + +- [Visibility and access controls](../settings/visibility_and_access_controls.md) - Set default and + restrict visibility levels. Configure import sources and Git access protocol. +- [Account and limit](../settings/account_and_limit_settings.md) - Set projects and maximum size limits, + session duration, user options, and check feature availability for namespace plan. +- [Diff limits](../diff_limits.md) - Diff content limits. +- [Sign-up restrictions](../settings/sign_up_restrictions.md) - Configure the way a user creates a new account. +- [Sign in restrictions](../settings/sign_in_restrictions.md) - Set requirements for a user to sign in. + Enable mandatory two-factor authentication. +- [Terms of Service and Privacy Policy](../settings/terms.md) - Include a Terms of Service agreement + and Privacy Policy that all users must accept. +- [External Authentication](../../administration/settings/external_authorization.md#configuration) - External Classification Policy Authorization. +- [Web terminal](../integration/terminal.md#limiting-websocket-connection-time) - + Set max session time for web terminal. +- [FLoC](floc.md) - Enable or disable + [Federated Learning of Cohorts (FLoC)](https://en.wikipedia.org/wiki/Federated_Learning_of_Cohorts) tracking. +- [GitLab for Slack app](../../user/admin_area/settings/slack_app.md) - Enable and configure the GitLab for Slack app. + +### CI/CD + +The **CI/CD** settings contain: + +- [Continuous Integration and Deployment](../../administration/settings/continuous_integration.md) - + Auto DevOps, runners and job artifacts. +- [Required pipeline configuration](../../administration/settings/continuous_integration.md#required-pipeline-configuration) - + Set an instance-wide auto included [pipeline configuration](../../ci/yaml/index.md). + This pipeline configuration is run after the project's own configuration. +- [Package Registry](../../administration/settings/continuous_integration.md#package-registry-configuration) - + Settings related to the use and experience of using the GitLab Package Registry. Some + [risks are involved](../../user/packages/container_registry/reduce_container_registry_storage.md#use-with-external-container-registries) + in enabling some of these settings. + +## Security and Compliance settings + +- [License compliance settings](security_and_compliance.md#choose-package-registry-metadata-to-sync): Enable or disable synchronization of package metadata by a registry type. + +### Geo **(PREMIUM SELF)** + +The **Geo** setting contains: + +- [Geo](../geo/index.md) - Replicate your GitLab instance to other + geographical locations. Redirects to **Admin Area > Geo > Settings** are no + longer available at **Admin Area > Settings > Geo** in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/36896). + +### Integrations + +The **Integrations** settings contain: + +- [Elasticsearch](../../integration/advanced_search/elasticsearch.md#enable-advanced-search) - + Elasticsearch integration. Elasticsearch AWS IAM. +- [Kroki](../integration/kroki.md#enable-kroki-in-gitlab) - + Allow rendering of diagrams in AsciiDoc and Markdown documents using [kroki.io](https://kroki.io). +- [Mailgun](../integration/mailgun.md) - Enable your GitLab instance + to receive invite email bounce events from Mailgun, if it is your email provider. +- [PlantUML](../integration/plantuml.md) - Allow rendering of PlantUML + diagrams in documents. +- [Customer experience improvement and third-party offers](../settings/third_party_offers.md) - + Control the display of customer experience improvement content and third-party offers. +- [Snowplow](../../development/internal_analytics/snowplow/index.md) - Configure the Snowplow integration. +- [Google GKE](../../user/project/clusters/add_gke_clusters.md) - Google GKE integration enables + you to provision GKE clusters from GitLab. +- [Amazon EKS](../../user/project/clusters/add_eks_clusters.md) - Amazon EKS integration enables + you to provision EKS clusters from GitLab. + +### Metrics and profiling + +The **Metrics and profiling** settings contain: + +- [Metrics - Prometheus](../monitoring/prometheus/gitlab_metrics.md) - + Enable and configure Prometheus metrics. +- [Metrics - Grafana](../monitoring/performance/grafana_configuration.md#integrate-with-gitlab-ui) - + Enable and configure Grafana. +- [Profiling - Performance bar](../monitoring/performance/performance_bar.md#enable-the-performance-bar-for-non-administrators) - + Enable access to the Performance Bar for non-administrator users in a given group. +- [Usage statistics](../settings/usage_statistics.md) - Enable or disable version check and Service Ping. + +### Network + +The **Network** settings contain: + +- Performance optimization - Various settings that affect GitLab performance, including: + - [Write to `authorized_keys` file](../operations/fast_ssh_key_lookup.md#set-up-fast-lookup). + - [Push event activities limit and bulk push events](../settings/push_event_activities_limit.md). +- [User and IP rate limits](../settings/user_and_ip_rate_limits.md) - Configure limits for web and API requests. + These rate limits can be overridden: + - [Package Registry Rate Limits](../settings/package_registry_rate_limits.md) - Configure specific + limits for Packages API requests that supersede the user and IP rate limits. + - [Git LFS Rate Limits](../settings/git_lfs_rate_limits.md) - Configure specific limits for + Git LFS requests that supersede the user and IP rate limits. + - [Files API Rate Limits](../settings/files_api_rate_limits.md) - Configure specific limits for + Files API requests that supersede the user and IP rate limits. + - [Search rate limits](../instance_limits.md#search-rate-limit) - Configure global search request rate limits for authenticated and unauthenticated users. + - [Deprecated API Rate Limits](../settings/deprecated_api_rate_limits.md) - Configure specific limits + for deprecated API requests that supersede the user and IP rate limits. +- [Outbound requests](../../security/webhooks.md) - Allow requests to the local network from webhooks and integrations, or deny all outbound requests. +- [Protected Paths](../settings/protected_paths.md) - Configure paths to be protected by Rack Attack. +- [Incident Management Limits](../../operations/incident_management/index.md) - Limit the + number of inbound alerts that can be sent to a project. +- [Notes creation limit](../settings/rate_limit_on_notes_creation.md) - Set a rate limit on the note creation requests. +- [Get single user limit](../settings/rate_limit_on_users_api.md) - Set a rate limit on users API endpoint to get a user by ID. +- [Projects API rate limits for unauthenticated requests](../settings/rate_limit_on_projects_api.md) - Set a rate limit on Projects list API endpoint for unauthenticated requests. + +### Preferences + +The **Preferences** settings contain: + +- [Email](../settings/email.md) - Various email settings. +- [What's new](../whats-new.md) - Configure **What's new** drawer and content. +- [Help page](help_page.md) - Help page text and support page URL. +- [Pages](../pages/index.md#custom-domain-verification) - + Size and domain settings for static websites. +- [Polling interval multiplier](../polling.md) - + Configure how frequently the GitLab UI polls for updates. +- [Gitaly timeouts](gitaly_timeouts.md) - Configure Gitaly timeouts. +- Localization: + - [Default first day of the week](../../user/profile/preferences.md). + - [Time tracking](../../user/project/time_tracking.md#limit-displayed-units-to-hours). +- [Sidekiq Job Limits](../settings/sidekiq_job_limits.md) - Limit the size of Sidekiq jobs stored in Redis. + +### Reporting + +The **Reporting** settings contain: + +- Spam and Anti-bot protection: + - Anti-spam services, such as [reCAPTCHA](../../integration/recaptcha.md), + [Akismet](../../integration/akismet.md), or [Spamcheck](../reporting/spamcheck.md). + - [IP address restrictions](../reporting/ip_addr_restrictions.md). +- [Abuse reports](../review_abuse_reports.md) - Set notification email for abuse reports. +- [Git abuse rate limit](../reporting/git_abuse_rate_limit.md) - Configure Git abuse rate limit settings. **(ULTIMATE SELF)** + +### Repository + +The **Repository** settings contain: + +- [Repository's custom initial branch name](../../user/project/repository/branches/default.md#instance-level-custom-initial-branch-name) - + Set a custom branch name for new repositories created in your instance. +- [Repository's initial default branch protection](../../user/project/repository/branches/default.md#instance-level-default-branch-protection) - + Configure the branch protections to apply to every repository's default branch. +- [Repository mirror](visibility_and_access_controls.md#enable-project-mirroring) - + Configure repository mirroring. +- [Repository storage](../repository_storage_types.md) - Configure storage path settings. +- Repository maintenance: + - [Repository checks](../repository_checks.md) - Configure + automatic Git checks on repositories. + - [Housekeeping](../housekeeping.md). Configure automatic + Git housekeeping on repositories. + - [Inactive project deletion](../inactive_project_deletion.md). Configure inactive + project deletion. +- [Repository static objects](../static_objects_external_storage.md) - + Serve repository static objects (for example, archives and blobs) from an external storage (for example, a CDN). + +### Templates **(PREMIUM SELF)** + +The **Templates** settings contain: + +- [Templates](instance_template_repository.md#configuration) - Set instance-wide template repository. +- [Custom project templates](../custom_project_templates.md) - Select the custom project template source group. + +## Default first day of the week + +You can change the [Default first day of the week](../../user/profile/preferences.md) +for the entire GitLab instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Scroll to the **Localization** section, and select your desired first day of the week. + +## Default language + +You can change the [Default language](../../user/profile/preferences.md) +for the entire GitLab instance: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Scroll to the **Localization** section, and select your desired default language. diff --git a/doc/administration/settings/instance_template_repository.md b/doc/administration/settings/instance_template_repository.md new file mode 100644 index 00000000000..8c8c9f44998 --- /dev/null +++ b/doc/administration/settings/instance_template_repository.md @@ -0,0 +1,84 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +type: reference +--- + +# Instance template repository **(PREMIUM SELF)** + +> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52360) to behave like group-level templates in GitLab 13.9. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321247) in GitLab 14.0. + +In hosted systems, enterprises often have a need to share their own templates +across teams. This feature allows an administrator to pick a project to be the +instance-wide collection of file templates. These templates are then exposed to +all users through the [Web Editor](../../user/project/repository/web_editor.md) +while the project remains secure. + +## Configuration + +To select a project to serve as the custom template repository: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Templates**. +1. Expand **Templates** +1. From the dropdown list, select the project to use as the template repository. +1. Select **Save changes**. +1. Add custom templates to the selected repository. + +After you add templates, you can use them for the entire instance. +They are available in the [Web Editor](../../user/project/repository/web_editor.md) +and through the [API settings](../../api/settings.md). + +## Supported file types and locations + +Templates must be added to a specific subdirectory in the repository, +corresponding to the kind of template. The following types of custom templates +are supported: + +| Type | Directory | Extension | +| :---------------: | :-----------: | :-----------: | +| `Dockerfile` | `Dockerfile` | `.dockerfile` | +| `.gitignore` | `gitignore` | `.gitignore` | +| `.gitlab-ci.yml` | `gitlab-ci` | `.yml` | +| `LICENSE` | `LICENSE` | `.txt` | + +Each template must go in its respective subdirectory, have the correct +extension and not be empty. So, the hierarchy should look like this: + +```plaintext +|-- README.md +|-- Dockerfile + |-- custom_dockerfile.dockerfile + |-- another_dockerfile.dockerfile +|-- gitignore + |-- custom_gitignore.gitignore + |-- another_gitignore.gitignore +|-- gitlab-ci + |-- custom_gitlab-ci.yml + |-- another_gitlab-ci.yml +|-- LICENSE + |-- custom_license.txt + |-- another_license.txt +``` + +Your custom templates are displayed on the dropdown list when a new file is added through the GitLab UI: + + + +If this feature is disabled or no templates are present, +no **Custom** section displays in the selection dropdown. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/package_registry_rate_limits.md b/doc/administration/settings/package_registry_rate_limits.md new file mode 100644 index 00000000000..ffba5bbf15a --- /dev/null +++ b/doc/administration/settings/package_registry_rate_limits.md @@ -0,0 +1,57 @@ +--- +stage: Package +group: Package Registry +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Package Registry Rate Limits **(FREE SELF)** + +With the [GitLab Package Registry](../../user/packages/package_registry/index.md), +you can use GitLab as a private or public registry for a variety of common package managers. You can +publish and share packages, which others can consume as a dependency in downstream projects through +the [Packages API](../../api/packages.md). + +If downstream projects frequently download such dependencies, many requests are made through the +Packages API. You may therefore reach enforced [user and IP rate limits](../settings/user_and_ip_rate_limits.md). +To address this issue, you can define specific rate limits for the Packages API: + +- [Unauthenticated requests (per IP)](#enable-unauthenticated-request-rate-limit-for-packages-api). +- [Authenticated API requests (per user)](#enable-authenticated-api-request-rate-limit-for-packages-api). + +These limits are disabled by default. + +When enabled, they supersede the general user and IP rate limits for requests to +the Packages API. You can therefore keep the general user and IP rate limits, and +increase the rate limits for the Packages API. Besides this precedence, there is +no difference in functionality compared to the general user and IP rate limits. + +## Enable unauthenticated request rate limit for packages API + +To enable the unauthenticated request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Package registry rate limits**. +1. Select **Enable unauthenticated request rate limit**. + + - Optional. Update the **Maximum unauthenticated requests per rate limit period per IP** value. + Defaults to `800`. + - Optional. Update the **Unauthenticated rate limit period in seconds** value. + Defaults to `15`. + +## Enable authenticated API request rate limit for packages API + +To enable the authenticated API request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network** +1. Expand **Package registry rate limits**. +1. Select **Enable authenticated API request rate limit**. + + - Optional. Update the **Maximum authenticated API requests per rate limit period per user** value. + Defaults to `1000`. + - Optional. Update the **Authenticated API rate limit period in seconds** value. + Defaults to `15`. diff --git a/doc/administration/settings/project_integration_management.md b/doc/administration/settings/project_integration_management.md new file mode 100644 index 00000000000..1bb4465020c --- /dev/null +++ b/doc/administration/settings/project_integration_management.md @@ -0,0 +1,138 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Project integration management **(FREE)** + +Project integrations can be configured and enabled by project administrators. As a GitLab instance +administrator, you can set default configuration parameters for a given integration that all projects +can inherit and use, enabling the integration for all projects that are not already using custom +settings. + +You can update these default settings at any time, changing the settings used for all projects that +are set to use instance-level or group-level defaults. Updating the default settings also enables the integration +for all projects that didn't have it already enabled. + +Only the complete settings for an integration can be inherited. Per-field inheritance is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). + +## Manage instance-level default settings for a project integration **(FREE SELF)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2137) in GitLab 13.3 for project-level integrations. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6 for group-level integrations. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Integrations**. +1. Select an integration. +1. Enter configuration details and select **Save changes**. + +WARNING: +This may affect all or most of the groups and projects on your GitLab instance. Review the details +below. + +If this is the first time you are setting up instance-level settings for an integration: + +- The integration is enabled for all groups and projects that don't already have this integration configured, + if you have the **Enable integration** toggle turned on in the instance-level settings. +- Groups and projects that already have the integration configured are not affected, but can choose to use the + inherited settings at any time. + +When you make further changes to the instance defaults: + +- They are immediately applied to all groups and projects that have the integration set to use default settings. +- They are immediately applied to newer groups and projects, created after you last saved defaults for the + integration. If your instance-level default setting has the **Enable integration** toggle turned + on, the integration is automatically enabled for all such groups and projects. +- Groups and projects with custom settings selected for the integration are not immediately affected and may + choose to use the latest defaults at any time. + +Only the complete settings for an integration can be inherited. Per-field inheritance +is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow +administrators to update settings inherited by groups and projects without enabling the +integration on all non-configured groups and projects by default. + +### Remove an instance-level default setting + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Integrations**. +1. Select an integration. +1. Select **Reset** and confirm. + +Resetting an instance-level default setting removes the integration from all projects that have the integration set to use default settings. + +### View projects that override the default settings + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218252) in GitLab 14.2. + +You can view which projects in your instance use custom settings that [override the instance-level default settings](#use-custom-settings-for-a-group-or-project-integration) +for an integration. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Integrations**. +1. Select an integration. +1. Select the **Projects using custom settings** tab. + +## Manage group-level default settings for a project integration + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6. + +1. Navigate to the group's **Settings > Integrations**. +1. Select an integration. +1. Enter configuration details and select **Save changes**. + +WARNING: +This may affect all or most of the subgroups and projects belonging to the group. Review the details below. + +If this is the first time you are setting up group-level settings for an integration: + +- The integration is enabled for all subgroups and projects belonging to the group that don't already have + this integration configured, if you have the **Enable integration** toggle turned on in the group-level + settings. +- Subgroups and projects that already have the integration configured are not affected, but can choose to use + the inherited settings at any time. + +When you make further changes to the group defaults: + +- They are immediately applied to all subgroups and projects belonging to the group that have the integration + set to use default settings. +- They are immediately applied to newer subgroups and projects, even those created after you last saved defaults for the + integration. If your group-level default setting has the **Enable integration** toggle turned on, + the integration is automatically enabled for all such subgroups and projects. + +- Subgroups and projects with custom settings selected for the integration are not immediately affected and + may choose to use the latest defaults at any time. + +Only the complete settings for an integration can be inherited. Per-field inheritance +is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow +administrators to update settings inherited by subgroups and projects without enabling the +integration on all non-configured groups and projects by default. + +### Remove a group-level default setting + +1. Navigate to the group's **Settings > Integrations**. +1. Select an integration. +1. Select **Reset** and confirm. + +Resetting a group-level default setting removes integrations that use default settings and belong to a project or subgroup of the group. + +## Use instance-level or group-level default settings for a project integration + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6 for group-level settings. + +1. Navigate to **Project > Settings > Integrations**. +1. Choose the integration you want to enable or update. +1. From the dropdown list, select **Use default settings**. +1. Ensure the toggle is set to **Enable integration**. +1. Select **Save changes**. + +## Use custom settings for a group or project integration + +1. Navigate to project or group's **Settings > Integrations**. +1. Choose the integration you want to enable or update. +1. From the dropdown list, select **Use custom settings**. +1. Ensure the toggle is set to **Enable integration** and enter all required settings. +1. Select **Save changes**. diff --git a/doc/administration/settings/protected_paths.md b/doc/administration/settings/protected_paths.md new file mode 100644 index 00000000000..ba257983619 --- /dev/null +++ b/doc/administration/settings/protected_paths.md @@ -0,0 +1,43 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Protected paths **(FREE SELF)** + +Rate limiting is a technique that improves the security and durability of a web +application. For more details, see [Rate limits](../../security/rate_limits.md). + +You can rate limit (protect) specified paths. For these paths, GitLab responds with HTTP status +code `429` to POST requests at protected paths that exceed 10 requests per minute per IP address. + +For example, the following are limited to a maximum 10 requests per minute: + +- User sign-in +- User sign-up (if enabled) +- User password reset + +After 10 requests, the client must wait 60 seconds before it can try again. + +See also: + +- List of paths [protected by default](../instance_limits.md#by-protected-path). +- [User and IP rate limits](user_and_ip_rate_limits.md#response-headers) + for the headers returned to blocked requests. + +## Configure protected paths + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31246) in GitLab 12.4. + +Throttling of protected paths is enabled by default and can be disabled or +customized on **Admin > Network > Protected Paths**, along with these options: + +- Maximum number of requests per period per user. +- Rate limit period in seconds. +- Paths to be protected. + + + +Requests over the rate limit are logged into `auth.log`. diff --git a/doc/administration/settings/push_event_activities_limit.md b/doc/administration/settings/push_event_activities_limit.md new file mode 100644 index 00000000000..117e7322e30 --- /dev/null +++ b/doc/administration/settings/push_event_activities_limit.md @@ -0,0 +1,38 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +type: reference +--- + +# Push event activities limit and bulk push events **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31007) in GitLab 12.4. + +Set the number of branches or tags to limit the number of single push events +allowed at once. If the number of events is greater than this, GitLab creates +bulk push event instead. + +For example, if 4 branches are pushed and the limit is currently set to 3, +the activity feed displays: + + + +With this feature, when a single push includes a lot of changes (for example, 1,000 +branches), only 1 bulk push event is created instead of 1,000 push +events. This helps in maintaining good system performance and preventing spam on +the activity feed. + +To modify this setting: + +- In the Admin Area: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. Select **Settings > Network**. + 1. Expand **Performance optimization**. +- Through the [Application settings API](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls) + as `push_event_activities_limit`. + +The default value is `3`, but the value can be greater than or equal to `0`. Setting this value to `0` does not disable throttling. + + diff --git a/doc/administration/settings/rate_limit_on_issues_creation.md b/doc/administration/settings/rate_limit_on_issues_creation.md new file mode 100644 index 00000000000..d8bc04fcdd3 --- /dev/null +++ b/doc/administration/settings/rate_limit_on_issues_creation.md @@ -0,0 +1,36 @@ +--- +type: reference +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limits on issue creation **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28129) in GitLab 12.10. + +This setting allows you to rate limit the requests to the issue and epic creation endpoints. +To can change its value: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Issues Rate Limits**. +1. Under **Max requests per minute**, enter the new value. +1. Select **Save changes**. + +For example, if you set a limit of 300, requests using the +[Projects::IssuesController#create](https://gitlab.com/gitlab-org/gitlab/blob/master/app/controllers/projects/issues_controller.rb) +action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. + +When using [epics](../../user/group/epics/index.md), epic creation shares this rate limit with issues. + + + +This limit is: + +- Applied independently per project and per user. +- Not applied per IP address. +- Disabled by default. To enable it, set the option to any value other than `0`. + +Requests over the rate limit are logged into the `auth.log` file. diff --git a/doc/administration/settings/rate_limit_on_notes_creation.md b/doc/administration/settings/rate_limit_on_notes_creation.md new file mode 100644 index 00000000000..59548836e78 --- /dev/null +++ b/doc/administration/settings/rate_limit_on_notes_creation.md @@ -0,0 +1,35 @@ +--- +type: reference +stage: Plan +group: Project Management +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limits on note creation **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53637) in GitLab 13.9. + +You can configure the per-user rate limit for requests to the note creation endpoint. + +To change the note creation rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Notes rate limit**. +1. In the **Maximum requests per minute** box, enter the new value. +1. Optional. In the **Users to exclude from the rate limit** box, list users allowed to exceed the limit. +1. Select **Save changes**. + +This limit is: + +- Applied independently per user. +- Not applied per IP address. + +The default value is `300`. + +Requests over the rate limit are logged into the `auth.log` file. + +For example, if you set a limit of 300, requests using the +[Projects::NotesController#create](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/notes_controller.rb) +action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. diff --git a/doc/administration/settings/rate_limit_on_pipelines_creation.md b/doc/administration/settings/rate_limit_on_pipelines_creation.md new file mode 100644 index 00000000000..19e1410ef73 --- /dev/null +++ b/doc/administration/settings/rate_limit_on_pipelines_creation.md @@ -0,0 +1,35 @@ +--- +type: reference +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limits on pipeline creation **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362475) in GitLab 15.0. + +You can set a limit so that users and processes can't request more than a certain number of pipelines each minute. This limit can help save resources and improve stability. + +For example, if you set a limit of `10`, and `11` requests are sent to the [trigger API](../../ci/triggers/index.md) within one minute, +the eleventh request is blocked. Access to the endpoint is allowed again after one minute. + +This limit is: + +- Applied to the number of pipelines created for the same combination of project, commit, and user. +- Not applied per IP address. +- Disabled by default. + +Requests that exceed the limit are logged in the `application_json.log` file. + +## Set a pipeline request limit + +To limit the number of pipeline requests: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Pipelines Rate Limits**. +1. Under **Max requests per minute**, enter a value greater than `0`. +1. Select **Save changes**. +1. Enable `ci_enforce_throttle_pipelines_creation` feature flag to enable the rate limit. diff --git a/doc/administration/settings/rate_limit_on_projects_api.md b/doc/administration/settings/rate_limit_on_projects_api.md new file mode 100644 index 00000000000..2192e4355c0 --- /dev/null +++ b/doc/administration/settings/rate_limit_on_projects_api.md @@ -0,0 +1,37 @@ +--- +type: reference +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limit on Projects API **(FREE SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112283) in GitLab 15.10 with a [flag](../feature_flags.md) named `rate_limit_for_unauthenticated_projects_api_access`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/391922) on May 08, 2023. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119603) in GitLab 16.0 by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120445) in GitLab 16.0. Feature flag `rate_limit_for_unauthenticated_projects_api_access` removed. + +You can configure the rate limit per IP address for unauthenticated requests to the [list all projects API](../../api/projects.md#list-all-projects). + +To change the rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Projects API rate limit**. +1. In the **Maximum requests per 10 minutes per IP address** text box, enter the new value. +1. Select **Save changes**. + +The rate limit: + +- Applies per IP address. +- Doesn't apply to authenticated requests. +- Can be set to 0 to disable rate limiting. + +The default value of the rate limit is `400`. + +Requests over the rate limit are logged into the `auth.log` file. + +For example, if you set a limit of 400, unauthenticated requests to the `GET /projects` API endpoint that +exceed a rate of 400 within 10 minutes are blocked. Access to the endpoint is restored after ten minutes have elapsed. diff --git a/doc/administration/settings/rate_limit_on_users_api.md b/doc/administration/settings/rate_limit_on_users_api.md new file mode 100644 index 00000000000..9424e508d86 --- /dev/null +++ b/doc/administration/settings/rate_limit_on_users_api.md @@ -0,0 +1,34 @@ +--- +type: reference +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Rate limits on Users API **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78364) in GitLab 14.8. + +You can configure the per user rate limit for requests to [Users API](../../api/users.md). + +To change the rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Users API rate limit**. +1. In the **Maximum requests per 10 minutes** text box, enter the new value. +1. Optional. In the **Users to exclude from the rate limit** box, list users allowed to exceed the limit. +1. Select **Save changes**. + +This limit is: + +- Applied independently per user. +- Not applied per IP address. + +The default value is `300`. + +Requests over the rate limit are logged into the `auth.log` file. + +For example, if you set a limit of 300, requests to the `GET /users/:id` API endpoint +exceeding a rate of 300 per 10 minutes are blocked. Access to the endpoint is allowed after ten minutes have elapsed. diff --git a/doc/administration/settings/rate_limits_on_git_ssh_operations.md b/doc/administration/settings/rate_limits_on_git_ssh_operations.md new file mode 100644 index 00000000000..64acb15b8ac --- /dev/null +++ b/doc/administration/settings/rate_limits_on_git_ssh_operations.md @@ -0,0 +1,33 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Rate limits on Git SSH operations **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78373) in GitLab 14.7 [with a flag](../feature_flags.md) named `rate_limit_gitlab_shell`. Available by default without a feature flag from 15.8. + +GitLab applies rate limits to Git operations that use SSH by user account and project. When the rate limit is exceeded, GitLab rejects +further connection requests from that user for the project. + +The rate limit applies at the Git command ([plumbing](https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain)) level. +Each command has a rate limit of 600 per minute. For example: + +- `git push` has a rate limit of 600 per minute. +- `git pull` has its own rate limit of 600 per minute. + +Because the same commands are shared by `git-upload-pack`, `git pull`, and `git clone`, they share a rate limit. + +Users on self-managed GitLab can disable this rate limit. + +## Configure GitLab Shell operation limit + +`Git operations using SSH` is enabled by default. Defaults to 600 per user per minute. + +1. On the left sidebar, select **Your work > Admin Area**. +1. On the left sidebar, select **Settings > Network**. +1. Expand **Git SSH operations rate limit**. +1. Enter a value for **Maximum number of Git operations per minute**. +1. Select **Save changes**. diff --git a/doc/administration/settings/rate_limits_on_raw_endpoints.md b/doc/administration/settings/rate_limits_on_raw_endpoints.md new file mode 100644 index 00000000000..78e65f7ba7b --- /dev/null +++ b/doc/administration/settings/rate_limits_on_raw_endpoints.md @@ -0,0 +1,29 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Rate limits on raw endpoints **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30635) in GitLab 12.2. + +This setting defaults to `300` requests per minute, and allows you to rate limit the requests to raw endpoints: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **Performance optimization**. + +For example, requests over `300` per minute to `https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/controllers/application_controller.rb` are blocked. Access to the raw file is released after 1 minute. + + + +This limit is: + +- Applied independently per project, per file path. +- Not applied per IP address. +- Active by default. To disable, set the option to `0`. + +Requests over the rate limit are logged into `auth.log`. diff --git a/doc/administration/settings/scim_setup.md b/doc/administration/settings/scim_setup.md new file mode 100644 index 00000000000..6a02a5b832c --- /dev/null +++ b/doc/administration/settings/scim_setup.md @@ -0,0 +1,43 @@ +--- +type: reference, howto +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Configure SCIM for self-managed GitLab instances **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8902) in GitLab 15.8. + +You can use the open standard System for Cross-domain Identity Management (SCIM) to automatically: + +- Create users. +- Block users. + +The [internal GitLab SCIM API](../../development/internal_api/index.md#instance-scim-api) implements part of [the RFC7644 protocol](https://www.rfc-editor.org/rfc/rfc7644). + +If you are a GitLab.com user, see [configuring SCIM for GitLab.com groups](../../user/group/saml_sso/scim_setup.md). + +## Configure GitLab + +Prerequisites: + +- Configure [SAML single sign-on](../../integration/saml.md). + +To configure GitLab SCIM: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **SCIM Token** section and select **Generate a SCIM token**. +1. For configuration of your identity provider, save the: + - Token from the **Your SCIM token** field. + - URL from the **SCIM API endpoint URL** field. + +## Remove access + +Removing or deactivating a user on the identity provider blocks the user on +the GitLab instance, while the SCIM identity remains linked to the GitLab user. + +To update the user SCIM identity, use the +[internal GitLab SCIM API](../../development/internal_api/index.md#update-a-single-scim-provisioned-user-1). diff --git a/doc/administration/settings/security_and_compliance.md b/doc/administration/settings/security_and_compliance.md new file mode 100644 index 00000000000..2237866ad9c --- /dev/null +++ b/doc/administration/settings/security_and_compliance.md @@ -0,0 +1,25 @@ +--- +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: howto +--- + +# Security and Compliance Admin Area settings **(ULTIMATE SELF)** + +The settings for package metadata synchronization are located in the [Admin Area](index.md). + +## Choose package registry metadata to sync + +WARNING: +The full package metadata sync can add up to 30 GB to the PostgreSQL database. Ensure you have provisioned enough disk space for the database before enabling this feature. +We are actively working on reducing this data size in [epic 10415](https://gitlab.com/groups/gitlab-org/-/epics/10415). + +To choose the packages you want to synchronize with the GitLab License Database for [License Compliance](../../user/compliance/license_scanning_of_cyclonedx_files/index.md): + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Security and Compliance**. +1. Expand **License Compliance**. +1. Select or clear checkboxes for the package registries that you want to sync. +1. Select **Save changes**. diff --git a/doc/administration/settings/sidekiq_job_limits.md b/doc/administration/settings/sidekiq_job_limits.md new file mode 100644 index 00000000000..d5cd24c5237 --- /dev/null +++ b/doc/administration/settings/sidekiq_job_limits.md @@ -0,0 +1,35 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Sidekiq job size limits **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68982) in GitLab 14.3. + +[Sidekiq](../sidekiq/index.md) jobs get stored in +Redis. To avoid excessive memory for Redis, we: + +- Compress job arguments before storing them in Redis. +- Reject jobs that exceed the specified threshold limit after compression. + +To access Sidekiq job size limits: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Sidekiq job size limits**. +1. Adjust the compression threshold or size limit. The compression can + be disabled by selecting the **Track** mode. + +## Available settings + +| Setting | Default | Description | +|-------------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Limiting mode | Compress | This mode compresses the jobs at the specified threshold and rejects them if they exceed the specified limit after compression. | +| Sidekiq job compression threshold (bytes) | 100 000 (100 KB) | When the size of arguments exceeds this threshold, they are compressed before being stored in Redis. | +| Sidekiq job size limit (bytes) | 0 | The jobs exceeding this size after compression are rejected. This avoids excessive memory usage in Redis leading to instability. Setting it to 0 prevents rejecting jobs. | + +After changing these values, [restart Sidekiq](../restart_gitlab.md). diff --git a/doc/administration/settings/sign_in_restrictions.md b/doc/administration/settings/sign_in_restrictions.md new file mode 100644 index 00000000000..393c7dd6aeb --- /dev/null +++ b/doc/administration/settings/sign_in_restrictions.md @@ -0,0 +1,203 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Sign-in restrictions **(FREE SELF)** + +You can use **Sign-in restrictions** to customize authentication restrictions for web interfaces as well as Git over HTTP(S). + +## Settings + +To access sign-in restriction settings: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Sign-in restrictions** section. + +## Password authentication enabled + +You can restrict the password authentication for web interface and Git over HTTP(S): + +- **Web interface**: When this feature is disabled, the **Standard** sign-in tab + is removed and an [external authentication provider](../auth/index.md) + must be used. +- **Git over HTTP(S)**: When this feature is disabled, a [Personal Access Token](../../user/profile/personal_access_tokens.md) + or LDAP password must be used to authenticate. + +In the event of an external authentication provider outage, use the [GitLab Rails console](../operations/rails_console.md) to [re-enable the standard web sign-in form](#re-enable-standard-web-sign-in-form-in-rails-console). This configuration can also be changed over the [Application settings REST API](../../api/settings.md#change-application-settings) while authenticating with an administrator account's personal access token. + +## Admin Mode + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2158) in GitLab 13.10. + +If you're an administrator, you might want to work in GitLab without administrator access. +You could either create a separate user account that does not have +administrator access or use Admin Mode. + +With Admin Mode, your account does not have administrator access by default. +You can continue to access groups and projects you're a member of. However, for administrative tasks, +you must authenticate (except for [certain features](#limitations-of-admin-mode)). + +When Admin Mode is enabled, it applies to all administrators on the instance. + +When Admin Mode is enabled for an instance, administrators: + +- Are allowed to access group and projects for which they are members. +- Cannot access the **Admin Area**. + +### Enable Admin Mode for your instance + +Administrators can enable Admin Mode though the API, the Rails console, or the UI. + +#### Use the API to enable Admin Mode + +Make the following request to your instance endpoint: + +```shell +curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab.example.com>/api/v4/application/settings?admin_mode=true" +``` + +Replace `<gitlab.example.com>` with your instance URL. + +For more information, see the [list of settings that can be accessed through API calls](../../api/settings.md). + +#### Use the Rails console to enable Admin Mode + +Open the [Rails console](../operations/rails_console.md) and run the following: + +```ruby +::Gitlab::CurrentSettings.update!(admin_mode: true) +``` + +#### Use the UI to enable Admin Mode + +To enable Admin Mode through the UI: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-in restrictions**. +1. In the **Admin Mode** section, select the **Require additional authentication for administrative tasks** checkbox. + +### Turn on Admin Mode for your session + +To turn on Admin Mode for your current session and access potentially dangerous resources: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Enter Admin Mode**. +1. Try to access any part of the UI with `/admin` in the URL (which requires administrator access). + +When Admin Mode status is disabled or turned off, administrators cannot access resources unless +they've been explicitly granted access. For example, administrators get a `404` error +if they try to open a private group or project, unless they are members of that group or project. + +2FA should be enabled for administrators. 2FA, OmniAuth providers, and LDAP +authentication are supported by Admin Mode. Admin Mode status is stored in the current user session and remains active until either: + +- It is explicitly disabled. +- It is disabled automatically after six hours. + +### Turn off Admin Mode for your session + +To turn off Admin Mode for your current session: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Leave Admin Mode**. + +### Limitations of Admin Mode + +Admin Mode times out after six hours, and you cannot change this timeout limit. + +The following access methods are **not** protected by Admin Mode: + +- Git client access (SSH using public keys or HTTPS using Personal Access Tokens). + +In other words, administrators who are otherwise limited by Admin Mode can still use +Git clients without additional authentication steps. + +To use the GitLab REST- or GraphQL API, administrators must [create a personal access token](../../user/profile/personal_access_tokens.md#create-a-personal-access-token) with the [`admin_mode` scope](../../user/profile/personal_access_tokens.md#personal-access-token-scopes). + +If an administrator with a personal access token with the `admin_mode` scope loses their administrator access, that user cannot access the API as an administrator even though they still have the token with the `admin_mode` scope. + +We may address these limitations in the future. For more information see the following epic: +[Admin Mode for GitLab Administrators](https://gitlab.com/groups/gitlab-org/-/epics/2158). + +Also, when GitLab Geo is enabled, you can't view the replication status of projects and designs while +on a secondary node. A fix is proposed when projects ([issue 367926](https://gitlab.com/gitlab-org/gitlab/-/issues/367926)) and designs ([issue 355660](https://gitlab.com/gitlab-org/gitlab/-/issues/355660)) move to the new Geo framework. + +### Troubleshooting Admin Mode + +If necessary, you can disable **Admin Mode** as an administrator by using one of these two methods: + +- **API**: + + ```shell + curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab-url>/api/v4/application/settings?admin_mode=false" + ``` + +- [**Rails console**](../operations/rails_console.md#starting-a-rails-console-session): + + ```ruby + ::Gitlab::CurrentSettings.update!(admin_mode: false) + ``` + +## Two-factor authentication + +When this feature is enabled, all users must use the [two-factor authentication](../../user/profile/account/two_factor_authentication.md). + +After the two-factor authentication is configured as mandatory, users are allowed +to skip forced configuration of two-factor authentication for the configurable grace +period in hours. + + + +## Email notification for unknown sign-ins + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218457) in GitLab 13.2. + +When enabled, GitLab notifies users of sign-ins from unknown IP addresses or devices. For more information, +see [Email notification for unknown sign-ins](../../user/profile/notifications.md#notifications-for-unknown-sign-ins). + + + +## Sign-in information + +All users that are not logged in are redirected to the page represented by the configured +**Home page URL** if value is not empty. + +All users are redirected to the page represented by the configured **Sign-out page URL** +after sign out if value is not empty. + +In the **Sign-in restrictions** section, scroll to the **Sign-in text** field. You can add a +custom message for your users in Markdown format. + +For example, if you include the following information in the noted text box: + +```markdown +# Custom sign-in text + +To access this text box: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Sign-in restrictions** section. +``` + +Your users see the **Custom sign-in text** when they navigate to the sign-in screen for your +GitLab instance. + +## Troubleshooting + +### Re-enable standard web sign-in form in rails console + +Re-enable the standard username and password-based sign-in form if it was disabled as a [Sign-in restriction](#password-authentication-enabled). + +You can use this method through the [rails console](../operations/rails_console.md#starting-a-rails-console-session) when a configured external authentication provider (through SSO or an LDAP configuration) is facing an outage and direct sign-in access to GitLab is required. + +```ruby +Gitlab::CurrentSettings.update!(password_authentication_enabled_for_web: true) +``` diff --git a/doc/administration/settings/sign_up_restrictions.md b/doc/administration/settings/sign_up_restrictions.md new file mode 100644 index 00000000000..f213794eb75 --- /dev/null +++ b/doc/administration/settings/sign_up_restrictions.md @@ -0,0 +1,206 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Sign-up restrictions **(FREE SELF)** + +You can enforce the following restrictions on sign ups: + +- Disable new sign ups. +- Require administrator approval for new sign ups. +- Require user email confirmation. +- Allow or deny sign ups using specific email domains. + +## Disable new sign ups + +By default, any user visiting your GitLab domain can sign up for an account. For customers running +public-facing GitLab instances, we **highly** recommend that you consider disabling new sign ups if +you do not expect public users to sign up for an account. + +To disable sign ups: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Clear the **Sign-up enabled** checkbox, then select **Save changes**. + +## Require administrator approval for new sign ups + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4491) in GitLab 13.5. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/267568) in GitLab 13.6. + +When this setting is enabled, any user visiting your GitLab domain and signing up for a new account using the registration form +must be explicitly [approved](../../administration/moderate_users.md#approve-or-reject-a-user-sign-up) by an +administrator before they can start using their account. In GitLab 13.6 and later, this setting is +enabled by default for new GitLab instances. It is only applicable if sign ups are enabled. + +To require administrator approval for new sign ups: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Select the **Require admin approval for new sign-ups** checkbox, then select **Save changes**. + +In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/273258), if an administrator disables this setting, the users in pending approval state are +automatically approved in a background job. + +NOTE: +This setting doesn't apply to LDAP or OmniAuth users. To enforce approvals for new users +signing up using OmniAuth or LDAP, set `block_auto_created_users` to `true` in the +[OmniAuth configuration](../../integration/omniauth.md#configure-common-settings) or +[LDAP configuration](../auth/ldap/index.md#basic-configuration-settings). +A [user cap](#user-cap) can also be used to enforce approvals for new users. + +## Confirm user email + +> - Soft email confirmation [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47003) in GitLab 12.2 [with a flag](../../operations/feature_flags.md) named `soft_email_confirmation`. +> - Soft email confirmation [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107302/diffs) from a feature flag to an application setting in GitLab 15.9. + +You can send confirmation emails during sign up and require that users confirm +their email address before they are allowed to sign in. + +For example, to enforce confirmation of the email address used for new sign ups: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Under **Email confirmation settings**, select **Hard**. + +The following settings are available: + +- **Hard** - Send a confirmation email during sign up. New users must confirm their email address before they can log in. +- **Soft** - Send a confirmation email during sign up. New users can log in immediately, but must confirm their email in three days. Unconfirmed accounts are deleted. +- **Off** - New users can sign up without confirming their email address. + +## User cap + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4315) in GitLab 13.7. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292600) in GitLab 13.9. + +When the number of billable users reaches the user cap, any user who is added or requests access must be +[approved](../../administration/moderate_users.md#approve-or-reject-a-user-sign-up) by an administrator before they can start using +their account. + +If an administrator [increases](#set-the-user-cap-number) or [removes](#remove-the-user-cap) the +user cap, the users in pending approval state are automatically approved in a background job. + +NOTE: +The amount of billable users [is updated once a day](../../subscriptions/self_managed/index.md#billable-users). +This means the user cap might apply only retrospectively after the cap has already been exceeded. +To ensure the cap is enabled immediately, set it to a low value below the current number of +billable users, for example: `1`. + +On instances that use LDAP or OmniAuth, enabling and disabling +[administrator approval for new sign ups](#require-administrator-approval-for-new-sign-ups) +involves changing the Rails configuration, and may require downtime. +User cap can be used instead. As noted above, set the cap to value that ensures it is enforced immediately. + +### Set the user cap number + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Enter a number in **User cap**. +1. Select **Save changes**. + +New user sign ups are subject to the user cap restriction. + +## Remove the user cap + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Remove the number from **User cap**. +1. Select **Save changes**. + +New users sign ups are not subject to the user cap restriction. Users in pending approval state are +automatically approved in a background job. + +## Minimum password length limit + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) in GitLab 12.6 + +You can [change](../../security/password_length_limits.md#modify-minimum-password-length) +the minimum number of characters a user must have in their password using the GitLab UI. + +### Password complexity requirements **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354965) in GitLab 15.2. + +By default, the only requirement for user passwords is [minimum password length](#minimum-password-length-limit). +You can add additional complexity requirements. Changes to password complexity requirements apply to new passwords: + +- For new users that sign up. +- For existing users that reset their password. + +Existing passwords are unaffected. To change password complexity requirements: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. Under **Minimum password length (number of characters)**, select additional password complexity requirements. You can require numbers, uppercase letters, lowercase letters, + and symbols. +1. Select **Save changes**. + +## Allow or deny sign ups using specific email domains + +You can specify an inclusive or exclusive list of email domains which can be used for user sign up. + +These restrictions are only applied during sign up from an external user. An administrator can add a +user through the administrator panel with a disallowed domain. The users can also change their +email addresses to disallowed domains after sign up. + +### Allowlist email domains + +You can restrict users only to sign up using email addresses matching the given +domains list. + +### Denylist email domains + +You can block users from signing up when using an email addresses of specific domains. This can +reduce the risk of malicious users creating spam accounts with disposable email addresses. + +### Create email domain allowlist or denylist + +To create an email domain allowlist or denylist: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Sign-up restrictions**. +1. For the allowlist, you must enter the list manually. For the denylist, you can enter the list + manually or upload a `.txt` file that contains list entries. + + Both the allowlist and denylist accept wildcards. For example, you can use +`*.company.com` to accept every `company.com` subdomain, or `*.io` to block all +domains ending in `.io`. Domains must be separated by a whitespace, +semicolon, comma, or a new line. + +  + +## Set up LDAP user filter + +You can limit GitLab access to a subset of the LDAP users on your LDAP server. + +See the [documentation on setting up an LDAP user filter](../auth/ldap/index.md#set-up-ldap-user-filter) for more information. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/terms.md b/doc/administration/settings/terms.md new file mode 100644 index 00000000000..4b4972acc8e --- /dev/null +++ b/doc/administration/settings/terms.md @@ -0,0 +1,49 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Terms of Service and Privacy Policy **(FREE SELF)** + +An administrator can enforce acceptance of a terms of service and privacy policy. +When this option is enabled, new and existing users must accept the terms. + +When enabled, you can view the Terms of Service at the `-/users/terms` page on the instance, +for example `https://gitlab.example.com/-/users/terms`. + +## Enforce a Terms of Service and Privacy Policy + +To enforce acceptance of a Terms of Service and Privacy Policy: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Terms of Service and Privacy Policy** section. +1. Check the **All users must accept the Terms of Service and Privacy Policy to access GitLab** checkbox. +1. Input the text of the **Terms of Service and Privacy Policy**. You can use [Markdown](../../user/markdown.md) + in this text box. +1. Select **Save changes**. + +For each update to the terms, a new version is stored. When a user accepts or declines the terms, +GitLab records which version they accepted or declined. + +Existing users must accept the terms on their next GitLab interaction. +If an authenticated user declines the terms, they are signed out. + +When enabled, it adds a mandatory checkbox to the sign up page for new users: + + + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/terraform_limits.md b/doc/administration/settings/terraform_limits.md new file mode 100644 index 00000000000..90ab1c25522 --- /dev/null +++ b/doc/administration/settings/terraform_limits.md @@ -0,0 +1,28 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Terraform limits **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352951) in GitLab 15.7. + +You can limit the total storage of [Terraform state files](../terraform_state.md). +The limit applies to each individual +state file version, and is checked whenever a new version is created. + +To add a storage limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Preferences**. +1. Expand **Terraform limits**. +1. Adjust the size limit. + +## Available settings + +| Setting | Default | Description | +|------------------------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------| +| Terraform state size limit (bytes) | 0 | Terraform state files that exceed this size are not saved, and associated Terraform operations are rejected. Set to 0 to allow files of unlimited size. | diff --git a/doc/administration/settings/third_party_offers.md b/doc/administration/settings/third_party_offers.md new file mode 100644 index 00000000000..39e2275f411 --- /dev/null +++ b/doc/administration/settings/third_party_offers.md @@ -0,0 +1,38 @@ +--- +stage: Data Stores +group: Tenant Scale +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# Customer experience improvement and third-party offers **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20379) in GitLab 11.1. + +Within GitLab, we inform users of available third-party offers they might find valuable in order +to enhance the development of their projects. An example is the Google Cloud Platform free credit +for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). + +Furthermore, we use content to improve customer experience. An example are the personalization +questions when creating a group. + +To toggle the display of customer experience improvement content and third-party offers: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand **Customer experience improvement and third-party offers**. +1. Select **Do not display content for customer experience improvement and offers from third parties**. +1. Select **Save changes**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/usage_statistics.md b/doc/administration/settings/usage_statistics.md new file mode 100644 index 00000000000..ba15ee4e33e --- /dev/null +++ b/doc/administration/settings/usage_statistics.md @@ -0,0 +1,230 @@ +--- +stage: Analytics +group: Analytics Instrumentation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Usage statistics **(FREE SELF)** + +GitLab Inc. periodically collects information about your instance in order +to perform various actions. + +All usage statistics are [opt-out](#enable-or-disable-usage-statistics). + +## Service Ping + +Service Ping is a process that collects and sends a weekly payload to GitLab Inc. +For more information, see the [Service Ping guide](../../development/internal_analytics/service_ping/index.md). When Service Ping is enabled, GitLab gathers data from other instances and enables certain [instance-level analytics features](../../user/analytics/index.md) +that are dependent on Service Ping. + +### Why enable Service Ping? + +The main purpose of Service Ping is to build a better GitLab. We collect data about how GitLab is used +to understand feature or stage adoption and usage. This data gives an insight into how GitLab adds +value and helps our team understand the reasons why people use GitLab, and with this knowledge we're able to make better product decisions. + +There are several other benefits to enabling Service Ping: + +- Analyze the users' activities over time of your GitLab installation. +- A [DevOps Score](../analytics/dev_ops_reports.md) to give you an overview of your entire instance's adoption of concurrent DevOps from planning to monitoring. +- More proactive support (assuming that our [Customer Success Managers (CSMs)](https://handbook.gitlab.com/job-families/sales/customer-success-management/) and support organization used the data to deliver more value). +- Insight and advice into how to get the most value out of your investment in GitLab. +- Reports that show how you compare against other similar organizations (anonymized), with specific advice and recommendations on how to improve your DevOps processes. +- Participation in our [Registration Features Program](#registration-features-program) to receive free paid features. + +## Registration Features Program + +> Introduced in GitLab 14.1. + +In GitLab versions 14.1 and later, GitLab Free customers with a self-managed instance running +GitLab Enterprise Edition can receive paid features by registering with GitLab and sending us +activity data through Service Ping. Features introduced here do not remove the feature from its paid +tier. Users can continue to access the features in a paid tier without sharing usage data. + +### Features available in 14.1 and later + +- [Email from GitLab](../email_from_gitlab.md). + +### Features available in 14.4 and later + +- [Repository size limit](../../administration/settings/account_and_limit_settings.md#repository-size-limit). +- [Group access restriction by IP address](../../user/group/access_and_permissions.md#restrict-group-access-by-ip-address). + +### Features available in 16.0 and later + +- [View description change history](../../user/discussions/index.md#view-description-change-history). +- [Maintenance mode](../maintenance_mode/index.md). +- [Configurable issue boards](../../user/project/issue_board.md#configurable-issue-boards). +- [Coverage-guided fuzz testing](../../user/application_security/coverage_fuzzing/index.md). +- [Password complexity requirements](../../administration/settings/sign_up_restrictions.md#password-complexity-requirements). + +NOTE: +Registration is not yet required for participation, but may be added in a future milestone. + +### Enable registration features + +1. Sign in as a user with administrator access. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Metrics and profiling**. +1. Expand the **Usage statistics** section. +1. If not enabled, select the **Enable Service Ping** checkbox. +1. Select the **Enable Registration Features** checkbox. +1. Select **Save changes**. + +## Version check + +If enabled, version check informs you if a new version is available and the +importance of it through a status. The status displays on the help pages (`/help`) +for all authenticated users, and on the Admin Area pages. The statuses are: + +- Green: You are running the latest version of GitLab. +- Orange: An updated version of GitLab is available. +- Red: The version of GitLab you are running is vulnerable. You should install + the latest version with security fixes as soon as possible. + + + +GitLab Inc. collects your instance's version and hostname (through the HTTP +referer) as part of the version check. No other information is collected. + +This information is used, among other things, to identify to which versions +patches must be backported, making sure active GitLab instances remain +secure. + +If you [disable version check](#enable-or-disable-usage-statistics), this information isn't collected. + +### Request flow example + +The following example shows a basic request/response flow between a +self-managed GitLab instance and the GitLab Version Application: + +```mermaid +sequenceDiagram + participant GitLab instance + participant Version Application + GitLab instance->>Version Application: Is there a version update? + loop Version Check + Version Application->>Version Application: Record version info + end + Version Application->>GitLab instance: Response (PNG/SVG) +``` + +## Configure your network + +To send usage statistics to GitLab Inc., you must allow network traffic from your +GitLab instance to the host `version.gitlab.com` on port `443`. + +If your GitLab instance is behind a proxy, set the appropriate +[proxy configuration variables](https://docs.gitlab.com/omnibus/settings/environment-variables.html). + +## Enable or disable usage statistics + +To enable or disable Service Ping and version check: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Metrics and profiling**. +1. Expand **Usage statistics**. +1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. +1. Select **Save changes**. + +NOTE: +Service Ping settings only control whether the data is being shared with GitLab, or used only internally. +Even if you disable Service Ping, the `gitlab_service_ping_worker` background job still periodically generates a Service Ping payload for your instance. +The payload is available in the [Service Usage data](#manually-upload-service-ping-payload) admin section. + +## Disable usage statistics with the configuration file + +NOTE: +The method to disable Service Ping in the GitLab configuration file does not work in +GitLab versions 9.3 to 13.12.3. For more information about how to disable it, see [troubleshooting](../../development/internal_analytics/service_ping/troubleshooting.md#cannot-disable-service-ping-with-the-configuration-file). + +To disable Service Ping and prevent it from being configured in the future through +the Admin Area: + +**For installations using the Linux package:** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['usage_ping_enabled'] = false + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +**For installations from source:** + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: &base + # ... + gitlab: + # ... + usage_ping_enabled: false + ``` + +1. Restart GitLab: + + ```shell + sudo service gitlab restart + ``` + +## View the Service Ping payload + +You can view the exact JSON payload sent to GitLab Inc. in the Admin Area. To view the payload: + +1. Sign in as a user with administrator access. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Metrics and profiling**. +1. Expand the **Usage statistics** section. +1. Select **Preview payload**. + +For an example payload, see [Example Service Ping payload](../../development/internal_analytics/service_ping/index.md#example-service-ping-payload). + +## Manually upload Service Ping payload + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7388) in GitLab 14.8 with a flag named `admin_application_settings_service_usage_data_center`. Disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83265) in GitLab 14.10. + +You can upload the Service Ping payload to GitLab even if your instance doesn't have internet access, +or if the Service Ping [cron job](../../development/internal_analytics/service_ping/index.md#how-service-ping-works) is not enabled. + +To upload the payload manually: + +1. Sign in as a user with administrator access. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Service** usage data. +1. Select **Download payload**. +1. Save the JSON file. +1. Visit [Service usage data center](https://version.gitlab.com/usage_data/new). +1. Select **Choose file** and choose the file from p5. +1. Select **Upload**. + +The uploaded file is encrypted and sent using secure HTTPS protocol. HTTPS creates a secure +communication channel between web browser and the server, and protects transmitted data against man-in-the-middle attacks. + +If there are problems with the manual upload: + +1. Open a confidential issue in the [security fork of version app project](https://gitlab.com/gitlab-org/security/version.gitlab.com). +1. Attach the JSON payload if possible. +1. Tag `@gitlab-org/analytics-section/analytics-instrumentation` who will triage the issue. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/user_and_ip_rate_limits.md b/doc/administration/settings/user_and_ip_rate_limits.md new file mode 100644 index 00000000000..44bd08a8824 --- /dev/null +++ b/doc/administration/settings/user_and_ip_rate_limits.md @@ -0,0 +1,240 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: reference +--- + +# User and IP rate limits **(FREE SELF)** + +Rate limiting is a common technique used to improve the security and durability +of a web application. For more details, see +[Rate limits](../../security/rate_limits.md). + +The following limits are disabled by default: + +- [Unauthenticated API requests (per IP)](#enable-unauthenticated-api-request-rate-limit). +- [Unauthenticated web requests (per IP)](#enable-unauthenticated-web-request-rate-limit). +- [Authenticated API requests (per user)](#enable-authenticated-api-request-rate-limit). +- [Authenticated web requests (per user)](#enable-authenticated-web-request-rate-limit). + +NOTE: +By default, all Git operations are first tried unauthenticated. Because of this, HTTP Git operations +may trigger the rate limits configured for unauthenticated requests. + +NOTE: +[In GitLab 14.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/344807), +the rate limits for API requests don't affect requests made by the frontend, as these are always +counted as web traffic. + +## Enable unauthenticated API request rate limit + +To enable the unauthenticated request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **User and IP rate limits**. +1. Select **Enable unauthenticated API request rate limit**. + + - Optional. Update the **Maximum unauthenticated API requests per rate limit period per IP** value. + Defaults to `3600`. + - Optional. Update the **Unauthenticated rate limit period in seconds** value. + Defaults to `3600`. + +## Enable unauthenticated web request rate limit + +To enable the unauthenticated request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **User and IP rate limits**. +1. Select **Enable unauthenticated web request rate limit**. + + - Optional. Update the **Maximum unauthenticated web requests per rate limit period per IP** value. + Defaults to `3600`. + - Optional. Update the **Unauthenticated rate limit period in seconds** value. + Defaults to `3600`. + +## Enable authenticated API request rate limit + +To enable the authenticated API request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **User and IP rate limits**. +1. Select **Enable authenticated API request rate limit**. + + - Optional. Update the **Maximum authenticated API requests per rate limit period per user** value. + Defaults to `7200`. + - Optional. Update the **Authenticated API rate limit period in seconds** value. + Defaults to `3600`. + +## Enable authenticated web request rate limit + +To enable the unauthenticated request rate limit: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **User and IP rate limits**. +1. Select **Enable authenticated web request rate limit**. + + - Optional. Update the **Maximum authenticated web requests per rate limit period per user** value. + Defaults to `7200`. + - Optional. Update the **Authenticated web rate limit period in seconds** value. + Defaults to `3600`. + +## Use a custom rate limit response + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50693) in GitLab 13.8. + +A request that exceeds a rate limit returns a `429` response code and a +plain-text body, which by default is `Retry later`. + +To use a custom response: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > Network**. +1. Expand **User and IP rate limits**. +1. In the **Plain-text response to send to clients that hit a rate limit** text box, + add the plain-text response message. + +## Response headers + +> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/731) in GitLab 13.8, the `RateLimit` headers. `Retry-After` was introduced in an earlier version. + +When a client exceeds the associated rate limit, the following requests are +blocked. The server may respond with rate-limiting information allowing the +requester to retry after a specific period of time. These information are +attached into the response headers. + +| Header | Example | Description | +|:----------------------|:--------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the Admin Area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | +| `RateLimit-Name` | `throttle_authenticated_web` | Name of the throttle blocking the requests. | +| `RateLimit-Observed` | `67` | Number of requests associated to the client in the time window. | +| `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Observed`. | +| `RateLimit-Reset` | `1609844400` | [Unix time](https://en.wikipedia.org/wiki/Unix_time)-formatted time when the request quota is reset. | +| `RateLimit-ResetTime` | `Tue, 05 Jan 2021 11:00:00 GMT` | [RFC2616](https://www.rfc-editor.org/rfc/rfc2616#section-3.3.1)-formatted date and time when the request quota is reset. | +| `Retry-After` | `30` | Remaining duration **in seconds** until the quota is reset. This is a [standard HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After). | + +## Use an HTTP header to bypass rate limiting + +> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/622) in GitLab 13.6. + +Depending on the needs of your organization, you may want to enable rate limiting +but have some requests bypass the rate limiter. + +You can do this by marking requests that should bypass the rate limiter with a custom +header. You must do this somewhere in a load balancer or reverse proxy in front of +GitLab. For example: + +1. Pick a name for your bypass header. For example, `Gitlab-Bypass-Rate-Limiting`. +1. Configure your load balancer to set `Gitlab-Bypass-Rate-Limiting: 1` on requests + that should bypass GitLab rate limiting. +1. Configure your load balancer to either: + - Erase `Gitlab-Bypass-Rate-Limiting`. + - Set `Gitlab-Bypass-Rate-Limiting` to a value other than `1` on all requests that + should be affected by rate limiting. +1. Set the environment variable `GITLAB_THROTTLE_BYPASS_HEADER`. + - For [Linux package installations](https://docs.gitlab.com/omnibus/settings/environment-variables.html), + set `'GITLAB_THROTTLE_BYPASS_HEADER' => 'Gitlab-Bypass-Rate-Limiting'` in `gitlab_rails['env']`. + - For source installations, set `export GITLAB_THROTTLE_BYPASS_HEADER=Gitlab-Bypass-Rate-Limiting` + in `/etc/default/gitlab`. + +It is important that your load balancer erases or overwrites the bypass +header on all incoming traffic. Otherwise, you must trust your +users to not set that header and bypass the GitLab rate limiter. + +The bypass works only if the header is set to `1`. + +Requests that bypassed the rate limiter because of the bypass header +are marked with `"throttle_safelist":"throttle_bypass_header"` in +[`production_json.log`](../logs/index.md#production_jsonlog). + +To disable the bypass mechanism, make sure the environment variable +`GITLAB_THROTTLE_BYPASS_HEADER` is unset or empty. + +## Allow specific users to bypass authenticated request rate limiting + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49127) in GitLab 13.7. + +Similarly to the bypass header described above, it is possible to allow +a certain set of users to bypass the rate limiter. This only applies +to authenticated requests: with unauthenticated requests, by definition +GitLab does not know who the user is. + +The allowlist is configured as a comma-separated list of user IDs in +the `GITLAB_THROTTLE_USER_ALLOWLIST` environment variable. If you want +users 1, 53 and 217 to bypass the authenticated request rate limiter, +the allowlist configuration would be `1,53,217`. + +- For [Linux package installations](https://docs.gitlab.com/omnibus/settings/environment-variables.html), + set `'GITLAB_THROTTLE_USER_ALLOWLIST' => '1,53,217'` in `gitlab_rails['env']`. +- For source installations, set `export GITLAB_THROTTLE_USER_ALLOWLIST=1,53,217` + in `/etc/default/gitlab`. + +Requests that bypassed the rate limiter because of the user allowlist +are marked with `"throttle_safelist":"throttle_user_allowlist"` in +[`production_json.log`](../logs/index.md#production_jsonlog). + +At application startup, the allowlist is logged in [`auth.log`](../logs/index.md#authlog). + +## Try out throttling settings before enforcing them + +> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/629) in GitLab 13.6. + +You can try out throttling settings by setting the `GITLAB_THROTTLE_DRY_RUN` environment variable to +a comma-separated list of throttle names. + +The possible names are: + +- `throttle_unauthenticated` + - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/335300) in GitLab 14.3. Use `throttle_unauthenticated_api` or `throttle_unauthenticated_web` instead. + `throttle_unauthenticated` is still supported and selects both of them. +- `throttle_unauthenticated_api` +- `throttle_unauthenticated_web` +- `throttle_authenticated_api` +- `throttle_authenticated_web` +- `throttle_unauthenticated_protected_paths` +- `throttle_authenticated_protected_paths_api` +- `throttle_authenticated_protected_paths_web` +- `throttle_unauthenticated_packages_api` +- `throttle_authenticated_packages_api` +- `throttle_authenticated_git_lfs` +- `throttle_unauthenticated_files_api` +- `throttle_authenticated_files_api` +- `throttle_unauthenticated_deprecated_api` +- `throttle_authenticated_deprecated_api` + +For example, to try out throttles for all authenticated requests to +non-protected paths can be done by setting +`GITLAB_THROTTLE_DRY_RUN='throttle_authenticated_web,throttle_authenticated_api'`. + +To enable dry run mode for all throttles, the variable can be set to `*`. + +Setting a throttle to dry run mode logs a message to the +[`auth.log`](../logs/index.md#authlog) when it would hit the limit, while letting the +request continue. The log message contains an `env` field set to `track`. The `matched` +field contains the name of throttle that was hit. + +It is important to set the environment variable **before** enabling +the rate limiting in the settings. The settings in the Admin Area +take effect immediately, while setting the environment variable +requires a restart of all the Puma processes. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/settings/visibility_and_access_controls.md b/doc/administration/settings/visibility_and_access_controls.md new file mode 100644 index 00000000000..5e1e35db244 --- /dev/null +++ b/doc/administration/settings/visibility_and_access_controls.md @@ -0,0 +1,363 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +type: reference +--- + +# Control access and visibility **(FREE SELF)** + +GitLab enables users with administrator access to enforce +specific controls on branches, projects, snippets, groups, and more. + +To access the visibility and access control options: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. + +## Define which roles can create projects + +Instance-level protections for project creation define which roles can +[add projects to a group](../../user/group/index.md#specify-who-can-add-projects-to-a-group) +on the instance. To alter which roles have permission to create projects: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. For **Default project creation protection**, select the desired roles: + - No one. + - Maintainers. + - Developers and Maintainers. +1. Select **Save changes**. + +## Restrict project deletion to administrators **(PREMIUM SELF)** + +> User interface [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. + +By default both administrators and anyone with the **Owner** role can delete a project. To restrict project deletion to only administrators: + +1. Sign in to GitLab as a user with administrator access. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Scroll to: + - (GitLab 15.1 and later) **Allowed to delete projects**, and select **Administrators**. + - (GitLab 15.0 and earlier) **Default project deletion protection** and select **Only admins can delete project**. +1. Select **Save changes**. + +## Deletion protection **(PREMIUM SELF)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255449) in GitLab 14.2 for groups created after August 12, 2021. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) from default delayed project deletion in GitLab 15.1. +> - [Enabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89466) in GitLab 15.1. +> - [Disabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95495) in GitLab 15.3. +> - [Removed option to delete immediately](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. +> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. + +Instance-level protection against accidental deletion of groups and projects. + +### Retention period + +> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. + +Groups and projects remain restorable within a defined retention period. By default this is 7 days but it can be changed. +Setting the retention period to `0` means that groups and project are removed immediately and cannot be restored. + +In GitLab 15.1 and later, the retention period must be between `1` and `90`. If the retention period was `0` before the 15.1 update, +then it gets automatically changed to `1` while also disabling deletion protection the next time any application setting is changed. + +### Delayed project deletion + +> - User interface [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. +> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. + +To configure delayed project deletion: + +1. Sign in to GitLab as a user with administrator access. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Scroll to: + - (In GitLab 15.11 and later with `always_perform_delayed_deletion` feature flag enabled, or GitLab 16.0 and later) **Deletion protection** and set the retention period to a value between `1` and `90`. + - (GitLab 15.1 and later) **Deletion protection** and select keep deleted groups and projects, and select a retention period. + - (GitLab 15.0 and earlier) **Default delayed project protection** and select **Enable delayed project deletion by + default for newly-created groups.** Then set a retention period in **Default deletion delay**. +1. Select **Save changes**. + +Deletion protection is not available for projects only (without being also being enabled for groups). + +In GitLab 15.1, and later this setting is enforced on groups when disabled and it cannot be overridden. + +### Delayed group deletion + +> - User interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. +> - [Changed to default behavior](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) on the Premium and Ultimate tier in GitLab 16.0. + +Groups remain restorable if the retention period is `1` or more days. + +In GitLab 15.1 and later, delayed group deletion can be enabled by setting **Deletion projection** to **Keep deleted**. +In GitLab 15.11 and later with the `always_perform_delayed_deletion` feature flag enabled, or in GitLab 16.0 and later: + +- The **Keep deleted** option is removed. +- Delayed group deletion is the default. + +### Override defaults and delete immediately + +Alternatively, projects that are marked for removal can be deleted immediately. To do so: + +1. [Restore the project](../../user/project/settings/index.md#restore-a-project). +1. Delete the project as described in the + [Administering Projects page](../admin_area.md#administering-projects). + +## Configure project visibility defaults + +To set the default [visibility levels for new projects](../../user/public_access.md): + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Select the desired default project visibility: + - **Private** - Project access must be granted explicitly to each user. If this + project is part of a group, access is granted to members of the group. + - **Internal** - The project can be accessed by any authenticated user except external users. + - **Public** - The project can be accessed without any authentication. +1. Select **Save changes**. + +## Configure snippet visibility defaults + +To set the default visibility levels for new [snippets](../../user/snippets.md): + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Select the desired default snippet visibility. +1. Select **Save changes**. + +For more details on snippet visibility, read +[Project visibility](../../user/public_access.md). + +## Configure group visibility defaults + +To set the default visibility levels for new groups: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Select the desired default group visibility: + - **Private** - The group and its projects can only be viewed by members. + - **Internal** - The group and any internal projects can be viewed by any authenticated user except external users. + - **Public** - The group and any public projects can be viewed without any authentication. +1. Select **Save changes**. + +For more details on group visibility, see +[Group visibility](../../user/group/index.md#group-visibility). + +## Restrict visibility levels + +When restricting visibility levels, consider how these restrictions interact +with permissions for subgroups and projects that inherit their visibility from +the item you're changing. + +To restrict visibility levels for groups, projects, snippets, and selected pages: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. In the **Restricted visibility levels** section, select the desired visibility levels to restrict. + - If you restrict the **Public** level: + - Only administrators are able to create public groups, projects, and snippets. + - User profiles are only visible to authenticated users through the Web interface. + - User attributes through the GraphQL API are: + - Not visible in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88020). + - Only visible to authenticated users between [GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33195) and GitLab 15.0. + - If you restrict the **Internal** level: + - Only administrators are able to create internal groups, projects, and snippets. + - If you restrict the **Private** level: + - Only administrators are able to create private groups, projects, and snippets. +1. Select **Save changes**. + +For more details on project visibility, see +[Project visibility](../../user/public_access.md). + +## Configure allowed import sources + +Before you can import projects from other systems, you must enable the +[import source](../../user/gitlab_com/index.md#default-import-sources) for that system. + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Select each of **Import sources** to allow. +1. Select **Save changes**. + +## Enable project export + +To enable the export of +[projects and their data](../../user/project/settings/import_export.md#export-a-project-and-its-data): + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Scroll to **Project export**. +1. Select the **Enabled** checkbox. +1. Select **Save changes**. + +## Enable migration of groups and projects by direct transfer + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383268) in GitLab 15.8. + +You can enable migration of groups by direct transfer using the UI. + +To enable migration of groups by direct transfer: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Scroll to **Allow migrating GitLab groups and projects by direct transfer**. +1. Select the **Enabled** checkbox. +1. Select **Save changes**. + +The same setting +[is available](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls) in the API as the +`bulk_import_enabled` attribute. + +## Configure enabled Git access protocols + +With GitLab access restrictions, you can select the protocols users can use to +communicate with GitLab. Disabling an access protocol does not block port access to the +server itself. The ports used for the protocol, SSH or HTTP(S), are still accessible. +The GitLab restrictions apply at the application level. + +To specify the enabled Git access protocols: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Select the desired Git access protocols: + - Both SSH and HTTP(S) + - Only SSH + - Only HTTP(S) +1. Select **Save changes**. + +When both SSH and HTTP(S) are enabled, users can choose either protocol. +If only one protocol is enabled: + +- The project page shows only the allowed protocol's URL, with no option to + change it. +- GitLab shows a tooltip when you hover over the protocol for the URL, if user action + (such as adding a SSH key or setting a password) is required: + +  + +GitLab only allows Git actions for the protocols you select. + +WARNING: +GitLab versions [10.7 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18021), +allow the HTTP(S) protocol for Git clone or fetch requests done by GitLab Runner +from CI/CD jobs, even if you select **Only SSH**. + +## Customize Git clone URL for HTTP(S) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18422) in GitLab 12.4. + +You can customize project Git clone URLs for HTTP(S), which affects the clone +panel: + +For example, if: + +- Your GitLab instance is at `https://example.com`, then project clone URLs are like + `https://example.com/foo/bar.git`. +- You want clone URLs that look like `https://git.example.com/gitlab/foo/bar.git` instead, + you can set this setting to `https://git.example.com/gitlab/`. + + + +To specify a custom Git clone URL for HTTP(S): + +1. Enter a root URL for **Custom Git clone URL for HTTP(S)**. +1. Select **Save changes**. + +NOTE: +SSH clone URLs can be customized in `gitlab.rb` by setting `gitlab_rails['gitlab_ssh_host']` and +other related settings. + +## Configure defaults for RSA, DSA, ECDSA, ED25519, ECDSA_SK, ED25519_SK SSH keys + +These options specify the permitted types and lengths for SSH keys. + +To specify a restriction for each key type: + +1. Select the desired option from the dropdown list. +1. Select **Save changes**. + +For more details, see [SSH key restrictions](../../security/ssh_keys_restrictions.md). + +## Enable project mirroring + +This option is enabled by default. By disabling it, both +[pull mirroring](../../user/project/repository/mirror/pull.md) and [push mirroring](../../user/project/repository/mirror/push.md) no longer +work in every repository. They can only be re-enabled by an administrator user on a per-project basis. + + + +## Configure globally-allowed IP address ranges + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87579) in GitLab 15.1 [with a flag](../feature_flags.md) named `group_ip_restrictions_allow_global`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) in GitLab 15.4. [Feature flag `group_ip_restrictions_allow_global`](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) removed. + +Administrators can set IP address ranges to be combined with [group-level IP restrictions](../../user/group/access_and_permissions.md#restrict-group-access-by-ip-address). +Use globally-allowed IP addresses to allow aspects of the GitLab installation to work even when group-level IP address +restrictions are set. + +For example, if the GitLab Pages daemon runs on the `10.0.0.0/24` range, you can specify that range as globally-allowed. +This means GitLab Pages can still fetch artifacts from pipelines even if group-level IP address restrictions don't +include the `10.0.0.0/24` range. + +To add a IP address range to the group-level allowlist: + +1. Sign in to GitLab as a user with Administrator access level. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. In **Globally-allowed IP ranges**, provide a list of IP address ranges. This list: + - Has no limit on the number of IP address ranges. + - Has a size limit of 1 GB. + - Applies to both SSH or HTTP authorized IP address ranges. You cannot split + this list by type of authorization. +1. Select **Save changes**. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, for example `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/administration/sidekiq/extra_sidekiq_processes.md b/doc/administration/sidekiq/extra_sidekiq_processes.md index 31e713bbf06..3a522c7171d 100644 --- a/doc/administration/sidekiq/extra_sidekiq_processes.md +++ b/doc/administration/sidekiq/extra_sidekiq_processes.md @@ -105,8 +105,6 @@ concurrency is set to: When `min_concurrency` is greater than `max_concurrency`, it is treated as being equal to `max_concurrency`. -You can find example values used by GitLab.com by searching for `concurrency:` -in [the Helm charts](https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com/-/blob/master/releases/gitlab/values/gprd.yaml.gotmpl). The values vary according to the work each specific deployment of Sidekiq does. Any other specialized deployments with processes dedicated to specific queues should have the concurrency tuned according to: diff --git a/doc/administration/sidekiq/index.md b/doc/administration/sidekiq/index.md index bb517e21fd0..c3e1182cb05 100644 --- a/doc/administration/sidekiq/index.md +++ b/doc/administration/sidekiq/index.md @@ -383,7 +383,7 @@ blocking all jobs on that worker from proceeding. If Rugged calls performed by S background task processing. By default, Rugged is used when Git repository data is stored on local storage or on an NFS mount. -[Using Rugged is recommended when using NFS](../nfs.md#improving-nfs-performance-with-gitlab), but if +Using Rugged is recommended when using NFS, but if you are using local storage, disabling Rugged can improve Sidekiq performance: ```shell diff --git a/doc/administration/sidekiq/sidekiq_troubleshooting.md b/doc/administration/sidekiq/sidekiq_troubleshooting.md index 315714cd00b..6802bb68b31 100644 --- a/doc/administration/sidekiq/sidekiq_troubleshooting.md +++ b/doc/administration/sidekiq/sidekiq_troubleshooting.md @@ -529,6 +529,32 @@ The list of available jobs can be found in the [workers](https://gitlab.com/gitl For more information about Sidekiq jobs, see the [Sidekiq-cron](https://github.com/sidekiq-cron/sidekiq-cron#work-with-job) documentation. +## Clearing a Sidekiq job deduplication idempotency key + +Occasionally, jobs that are expected to run (for example, cron jobs) are observed to not run at all. When checking the logs, there might be instances where jobs are seen to not run with a `"job_status": "deduplicated"`. + +This can happen when a job failed and the idempotency key was not cleared properly. For example, [stopping Sidekiq kills any remaining jobs after 25 seconds](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/4918). + +[By default, the key expires after 6 hours](https://gitlab.com/gitlab-org/gitlab/-/blob/87c92f06eb92716a26679cd339f3787ae7edbdc3/lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb#L23), +but if you want to clear the idempotency key immediately, follow the following steps (the example provided is for `Geo::VerificationBatchWorker`): + +1. Find the worker class and `args` of the job in the Sidekiq logs: + + ```plaintext + { ... "class":"Geo::VerificationBatchWorker","args":["container_repository"] ... } + ``` + +1. Start a [Rails console session](../operations/rails_console.md#starting-a-rails-console-session). +1. Run the following snippet: + + ```ruby + worker_class = Geo::VerificationBatchWorker + args = ["container_repository"] + dj = Gitlab::SidekiqMiddleware::DuplicateJobs::DuplicateJob.new({ 'class' => worker_class.name, 'args' => args }, worker_class.queue) + dj.send(:idempotency_key) + dj.delete! + ``` + ## Omnibus GitLab 14.0 and later: remove the `sidekiq-cluster` service Omnibus GitLab instances that were configured to run `sidekiq-cluster` prior to GitLab 14.0 diff --git a/doc/administration/silent_mode/index.md b/doc/administration/silent_mode/index.md index 49c95d75768..0b04654beaa 100644 --- a/doc/administration/silent_mode/index.md +++ b/doc/administration/silent_mode/index.md @@ -71,6 +71,14 @@ Incoming emails still raise issues, but the users who sent the emails to [Servic Triggering webhook tests via the UI results in HTTP status 500 responses. +### Remote mirrors + +Updates on [remote mirrors](../../user/project/repository/mirror/index.md) (pushing to, and pulling from them) are suppressed. + +### Integrations + +Executable [integrations](../../user/project/integrations/index.md) are suppressed. + ### Outbound emails Outbound emails are suppressed. diff --git a/doc/administration/snippets/index.md b/doc/administration/snippets/index.md index 613d161a64c..9b485140070 100644 --- a/doc/administration/snippets/index.md +++ b/doc/administration/snippets/index.md @@ -25,7 +25,7 @@ content changes. ### Snippets size limit configuration -This setting is not available through the [Admin Area settings](../../user/admin_area/settings/index.md). +This setting is not available through the [Admin Area settings](../settings/index.md). To configure this setting, use either the Rails console or the [Application settings API](../../api/settings.md). diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md index 84f596dacf2..2b738f975ba 100644 --- a/doc/administration/terraform_state.md +++ b/doc/administration/terraform_state.md @@ -27,7 +27,7 @@ or because your instance doesn't use Terraform. When Terraform state administration is disabled: -- On the left sidebar, you cannot select **Infrastructure > Terraform states**. +- On the left sidebar, you cannot select **Operate > Terraform states**. - Any CI/CD jobs that access the Terraform state fail with this error: ```shell diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index fc319fad3e8..f164e8ccbad 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -50,7 +50,7 @@ This content has been moved to [Troubleshooting CI/CD](../../ci/troubleshooting. ## License -This content has been moved to [Activate GitLab EE with a license file or key](../../user/admin_area/license_file.md). +This content has been moved to [Activate GitLab EE with a license file or key](../../administration/license_file.md). ## Registry diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index 120d566a7e7..88e0d4e6c6b 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -152,7 +152,7 @@ without having to [reconfigure GitLab](../restart_gitlab.md#reconfigure-a-linux- which in this case would restart Puma and Sidekiq. For example, a backup may fail with the following errors in the output of the -[backup command](../../raketasks/backup_restore.md#back-up-gitlab) +[backup command](../../administration/backup_restore/index.md#back-up-gitlab) because the statement timeout was too short: ```plaintext diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index 99f19821916..d8f82f13875 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -19,7 +19,7 @@ This is the default configuration. To change the location where the uploads are stored locally, use the steps in this section based on your installation method: NOTE: -For historical reasons, uploads for the whole instance (for example the [favicon](../user/admin_area/appearance.md#favicon)) are stored in a base directory, +For historical reasons, uploads for the whole instance (for example the [favicon](../administration/appearance.md#favicon)) are stored in a base directory, which by default is `uploads/-/system`. Changing the base directory on an existing GitLab installation is strongly discouraged. diff --git a/doc/administration/user_cohorts.md b/doc/administration/user_cohorts.md new file mode 100644 index 00000000000..6f2798f437c --- /dev/null +++ b/doc/administration/user_cohorts.md @@ -0,0 +1,40 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Cohorts **(FREE SELF)** + +You can analyze your users' GitLab activities over time. + +How do you interpret the user cohorts table? Let's review an example with the +following user cohorts: + + + +For the cohort of March 2020, three users were added to this server and have +been active since this month. One month later (April 2020), two users are still +active. Five months later (August 2020), one user from this cohort is still +active, or 33% of the original cohort of three that joined in March. + +The **Inactive users** column shows the number of users who were added during +the month, but who never had any activity in the instance. + +How do we measure the activity of users? GitLab considers a user active if: + +- The user signs in. +- The user has Git activity (whether push or pull). +- The user visits pages related to dashboards, projects, issues, or merge + requests ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54947) in GitLab 11.8). +- The user uses the API. +- The user uses the GraphQL API. + +## View user cohorts + +To view user cohorts: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Overview > Users**. +1. Select the **Cohorts** tab. diff --git a/doc/administration/user_settings.md b/doc/administration/user_settings.md index c4b00d05f9d..f480f05f6a2 100644 --- a/doc/administration/user_settings.md +++ b/doc/administration/user_settings.md @@ -14,7 +14,7 @@ By default, new users can create top-level groups. To disable new users' ability to create top-level groups (does not affect existing users' setting), GitLab administrators can modify this setting: - In GitLab 15.5 and later, using either: - - The [GitLab UI](../user/admin_area/settings/account_and_limit_settings.md#prevent-new-users-from-creating-top-level-groups). + - The [GitLab UI](../administration/settings/account_and_limit_settings.md#prevent-new-users-from-creating-top-level-groups). - The [application setting API](../api/settings.md#change-application-settings). - In GitLab 15.4 and earlier, in a configuration file by following the steps in this section. @@ -44,7 +44,7 @@ For self-compiled installations: Administrators can: -- Use the Admin Area to [prevent an existing user from creating top-level groups](../user/admin_area/index.md#prevent-a-user-from-creating-groups). +- Use the Admin Area to [prevent an existing user from creating top-level groups](../administration/admin_area.md#prevent-a-user-from-creating-groups). - Use the [modify an existing user API endpoint](../api/users.md#user-modification) to change the `can_create_group` setting. ## Prevent users from changing their usernames diff --git a/doc/administration/wikis/index.md b/doc/administration/wikis/index.md index 540e50d5c70..fd7c1825e8d 100644 --- a/doc/administration/wikis/index.md +++ b/doc/administration/wikis/index.md @@ -26,7 +26,7 @@ is edited again and the content changes. ### Wiki page content size limit configuration -This setting is not available through the [Admin Area settings](../../user/admin_area/settings/index.md). +This setting is not available through the [Admin Area settings](../settings/index.md). To configure this setting, use either the Rails console or the [Application settings API](../../api/settings.md). @@ -76,7 +76,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a ### Reduce wiki repository size -The wiki counts as part of the [namespace storage size](../../user/admin_area/settings/account_and_limit_settings.md), +The wiki counts as part of the [namespace storage size](../settings/account_and_limit_settings.md), so you should keep your wiki repositories as compact as possible. For more information about tools to compact repositories, diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index 568acb76e5f..a97e4ad8adb 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -148,56 +148,57 @@ The following API resources are available in the group context: The following API resources are available outside of project and group contexts (including `/users`): -| Resource | Available endpoints | -|:----------------------------------------------------------------------------------------|:--------------------| -| [Appearance](appearance.md) **(FREE SELF)** | `/application/appearance` | -| [Applications](applications.md) | `/applications` | -| [Audit Events](audit_events.md) **(PREMIUM SELF)** | `/audit_events` | -| [Avatar](avatar.md) | `/avatar` | -| [Broadcast messages](broadcast_messages.md) | `/broadcast_messages` | -| [Code snippets](snippets.md) | `/snippets` | -| [Custom attributes](custom_attributes.md) | `/users/:id/custom_attributes` (also available for groups and projects) | -| [Deploy keys](deploy_keys.md) | `/deploy_keys` (also available for projects) | -| [Deploy tokens](deploy_tokens.md) | `/deploy_tokens` (also available for projects and groups) | -| [Events](events.md) | `/events`, `/users/:id/events` (also available for projects) | -| [Feature flags](features.md) | `/features` | -| [Geo Nodes](geo_nodes.md) **(PREMIUM SELF)** | `/geo_nodes` | -| [Group Activity Analytics](group_activity_analytics.md) | `/analytics/group_activity/{issues_count}` | -| [Group repository storage moves](group_repository_storage_moves.md) **(PREMIUM SELF)** | `/group_repository_storage_moves` | -| [Import repository from GitHub](import.md#import-repository-from-github) | `/import/github` | -| [Import repository from Bitbucket Server](import.md#import-repository-from-bitbucket-server) | `/import/bitbucket_server` | -| [Instance clusters](instance_clusters.md) **(FREE SELF)** | `/admin/clusters` | -| [Instance-level CI/CD variables](instance_level_ci_variables.md) **(FREE SELF)** | `/admin/ci/variables` | -| [Issues Statistics](issues_statistics.md) | `/issues_statistics` (also available for groups and projects) | -| [Issues](issues.md) | `/issues` (also available for groups and projects) | -| [Jobs](jobs.md) | `/job` | -| [Keys](keys.md) | `/keys` | -| [License](license.md) **(FREE SELF)** | `/license` | -| [Markdown](markdown.md) | `/markdown` | -| [Merge requests](merge_requests.md) | `/merge_requests` (also available for groups and projects) | -| [Metrics dashboard annotations](metrics_dashboard_annotations.md) | `/environments/:id/metrics_dashboard/annotations`, `/clusters/:id/metrics_dashboard/annotations` | -| [Namespaces](namespaces.md) | `/namespaces` | -| [Notification settings](notification_settings.md) | `/notification_settings` (also available for groups and projects) | -| [Pages domains](pages_domains.md) | `/pages/domains` (also available for projects) | -| [Personal access tokens](personal_access_tokens.md) | `/personal_access_tokens` | -| [Plan limits](plan_limits.md) | `/application/plan_limits` | -| [Project repository storage moves](project_repository_storage_moves.md) **(FREE SELF)** | `/project_repository_storage_moves` | -| [Projects](projects.md) | `/users/:id/projects` (also available for projects) | -| [Runners](runners.md) | `/runners` (also available for projects) | -| [Search](search.md) | `/search` (also available for groups and projects) | -| [Service Data](usage_data.md) | `/usage_data` (For GitLab instance [Administrator](../user/permissions.md) users only) | -| [Settings](settings.md) **(FREE SELF)** | `/application/settings` | -| [Sidekiq metrics](sidekiq_metrics.md) **(FREE SELF)** | `/sidekiq` | -| [Sidekiq queues administration](admin_sidekiq_queues.md) **(FREE SELF)** | `/admin/sidekiq/queues/:queue_name` | -| [Snippet repository storage moves](snippet_repository_storage_moves.md) **(FREE SELF)** | `/snippet_repository_storage_moves` | -| [Statistics](statistics.md) | `/application/statistics` | -| [Suggestions](suggestions.md) | `/suggestions` | -| [System hooks](system_hooks.md) | `/hooks` | -| [To-dos](todos.md) | `/todos` | -| [Topics](topics.md) | `/topics` | -| [Users](users.md) | `/users` | -| [Validate `.gitlab-ci.yml` file](lint.md) | `/lint` | -| [Version](version.md) | `/version` | +| Resource | Available endpoints | +|:---------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------| +| [Appearance](appearance.md) **(FREE SELF)** | `/application/appearance` | +| [Applications](applications.md) | `/applications` | +| [Audit Events](audit_events.md) **(PREMIUM SELF)** | `/audit_events` | +| [Avatar](avatar.md) | `/avatar` | +| [Broadcast messages](broadcast_messages.md) | `/broadcast_messages` | +| [Code snippets](snippets.md) | `/snippets` | +| [Code suggestions](code_suggestions.md) | `/code_suggestions` | +| [Custom attributes](custom_attributes.md) | `/users/:id/custom_attributes` (also available for groups and projects) | +| [Deploy keys](deploy_keys.md) | `/deploy_keys` (also available for projects) | +| [Deploy tokens](deploy_tokens.md) | `/deploy_tokens` (also available for projects and groups) | +| [Events](events.md) | `/events`, `/users/:id/events` (also available for projects) | +| [Feature flags](features.md) | `/features` | +| [Geo Nodes](geo_nodes.md) **(PREMIUM SELF)** | `/geo_nodes` | +| [Group Activity Analytics](group_activity_analytics.md) | `/analytics/group_activity/{issues_count}` | +| [Group repository storage moves](group_repository_storage_moves.md) **(PREMIUM SELF)** | `/group_repository_storage_moves` | +| [Import repository from GitHub](import.md#import-repository-from-github) | `/import/github` | +| [Import repository from Bitbucket Server](import.md#import-repository-from-bitbucket-server) | `/import/bitbucket_server` | +| [Instance clusters](instance_clusters.md) **(FREE SELF)** | `/admin/clusters` | +| [Instance-level CI/CD variables](instance_level_ci_variables.md) **(FREE SELF)** | `/admin/ci/variables` | +| [Issues Statistics](issues_statistics.md) | `/issues_statistics` (also available for groups and projects) | +| [Issues](issues.md) | `/issues` (also available for groups and projects) | +| [Jobs](jobs.md) | `/job` | +| [Keys](keys.md) | `/keys` | +| [License](license.md) **(FREE SELF)** | `/license` | +| [Markdown](markdown.md) | `/markdown` | +| [Merge requests](merge_requests.md) | `/merge_requests` (also available for groups and projects) | +| [Metrics dashboard annotations](metrics_dashboard_annotations.md) | `/environments/:id/metrics_dashboard/annotations`, `/clusters/:id/metrics_dashboard/annotations` | +| [Namespaces](namespaces.md) | `/namespaces` | +| [Notification settings](notification_settings.md) | `/notification_settings` (also available for groups and projects) | +| [Pages domains](pages_domains.md) | `/pages/domains` (also available for projects) | +| [Personal access tokens](personal_access_tokens.md) | `/personal_access_tokens` | +| [Plan limits](plan_limits.md) | `/application/plan_limits` | +| [Project repository storage moves](project_repository_storage_moves.md) **(FREE SELF)** | `/project_repository_storage_moves` | +| [Projects](projects.md) | `/users/:id/projects` (also available for projects) | +| [Runners](runners.md) | `/runners` (also available for projects) | +| [Search](search.md) | `/search` (also available for groups and projects) | +| [Service Data](usage_data.md) | `/usage_data` (For GitLab instance [Administrator](../user/permissions.md) users only) | +| [Settings](settings.md) **(FREE SELF)** | `/application/settings` | +| [Sidekiq metrics](sidekiq_metrics.md) **(FREE SELF)** | `/sidekiq` | +| [Sidekiq queues administration](admin_sidekiq_queues.md) **(FREE SELF)** | `/admin/sidekiq/queues/:queue_name` | +| [Snippet repository storage moves](snippet_repository_storage_moves.md) **(FREE SELF)** | `/snippet_repository_storage_moves` | +| [Statistics](statistics.md) | `/application/statistics` | +| [Suggestions](suggestions.md) | `/suggestions` | +| [System hooks](system_hooks.md) | `/hooks` | +| [To-dos](todos.md) | `/todos` | +| [Topics](topics.md) | `/topics` | +| [Users](users.md) | `/users` | +| [Validate `.gitlab-ci.yml` file](lint.md) | `/lint` | +| [Version](version.md) | `/version` | ## Templates API resources diff --git a/doc/api/audit_events.md b/doc/api/audit_events.md index e4856010b6c..89e89366de5 100644 --- a/doc/api/audit_events.md +++ b/doc/api/audit_events.md @@ -138,16 +138,10 @@ Example response: ## Group Audit Events -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34078) in GitLab 12.5. -> - Support for keyset pagination [added](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2. -> - Also returning project audit events for projects within the given group was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/feat/337757) in GitLab 16.1 behind feature flag `audit_event_group_rollup`. Disabled by default. - -FLAG: -On self-managed GitLab, by default returning project audit events for projects within the given group is not available. To make it available, ask an administrator -to [enable the feature flag](../administration/feature_flags.md) named `audit_event_group_rollup`. On GitLab.com, this feature is not available. The feature is not ready for -production use. +> Support for keyset pagination [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333968) in GitLab 15.2. The Group Audit Events API allows you to retrieve [group audit events](../administration/audit_events.md#group-events). +This API cannot retrieve project audit events. A user with: diff --git a/doc/api/award_emoji.md b/doc/api/award_emoji.md index a22c61b3391..2ff71a088e6 100644 --- a/doc/api/award_emoji.md +++ b/doc/api/award_emoji.md @@ -6,12 +6,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Emoji reactions API **(FREE)** -> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emojis" to "emoji reactions" in GitLab 16.0. +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emoji" to "emoji reactions" in GitLab 16.0. An [emoji reaction](../user/award_emojis.md) tells a thousand words. We call GitLab objects on which you can react with an emoji "awardables". -You can react with emojis on the following: +You can react with emoji on the following: - [Epics](../user/group/epics/index.md) ([API](epics.md)). **(PREMIUM)** - [Issues](../user/project/issues/index.md) ([API](issues.md)). diff --git a/doc/api/broadcast_messages.md b/doc/api/broadcast_messages.md index d91557523a9..9dbae6f5727 100644 --- a/doc/api/broadcast_messages.md +++ b/doc/api/broadcast_messages.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w - > `target_access_levels` [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default. - > `color` parameter [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95829) in GitLab 15.6. -Broadcast messages API operates on [broadcast messages](../user/admin_area/broadcast_messages.md). +Broadcast messages API operates on [broadcast messages](../administration/broadcast_messages.md). As of GitLab 12.8, GET requests do not require authentication. All other broadcast message API endpoints are accessible only to administrators. Non-GET requests by: diff --git a/doc/api/cluster_agents.md b/doc/api/cluster_agents.md index 1753757e5d9..5abdece3909 100644 --- a/doc/api/cluster_agents.md +++ b/doc/api/cluster_agents.md @@ -366,7 +366,8 @@ Example response: ## Create an agent token > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347046) in GitLab 15.0. -> - Two-token limit [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361030) in GitLab 16.1. +> - Two-token limit [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361030/) in GitLab 16.1 with a [flag](../administration/feature_flags.md) named `cluster_agents_limit_tokens_created`. +> - Two-token limit [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/412399) in GitLab 16.2. Feature flag `cluster_agents_limit_tokens_created` removed. Creates a new token for an agent. diff --git a/doc/api/code_suggestions.md b/doc/api/code_suggestions.md new file mode 100644 index 00000000000..8057686897f --- /dev/null +++ b/doc/api/code_suggestions.md @@ -0,0 +1,81 @@ +--- +stage: Manage +group: AI assisted +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Code Suggestions API + +Use the Code Suggestions API to access the Code Suggestions feature. + +## Create an access token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404427) in GitLab 16.1. + +Creates an access token to access Code Suggestions. + +```plaintext +POST /code_suggestions/tokens +``` + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/code_suggestions/tokens" +``` + +Example response: + +```json +{ + "access_token": "secret-access-token", + "expires_in": 3600, + "created_at": 1687865199 +} +``` + +## Generate code completions (Experiment) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/415581) in GitLab 16.2 [with a flag](../administration/feature_flags.md) named `code_suggestions_completion_api`. Disabled by default. This feature is an Experiment. + +FLAG: +On self-managed GitLab, by default this feature is not available. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +Use the AI abstraction layer to generate code completions. + +```plaintext +POST /code_suggestions/completions +``` + +Requests to this endpoint are proxied directly to the [model gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#completions). The documentation for the endpoint is currently the SSoT for named parameters. + +Authentication to this endpoint requires both a GitLab access token and a Code Suggestions JWT. The access token is used to authenticate the user and the JWT is used to authenticate the request to the model gateway. + +```shell +curl --header "Authorization: Bearer <YOUR_ACCESS_TOKEN>" --header "X-Gitlab-Oidc-Token: <TOKEN_GENERATED_FROM_TOKENS_ENDPOINT>" --data "<JSON_BODY>" https://gitlab.example.com/api/v4/code_suggestions/completions +``` + +Example body: + +The [model gateway](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#completions) is the SSoT for parameters. + +Example response: + +```json +{ + "id": "id", + "model": { + "engine": "vertex-ai", + "name": "code-gecko" + }, + "object": "text_completion", + "created": 1688557841, + "choices": [ + { + "text": "\n if self.is_running:\n self.speed += increment\n print(\"The car's speed is now", + "index": 0, + "finish_reason": "length" + } + ] +} +``` diff --git a/doc/api/database_migrations.md b/doc/api/database_migrations.md new file mode 100644 index 00000000000..d7aea7ad57e --- /dev/null +++ b/doc/api/database_migrations.md @@ -0,0 +1,33 @@ +--- +stage: Data Stores +group: Database +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Database migrations API **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123408) in GitLab 16.2. + +This API is for managing database migrations used in the development of GitLab. + +All methods require administrator authorization. + +## Mark a migration as successful + +Mark pending migrations as successfully executed to prevent them from being +executed by the `db:migrate` tasks. Use this API to skip failing +migrations after they are determined to be safe to skip. + +```plaintext +POST /api/v4/admin/migrations/:version/mark +``` + +| Attribute | Type | Required | Description | +|-----------------|----------------|----------|----------------------------------------------------------------------------------| +| `version` | integer | yes | Version timestamp of the migration to be skipped | +| `database` | string | no | The database name for which the migration is skipped. Defaults to `main`. | + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" \ + --url "https://gitlab.example.com/api/v4/admin/migrations/:version/mark" +``` diff --git a/doc/api/deployments.md b/doc/api/deployments.md index e937a234b08..9ef75741142 100644 --- a/doc/api/deployments.md +++ b/doc/api/deployments.md @@ -7,6 +7,8 @@ type: concepts, howto # Deployments API **(FREE)** +> Support for [GitLab CI/CD job token](../ci/jobs/ci_job_token.md) authentication [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414549) in GitLab 16.2. + ## List project deployments Get a list of deployments in a project. @@ -359,7 +361,7 @@ POST /projects/:id/deployments | `sha` | string | yes | The SHA of the commit that is deployed. | | `ref` | string | yes | The name of the branch or tag that is deployed. | | `tag` | boolean | yes | A boolean that indicates if the deployed ref is a tag (`true`) or not (`false`). | -| `status` | string | yes | The status to filter deployments by. One of `running`, `success`, `failed`, or `canceled`. | +| `status` | string | yes | The status of the deployment that is created. One of `running`, `success`, `failed`, or `canceled` | ```shell curl --data "environment=production&sha=a91957a858320c0e17f3a0eca7cfacbff50ea29a&ref=main&tag=false&status=success" \ diff --git a/doc/api/environments.md b/doc/api/environments.md index d2335149301..87f99bc9034 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -7,6 +7,8 @@ type: concepts, howto # Environments API **(FREE)** +> Support for [GitLab CI/CD job token](../ci/jobs/ci_job_token.md) authentication [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414549) in GitLab 16.2. + ## List environments Get all environments for a given project. diff --git a/doc/api/error_tracking.md b/doc/api/error_tracking.md index d1ab67a93ae..9037c1b58d2 100644 --- a/doc/api/error_tracking.md +++ b/doc/api/error_tracking.md @@ -1,5 +1,5 @@ --- -stage: Monitor +stage: Analytics group: Observability info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index b9d4f93d841..4ec57274bd7 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -560,7 +560,19 @@ Example response: "design_management_repositories_verified_count": 5, "design_management_repositories_verification_failed_count": 5, "design_management_repositories_synced_in_percentage": "100.00%", - "design_management_repositories_verified_in_percentage": "100.00%" + "design_management_repositories_verified_in_percentage": "100.00%", + "project_repositories_count": 5, + "project_repositories_checksum_total_count": 5, + "project_repositories_checksummed_count": 5, + "project_repositories_checksum_failed_count": 0, + "project_repositories_synced_count": 5, + "project_repositories_failed_count": 0, + "project_repositories_registry_count": 5, + "project_repositories_verification_total_count": 5, + "project_repositories_verified_count": 5, + "project_repositories_verification_failed_count": 0, + "project_repositories_synced_in_percentage": "100.00%", + "project_repositories_verified_in_percentage": "100.00%" }, { "geo_node_id": 2, @@ -775,7 +787,19 @@ Example response: "dependency_proxy_manifests_verified_count": 5, "dependency_proxy_manifests_verification_failed_count": 5, "dependency_proxy_manifests_synced_in_percentage": "100.00%", - "dependency_proxy_manifests_verified_in_percentage": "100.00%" + "dependency_proxy_manifests_verified_in_percentage": "100.00%", + "project_repositories_count": 5, + "project_repositories_checksum_total_count": 5, + "project_repositories_checksummed_count": 5, + "project_repositories_checksum_failed_count": 0, + "project_repositories_synced_count": 5, + "project_repositories_failed_count": 0, + "project_repositories_registry_count": 5, + "project_repositories_verification_total_count": 5, + "project_repositories_verified_count": 5, + "project_repositories_verification_failed_count": 0, + "project_repositories_synced_in_percentage": "100.00%", + "project_repositories_verified_in_percentage": "100.00%" } ] ``` @@ -1000,7 +1024,19 @@ Example response: "design_management_repositories_verified_count": 5, "design_management_repositories_verification_failed_count": 5, "design_management_repositories_synced_in_percentage": "100.00%", - "design_management_repositories_verified_in_percentage": "100.00%" + "design_management_repositories_verified_in_percentage": "100.00%", + "project_repositories_count": 5, + "project_repositories_checksum_total_count": 5, + "project_repositories_checksummed_count": 5, + "project_repositories_checksum_failed_count": 0, + "project_repositories_synced_count": 5, + "project_repositories_failed_count": 0, + "project_repositories_registry_count": 5, + "project_repositories_verification_total_count": 5, + "project_repositories_verified_count": 5, + "project_repositories_verification_failed_count": 0, + "project_repositories_synced_in_percentage": "100.00%", + "project_repositories_verified_in_percentage": "100.00%" } ``` diff --git a/doc/api/graphql/custom_emoji.md b/doc/api/graphql/custom_emoji.md index 25ae37b75a9..c39ac955c01 100644 --- a/doc/api/graphql/custom_emoji.md +++ b/doc/api/graphql/custom_emoji.md @@ -4,17 +4,17 @@ group: Project Management info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Use custom emojis with GraphQL **(FREE)** +# Use custom emoji with GraphQL **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6 [with a flag](../../administration/feature_flags.md) named `custom_emoji`. Disabled by default. > - Enabled on GitLab.com in GitLab 14.0. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `custom_emoji`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `custom_emoji`. On GitLab.com, this feature is available. This feature is ready for production use. -To use custom emojis in comments and descriptions, you can add them to a top-level group using the GraphQL API. +To use [custom emoji](../../user/award_emojis.md) in comments and descriptions, you can add them to a top-level group using the GraphQL API. Parameters: @@ -38,7 +38,7 @@ mutation CreateCustomEmoji($groupPath: ID!) { } ``` -After adding a custom emoji to the group, members can use it in the same way as other emojis in the comments. +After adding a custom emoji to the group, members can use it in the same way as other emoji in the comments. ## Get custom emoji for a group diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index abedb63d575..9a78d43ff4b 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -57,7 +57,7 @@ You can work with sample queries that pull data from public projects on GitLab.c - [Create an audit report](audit_report.md) - [Identify issue boards](sample_issue_boards.md) - [Query users](users_example.md) -- [Use custom emojis](custom_emoji.md) +- [Use custom emoji](custom_emoji.md) The [get started](getting_started.md) page includes different methods to customize GraphQL queries. diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 95420764226..425a2b7e980 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -253,6 +253,22 @@ Returns [`EpicList`](#epiclist). | <a id="queryepicboardlistepicfilters"></a>`epicFilters` | [`EpicFilters`](#epicfilters) | Filters applied when getting epic metadata in the epic board list. | | <a id="queryepicboardlistid"></a>`id` | [`BoardsEpicListID!`](#boardsepiclistid) | Global ID of the list. | +### `Query.explainVulnerabilityPrompt` + +Explain This Vulnerability Prompt for a specified Vulnerability. + +WARNING: +**Introduced** in 16.2. +This feature is an Experiment. It can be changed or removed at any time. + +Returns [`ExplainVulnerabilityPrompt`](#explainvulnerabilityprompt). + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="queryexplainvulnerabilitypromptvulnerabilityid"></a>`vulnerabilityId` | [`VulnerabilityID!`](#vulnerabilityid) | Vulnerability to generate a prompt for. | + ### `Query.geoNode` Find a Geo node. @@ -348,6 +364,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="queryissuesassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. | | <a id="queryissuesassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. | | <a id="queryissuesassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | +| <a id="queryissuesassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="queryissuesauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | | <a id="queryissuesclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. | | <a id="queryissuesclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. | @@ -357,13 +374,16 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="queryissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="queryissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="queryissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="queryissuesepicwildcardid"></a>`epicWildcardId` | [`EpicWildcardId`](#epicwildcardid) | Filter by epic ID wildcard. Incompatible with epicId. | | <a id="queryissueshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. | | <a id="queryissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="queryissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | | <a id="queryissuesin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="queryissuesincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Whether to include issues from archived projects. Defaults to `false`. | | <a id="queryissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | +| <a id="queryissuesiterationcadenceid"></a>`iterationCadenceId` | [`[IterationsCadenceID!]`](#iterationscadenceid) | Filter by a list of iteration cadence IDs. | | <a id="queryissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | +| <a id="queryissuesiterationtitle"></a>`iterationTitle` | [`String`](#string) | Filter by iteration title. | | <a id="queryissuesiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | | <a id="queryissueslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="queryissuesmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | @@ -378,6 +398,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="queryissuesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. | | <a id="queryissuesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. | | <a id="queryissuesweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. | +| <a id="queryissuesweightwildcardid"></a>`weightWildcardId` | [`WeightWildcardId`](#weightwildcardid) | Filter by weight ID wildcard. Incompatible with weight. | ### `Query.iteration` @@ -853,6 +874,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="queryworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="queryworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="queryworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | ## `Mutation` type @@ -1022,6 +1045,7 @@ Input type: `AdminSidekiqQueuesDeleteJobsInput` | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="mutationadminsidekiqqueuesdeletejobsairesource"></a>`aiResource` | [`String`](#string) | Delete jobs matching ai_resource in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsartifactsize"></a>`artifactSize` | [`String`](#string) | Delete jobs matching artifact_size in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsartifactusedcdn"></a>`artifactUsedCdn` | [`String`](#string) | Delete jobs matching artifact_used_cdn in the context metadata. | | <a id="mutationadminsidekiqqueuesdeletejobsartifactsdependenciescount"></a>`artifactsDependenciesCount` | [`String`](#string) | Delete jobs matching artifacts_dependencies_count in the context metadata. | @@ -1191,7 +1215,7 @@ Input type: `AuditEventsStreamingDestinationEventsAddInput` | ---- | ---- | ----------- | | <a id="mutationauditeventsstreamingdestinationeventsaddclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationauditeventsstreamingdestinationeventsadderrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | -| <a id="mutationauditeventsstreamingdestinationeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]`](#string) | Event type filters present. | +| <a id="mutationauditeventsstreamingdestinationeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]`](#string) | List of event type filters for the audit event external destination. | ### `Mutation.auditEventsStreamingDestinationEventsRemove` @@ -1212,6 +1236,26 @@ Input type: `AuditEventsStreamingDestinationEventsRemoveInput` | <a id="mutationauditeventsstreamingdestinationeventsremoveclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationauditeventsstreamingdestinationeventsremoveerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +### `Mutation.auditEventsStreamingDestinationInstanceEventsAdd` + +Input type: `AuditEventsStreamingDestinationInstanceEventsAddInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsaddclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsadddestinationid"></a>`destinationId` | [`AuditEventsInstanceExternalAuditEventDestinationID!`](#auditeventsinstanceexternalauditeventdestinationid) | Destination id. | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]!`](#string) | List of event type filters to add for streaming. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsaddclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsadderrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationauditeventsstreamingdestinationinstanceeventsaddeventtypefilters"></a>`eventTypeFilters` | [`[String!]`](#string) | List of event type filters for the audit event external destination. | + ### `Mutation.auditEventsStreamingHeadersCreate` Input type: `AuditEventsStreamingHeadersCreateInput` @@ -1293,6 +1337,24 @@ Input type: `AuditEventsStreamingInstanceHeadersCreateInput` | <a id="mutationauditeventsstreaminginstanceheaderscreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationauditeventsstreaminginstanceheaderscreateheader"></a>`header` | [`AuditEventsStreamingInstanceHeader`](#auditeventsstreaminginstanceheader) | Created header. | +### `Mutation.auditEventsStreamingInstanceHeadersDestroy` + +Input type: `AuditEventsStreamingInstanceHeadersDestroyInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationauditeventsstreaminginstanceheadersdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationauditeventsstreaminginstanceheadersdestroyheaderid"></a>`headerId` | [`AuditEventsStreamingInstanceHeaderID!`](#auditeventsstreaminginstanceheaderid) | Header to delete. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationauditeventsstreaminginstanceheadersdestroyclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationauditeventsstreaminginstanceheadersdestroyerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | + ### `Mutation.auditEventsStreamingInstanceHeadersUpdate` Input type: `AuditEventsStreamingInstanceHeadersUpdateInput` @@ -3142,6 +3204,7 @@ Input type: `EnvironmentCreateInput` | <a id="mutationenvironmentcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationenvironmentcreateclusteragentid"></a>`clusterAgentId` | [`ClustersAgentID`](#clustersagentid) | Cluster agent of the environment. | | <a id="mutationenvironmentcreateexternalurl"></a>`externalUrl` | [`String`](#string) | External URL of the environment. | +| <a id="mutationenvironmentcreatekubernetesnamespace"></a>`kubernetesNamespace` | [`String`](#string) | Kubernetes namespace of the environment. | | <a id="mutationenvironmentcreatename"></a>`name` | [`String!`](#string) | Name of the environment. | | <a id="mutationenvironmentcreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Full path of the project. | | <a id="mutationenvironmentcreatetier"></a>`tier` | [`DeploymentTier`](#deploymenttier) | Tier of the environment. | @@ -3210,6 +3273,7 @@ Input type: `EnvironmentUpdateInput` | <a id="mutationenvironmentupdateclusteragentid"></a>`clusterAgentId` | [`ClustersAgentID`](#clustersagentid) | Cluster agent of the environment. | | <a id="mutationenvironmentupdateexternalurl"></a>`externalUrl` | [`String`](#string) | External URL of the environment. | | <a id="mutationenvironmentupdateid"></a>`id` | [`EnvironmentID!`](#environmentid) | Global ID of the environment to update. | +| <a id="mutationenvironmentupdatekubernetesnamespace"></a>`kubernetesNamespace` | [`String`](#string) | Kubernetes namespace of the environment. | | <a id="mutationenvironmentupdatetier"></a>`tier` | [`DeploymentTier`](#deploymenttier) | Tier of the environment. | #### Fields @@ -3518,6 +3582,7 @@ Input type: `ExternalAuditEventDestinationCreateInput` | <a id="mutationexternalauditeventdestinationcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationexternalauditeventdestinationcreatedestinationurl"></a>`destinationUrl` | [`String!`](#string) | Destination URL. | | <a id="mutationexternalauditeventdestinationcreategrouppath"></a>`groupPath` | [`ID!`](#id) | Group path. | +| <a id="mutationexternalauditeventdestinationcreatename"></a>`name` | [`String`](#string) | Destination name. | | <a id="mutationexternalauditeventdestinationcreateverificationtoken"></a>`verificationToken` | [`String`](#string) | Verification token. | #### Fields @@ -3557,6 +3622,7 @@ Input type: `ExternalAuditEventDestinationUpdateInput` | <a id="mutationexternalauditeventdestinationupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationexternalauditeventdestinationupdatedestinationurl"></a>`destinationUrl` | [`String`](#string) | Destination URL to change. | | <a id="mutationexternalauditeventdestinationupdateid"></a>`id` | [`AuditEventsExternalAuditEventDestinationID!`](#auditeventsexternalauditeventdestinationid) | ID of external audit event destination to update. | +| <a id="mutationexternalauditeventdestinationupdatename"></a>`name` | [`String`](#string) | Destination name. | #### Fields @@ -3813,6 +3879,7 @@ Input type: `InstanceExternalAuditEventDestinationCreateInput` | ---- | ---- | ----------- | | <a id="mutationinstanceexternalauditeventdestinationcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationinstanceexternalauditeventdestinationcreatedestinationurl"></a>`destinationUrl` | [`String!`](#string) | Destination URL. | +| <a id="mutationinstanceexternalauditeventdestinationcreatename"></a>`name` | [`String`](#string) | Destination name. | #### Fields @@ -3851,6 +3918,7 @@ Input type: `InstanceExternalAuditEventDestinationUpdateInput` | <a id="mutationinstanceexternalauditeventdestinationupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationinstanceexternalauditeventdestinationupdatedestinationurl"></a>`destinationUrl` | [`String`](#string) | Destination URL to change. | | <a id="mutationinstanceexternalauditeventdestinationupdateid"></a>`id` | [`AuditEventsInstanceExternalAuditEventDestinationID!`](#auditeventsinstanceexternalauditeventdestinationid) | ID of the external instance audit event destination to update. | +| <a id="mutationinstanceexternalauditeventdestinationupdatename"></a>`name` | [`String`](#string) | Destination name. | #### Fields @@ -4846,7 +4914,7 @@ Input type: `MergeRequestUpdateApprovalRuleInput` | <a id="mutationmergerequestupdateapprovalruleiid"></a>`iid` | [`String!`](#string) | IID of the merge request to mutate. | | <a id="mutationmergerequestupdateapprovalrulename"></a>`name` | [`String!`](#string) | Name of the approval rule. | | <a id="mutationmergerequestupdateapprovalruleprojectpath"></a>`projectPath` | [`ID!`](#id) | Project the merge request to mutate is in. | -| <a id="mutationmergerequestupdateapprovalruleremovehiddengroups"></a>`removeHiddenGroups` | [`[Boolean!]`](#boolean) | Whether hidden groups should be removed. | +| <a id="mutationmergerequestupdateapprovalruleremovehiddengroups"></a>`removeHiddenGroups` | [`Boolean`](#boolean) | Whether hidden groups should be removed. | | <a id="mutationmergerequestupdateapprovalruleuserids"></a>`userIds` | [`[String!]`](#string) | IDs of users as approvers. | #### Fields @@ -5375,7 +5443,7 @@ Input type: `PrometheusIntegrationCreateInput` | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mutationprometheusintegrationcreateactive"></a>`active` | [`Boolean!`](#boolean) | Whether the integration is receiving alerts. | -| <a id="mutationprometheusintegrationcreateapiurl"></a>`apiUrl` | [`String!`](#string) | Endpoint at which Prometheus can be queried. | +| <a id="mutationprometheusintegrationcreateapiurl"></a>`apiUrl` | [`String`](#string) | Endpoint at which Prometheus can be queried. | | <a id="mutationprometheusintegrationcreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationprometheusintegrationcreateprojectpath"></a>`projectPath` | [`ID!`](#id) | Project to create the integration in. | @@ -6837,6 +6905,31 @@ Input type: `UserSetNamespaceCommitEmailInput` | <a id="mutationusersetnamespacecommitemailerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationusersetnamespacecommitemailnamespacecommitemail"></a>`namespaceCommitEmail` | [`NamespaceCommitEmail`](#namespacecommitemail) | User namespace commit email after mutation. | +### `Mutation.vulnerabilitiesDismiss` + +WARNING: +**Introduced** in 16.2. +This feature is an Experiment. It can be changed or removed at any time. + +Input type: `VulnerabilitiesDismissInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationvulnerabilitiesdismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationvulnerabilitiesdismisscomment"></a>`comment` | [`String`](#string) | Comment why vulnerability was dismissed (maximum 50,000 characters). | +| <a id="mutationvulnerabilitiesdismissdismissalreason"></a>`dismissalReason` | [`VulnerabilityDismissalReason`](#vulnerabilitydismissalreason) | Reason why vulnerability should be dismissed. | +| <a id="mutationvulnerabilitiesdismissvulnerabilityids"></a>`vulnerabilityIds` | [`[VulnerabilityID!]!`](#vulnerabilityid) | IDs of the vulnerabilities to be dismissed. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationvulnerabilitiesdismissclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationvulnerabilitiesdismisserrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationvulnerabilitiesdismissvulnerabilities"></a>`vulnerabilities` **{warning-solid}** | [`[Vulnerability!]!`](#vulnerability) | **Deprecated:** This feature is an Experiment. It can be changed or removed at any time. Introduced in 16.2. | + ### `Mutation.vulnerabilityConfirm` Input type: `VulnerabilityConfirmInput` @@ -8423,6 +8516,29 @@ The edge type for [`ComplianceFramework`](#complianceframework). | <a id="complianceframeworkedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="complianceframeworkedgenode"></a>`node` | [`ComplianceFramework`](#complianceframework) | The item at the end of the edge. | +#### `ComplianceStandardsAdherenceConnection` + +The connection type for [`ComplianceStandardsAdherence`](#compliancestandardsadherence). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="compliancestandardsadherenceconnectionedges"></a>`edges` | [`[ComplianceStandardsAdherenceEdge]`](#compliancestandardsadherenceedge) | A list of edges. | +| <a id="compliancestandardsadherenceconnectionnodes"></a>`nodes` | [`[ComplianceStandardsAdherence]`](#compliancestandardsadherence) | A list of nodes. | +| <a id="compliancestandardsadherenceconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `ComplianceStandardsAdherenceEdge` + +The edge type for [`ComplianceStandardsAdherence`](#compliancestandardsadherence). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="compliancestandardsadherenceedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="compliancestandardsadherenceedgenode"></a>`node` | [`ComplianceStandardsAdherence`](#compliancestandardsadherence) | The item at the end of the edge. | + #### `ComplianceViolationConnection` The connection type for [`ComplianceViolation`](#complianceviolation). @@ -9893,6 +10009,29 @@ The connection type for [`MergeRequest`](#mergerequest). | <a id="mergerequestconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | | <a id="mergerequestconnectiontotaltimetomerge"></a>`totalTimeToMerge` | [`Float`](#float) | Total sum of time to merge, in seconds, for the collection of merge requests. | +#### `MergeRequestDiffConnection` + +The connection type for [`MergeRequestDiff`](#mergerequestdiff). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestdiffconnectionedges"></a>`edges` | [`[MergeRequestDiffEdge]`](#mergerequestdiffedge) | A list of edges. | +| <a id="mergerequestdiffconnectionnodes"></a>`nodes` | [`[MergeRequestDiff]`](#mergerequestdiff) | A list of nodes. | +| <a id="mergerequestdiffconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `MergeRequestDiffEdge` + +The edge type for [`MergeRequestDiff`](#mergerequestdiff). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestdiffedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="mergerequestdiffedgenode"></a>`node` | [`MergeRequestDiff`](#mergerequestdiff) | The item at the end of the edge. | + #### `MergeRequestDiffLlmSummaryConnection` The connection type for [`MergeRequestDiffLlmSummary`](#mergerequestdiffllmsummary). @@ -9973,51 +10112,51 @@ The edge type for [`MergeRequestParticipant`](#mergerequestparticipant). | <a id="mergerequestparticipantedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="mergerequestparticipantedgenode"></a>`node` | [`MergeRequestParticipant`](#mergerequestparticipant) | The item at the end of the edge. | -#### `MergeRequestReviewerConnection` +#### `MergeRequestReviewLlmSummaryConnection` -The connection type for [`MergeRequestReviewer`](#mergerequestreviewer). +The connection type for [`MergeRequestReviewLlmSummary`](#mergerequestreviewllmsummary). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="mergerequestreviewerconnectionedges"></a>`edges` | [`[MergeRequestReviewerEdge]`](#mergerequestrevieweredge) | A list of edges. | -| <a id="mergerequestreviewerconnectionnodes"></a>`nodes` | [`[MergeRequestReviewer]`](#mergerequestreviewer) | A list of nodes. | -| <a id="mergerequestreviewerconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | +| <a id="mergerequestreviewllmsummaryconnectionedges"></a>`edges` | [`[MergeRequestReviewLlmSummaryEdge]`](#mergerequestreviewllmsummaryedge) | A list of edges. | +| <a id="mergerequestreviewllmsummaryconnectionnodes"></a>`nodes` | [`[MergeRequestReviewLlmSummary]`](#mergerequestreviewllmsummary) | A list of nodes. | +| <a id="mergerequestreviewllmsummaryconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | -#### `MergeRequestReviewerEdge` +#### `MergeRequestReviewLlmSummaryEdge` -The edge type for [`MergeRequestReviewer`](#mergerequestreviewer). +The edge type for [`MergeRequestReviewLlmSummary`](#mergerequestreviewllmsummary). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="mergerequestrevieweredgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | -| <a id="mergerequestrevieweredgenode"></a>`node` | [`MergeRequestReviewer`](#mergerequestreviewer) | The item at the end of the edge. | +| <a id="mergerequestreviewllmsummaryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="mergerequestreviewllmsummaryedgenode"></a>`node` | [`MergeRequestReviewLlmSummary`](#mergerequestreviewllmsummary) | The item at the end of the edge. | -#### `MetricsDashboardAnnotationConnection` +#### `MergeRequestReviewerConnection` -The connection type for [`MetricsDashboardAnnotation`](#metricsdashboardannotation). +The connection type for [`MergeRequestReviewer`](#mergerequestreviewer). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="metricsdashboardannotationconnectionedges"></a>`edges` | [`[MetricsDashboardAnnotationEdge]`](#metricsdashboardannotationedge) | A list of edges. | -| <a id="metricsdashboardannotationconnectionnodes"></a>`nodes` | [`[MetricsDashboardAnnotation]`](#metricsdashboardannotation) | A list of nodes. | -| <a id="metricsdashboardannotationconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | +| <a id="mergerequestreviewerconnectionedges"></a>`edges` | [`[MergeRequestReviewerEdge]`](#mergerequestrevieweredge) | A list of edges. | +| <a id="mergerequestreviewerconnectionnodes"></a>`nodes` | [`[MergeRequestReviewer]`](#mergerequestreviewer) | A list of nodes. | +| <a id="mergerequestreviewerconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | -#### `MetricsDashboardAnnotationEdge` +#### `MergeRequestReviewerEdge` -The edge type for [`MetricsDashboardAnnotation`](#metricsdashboardannotation). +The edge type for [`MergeRequestReviewer`](#mergerequestreviewer). ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="metricsdashboardannotationedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | -| <a id="metricsdashboardannotationedgenode"></a>`node` | [`MetricsDashboardAnnotation`](#metricsdashboardannotation) | The item at the end of the edge. | +| <a id="mergerequestrevieweredgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="mergerequestrevieweredgenode"></a>`node` | [`MergeRequestReviewer`](#mergerequestreviewer) | The item at the end of the edge. | #### `MilestoneConnection` @@ -10599,6 +10738,29 @@ The edge type for [`ProjectMember`](#projectmember). | <a id="projectmemberedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="projectmemberedgenode"></a>`node` | [`ProjectMember`](#projectmember) | The item at the end of the edge. | +#### `ProjectRepositoryRegistryConnection` + +The connection type for [`ProjectRepositoryRegistry`](#projectrepositoryregistry). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectrepositoryregistryconnectionedges"></a>`edges` | [`[ProjectRepositoryRegistryEdge]`](#projectrepositoryregistryedge) | A list of edges. | +| <a id="projectrepositoryregistryconnectionnodes"></a>`nodes` | [`[ProjectRepositoryRegistry]`](#projectrepositoryregistry) | A list of nodes. | +| <a id="projectrepositoryregistryconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `ProjectRepositoryRegistryEdge` + +The edge type for [`ProjectRepositoryRegistry`](#projectrepositoryregistry). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectrepositoryregistryedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="projectrepositoryregistryedgenode"></a>`node` | [`ProjectRepositoryRegistry`](#projectrepositoryregistry) | The item at the end of the edge. | + #### `ProjectWikiRepositoryRegistryConnection` The connection type for [`ProjectWikiRepositoryRegistry`](#projectwikirepositoryregistry). @@ -11987,6 +12149,19 @@ Representation of a GitLab user. | <a id="achievementupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp the achievement was last updated. | | <a id="achievementuserachievements"></a>`userAchievements` **{warning-solid}** | [`UserAchievementConnection`](#userachievementconnection) | **Introduced** in 15.10. This feature is an Experiment. It can be changed or removed at any time. Recipients for the achievement. | +### `AddOnPurchase` + +Represents AddOn purchase for Namespace. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="addonpurchaseassignedquantity"></a>`assignedQuantity` | [`Int!`](#int) | Number of seats assigned. | +| <a id="addonpurchaseid"></a>`id` | [`GitlabSubscriptionsAddOnPurchaseID!`](#gitlabsubscriptionsaddonpurchaseid) | ID of AddOnPurchase. | +| <a id="addonpurchasename"></a>`name` | [`String!`](#string) | Name of AddOn. | +| <a id="addonpurchasepurchasedquantity"></a>`purchasedQuantity` | [`Int!`](#int) | Number of seats purchased. | + ### `AgentConfiguration` Configuration details for an Agent. @@ -12068,7 +12243,6 @@ Describes an alert from the project's Alert Management. | <a id="alertmanagementalertiid"></a>`iid` | [`ID!`](#id) | Internal ID of the alert. | | <a id="alertmanagementalertissue"></a>`issue` | [`Issue`](#issue) | Issue attached to the alert. | | <a id="alertmanagementalertissueiid"></a>`issueIid` **{warning-solid}** | [`ID`](#id) | **Deprecated** in 13.10. Use issue field. | -| <a id="alertmanagementalertmetricsdashboardurl"></a>`metricsDashboardUrl` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.0. Returns no data. Underlying feature was removed in 16.0. | | <a id="alertmanagementalertmonitoringtool"></a>`monitoringTool` | [`String`](#string) | Monitoring tool the alert came from. | | <a id="alertmanagementalertnotes"></a>`notes` | [`NoteConnection!`](#noteconnection) | All notes on this noteable. (see [Connections](#connections)) | | <a id="alertmanagementalertprometheusalert"></a>`prometheusAlert` | [`PrometheusAlert`](#prometheusalert) | Alert condition for Prometheus. | @@ -12610,7 +12784,8 @@ Represents a list for an issue board. | <a id="boardlistmilestone"></a>`milestone` | [`Milestone`](#milestone) | Milestone of the list. | | <a id="boardlistposition"></a>`position` | [`Int`](#int) | Position of list within the board. | | <a id="boardlisttitle"></a>`title` | [`String!`](#string) | Title of the list. | -| <a id="boardlisttotalweight"></a>`totalWeight` | [`Int`](#int) | Total weight of all issues in the list. | +| <a id="boardlisttotalissueweight"></a>`totalIssueWeight` | [`BigInt`](#bigint) | Total weight of all issues in the list, encoded as a string. | +| <a id="boardlisttotalweight"></a>`totalWeight` **{warning-solid}** | [`Int`](#int) | **Deprecated** in 16.2. Use `totalIssueWeight`. | #### Fields with arguments @@ -12723,12 +12898,11 @@ Represents the total number of issues and their weights for a particular day. ##### `CiCatalogResource.versions` -Versions of the catalog resource. +Versions of the catalog resource. This field can only be resolved for one catalog resource in any single request. WARNING: -**Deprecated** in 16.1. -Causes performance degradation. -Use: `latest_version`. +**Introduced** in 16.2. +This feature is an Experiment. It can be changed or removed at any time. Returns [`ReleaseConnection`](#releaseconnection). @@ -12771,12 +12945,12 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="ciconfigincludeblob"></a>`blob` | [`String`](#string) | File blob location. It can be masked if it contains masked variables, e.g., "https://gitlab.com/gitlab-org/gitlab/-/blob/e52d6d0246d7375291850e61f0abc101fbda9dc2/.gitlab/ci/build-images.gitlab-ci.yml". | +| <a id="ciconfigincludeblob"></a>`blob` | [`String`](#string) | File blob location. It can be masked if it contains masked variables. For example, `"https://gitlab.com/gitlab-org/gitlab/-/blob/e52d6d0246d7375291850e61f0abc101fbda9dc2/.gitlab/ci/build-images.gitlab-ci.yml"`. | | <a id="ciconfigincludecontextproject"></a>`contextProject` | [`String`](#string) | Current project scope, e.g., "gitlab-org/gitlab". | | <a id="ciconfigincludecontextsha"></a>`contextSha` | [`String`](#string) | Current sha scope. | | <a id="ciconfigincludeextra"></a>`extra` | [`JSON`](#json) | Extra information for the `include`, which can contain `job_name`, `project`, and `ref`. Values can be masked if they contain masked variables. | -| <a id="ciconfigincludelocation"></a>`location` | [`String`](#string) | File location. It can be masked if it contains masked variables, e.g., ".gitlab/ci/build-images.gitlab-ci.yml". | -| <a id="ciconfigincluderaw"></a>`raw` | [`String`](#string) | File raw location. It can be masked if it contains masked variables, e.g., "https://gitlab.com/gitlab-org/gitlab/-/raw/e52d6d0246d7375291850e61f0abc101fbda9dc2/.gitlab/ci/build-images.gitlab-ci.yml". | +| <a id="ciconfigincludelocation"></a>`location` | [`String`](#string) | File location. It can be masked if it contains masked variables. For example, `".gitlab/ci/build-images.gitlab-ci.yml"`. | +| <a id="ciconfigincluderaw"></a>`raw` | [`String`](#string) | File raw location. It can be masked if it contains masked variables. For example, `"https://gitlab.com/gitlab-org/gitlab/-/raw/e52d6d0246d7375291850e61f0abc101fbda9dc2/.gitlab/ci/build-images.gitlab-ci.yml"`. | | <a id="ciconfigincludetype"></a>`type` | [`CiConfigIncludeType`](#ciconfigincludetype) | Include type. | ### `CiConfigJob` @@ -12882,6 +13056,7 @@ CI/CD variables for a group. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="cigroupvariabledescription"></a>`description` | [`String`](#string) | Description of the variable. | | <a id="cigroupvariableenvironmentscope"></a>`environmentScope` | [`String`](#string) | Scope defining the environments that can use the variable. | | <a id="cigroupvariableid"></a>`id` | [`ID!`](#id) | ID of the variable. | | <a id="cigroupvariablekey"></a>`key` | [`String`](#string) | Name of the variable. | @@ -13040,7 +13215,7 @@ CI/CD variables given to a manual job. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="ciminutesnamespacemonthlyusageminutes"></a>`minutes` | [`Int`](#int) | Total number of units of compute used by all projects in the namespace. | +| <a id="ciminutesnamespacemonthlyusageminutes"></a>`minutes` | [`Int`](#int) | Total number of compute minutes used by all projects in the namespace. | | <a id="ciminutesnamespacemonthlyusagemonth"></a>`month` | [`String`](#string) | Month related to the usage data. | | <a id="ciminutesnamespacemonthlyusagemonthiso8601"></a>`monthIso8601` | [`ISO8601Date`](#iso8601date) | Month related to the usage data in ISO 8601 date format. | | <a id="ciminutesnamespacemonthlyusageprojects"></a>`projects` | [`CiMinutesProjectMonthlyUsageConnection`](#ciminutesprojectmonthlyusageconnection) | Compute usage data for projects in the namespace. (see [Connections](#connections)) | @@ -13052,7 +13227,7 @@ CI/CD variables given to a manual job. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="ciminutesprojectmonthlyusageminutes"></a>`minutes` | [`Int`](#int) | Number of units of compute used by the project in the month. | +| <a id="ciminutesprojectmonthlyusageminutes"></a>`minutes` | [`Int`](#int) | Number of compute minutes used by the project in the month. | | <a id="ciminutesprojectmonthlyusagename"></a>`name` **{warning-solid}** | [`String`](#string) | **Deprecated** in 15.6. Use `project.name`. | | <a id="ciminutesprojectmonthlyusageproject"></a>`project` | [`Project`](#project) | Project having the recorded usage. | | <a id="ciminutesprojectmonthlyusagesharedrunnersduration"></a>`sharedRunnersDuration` | [`Int`](#int) | Total duration (in seconds) of shared runners use by the project for the month. | @@ -13065,6 +13240,7 @@ CI/CD variables for a project. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="ciprojectvariabledescription"></a>`description` | [`String`](#string) | Description of the variable. | | <a id="ciprojectvariableenvironmentscope"></a>`environmentScope` | [`String`](#string) | Scope defining the environments that can use the variable. | | <a id="ciprojectvariableid"></a>`id` | [`ID!`](#id) | ID of the variable. | | <a id="ciprojectvariablekey"></a>`key` | [`String`](#string) | Name of the variable. | @@ -13083,7 +13259,7 @@ CI/CD variables for a project. | <a id="cirunneraccesslevel"></a>`accessLevel` | [`CiRunnerAccessLevel!`](#cirunneraccesslevel) | Access level of the runner. | | <a id="cirunneractive"></a>`active` **{warning-solid}** | [`Boolean!`](#boolean) | **Deprecated** in 14.8. Use paused. | | <a id="cirunneradminurl"></a>`adminUrl` | [`String`](#string) | Admin URL of the runner. Only available for administrators. | -| <a id="cirunnerarchitecturename"></a>`architectureName` | [`String`](#string) | Architecture provided by the the runner. | +| <a id="cirunnerarchitecturename"></a>`architectureName` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | | <a id="cirunnercontactedat"></a>`contactedAt` | [`Time`](#time) | Timestamp of last contact from this runner. | | <a id="cirunnercreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of creation of this runner. | | <a id="cirunnercreatedby"></a>`createdBy` | [`UserCore`](#usercore) | User that created this runner. | @@ -13091,11 +13267,10 @@ CI/CD variables for a project. | <a id="cirunnereditadminurl"></a>`editAdminUrl` | [`String`](#string) | Admin form URL of the runner. Only available for administrators. | | <a id="cirunnerephemeralauthenticationtoken"></a>`ephemeralAuthenticationToken` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.9. This feature is an Experiment. It can be changed or removed at any time. Ephemeral authentication token used for runner manager registration. Only available for the creator of the runner for a limited time during registration. | | <a id="cirunnerephemeralregisterurl"></a>`ephemeralRegisterUrl` **{warning-solid}** | [`String`](#string) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. URL of the registration page of the runner manager. Only available for the creator of the runner for a limited time during registration. | -| <a id="cirunnerexecutorname"></a>`executorName` | [`String`](#string) | Executor last advertised by the runner. | +| <a id="cirunnerexecutorname"></a>`executorName` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | | <a id="cirunnergroups"></a>`groups` | [`GroupConnection`](#groupconnection) | Groups the runner is associated with. For group runners only. (see [Connections](#connections)) | | <a id="cirunnerid"></a>`id` | [`CiRunnerID!`](#cirunnerid) | ID of the runner. | -| <a id="cirunneripaddress"></a>`ipAddress` | [`String`](#string) | IP address of the runner. | -| <a id="cirunnerjobcount"></a>`jobCount` | [`Int`](#int) | Number of jobs processed by the runner (limited to 1000, plus one to indicate that more items exist). | +| <a id="cirunneripaddress"></a>`ipAddress` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | | <a id="cirunnerjobexecutionstatus"></a>`jobExecutionStatus` **{warning-solid}** | [`CiRunnerJobExecutionStatus`](#cirunnerjobexecutionstatus) | **Introduced** in 15.7. This feature is an Experiment. It can be changed or removed at any time. Job execution status of the runner. | | <a id="cirunnerlocked"></a>`locked` | [`Boolean`](#boolean) | Indicates the runner is locked. | | <a id="cirunnermaintenancenote"></a>`maintenanceNote` | [`String`](#string) | Runner's maintenance notes. | @@ -13104,12 +13279,12 @@ CI/CD variables for a project. | <a id="cirunnermaximumtimeout"></a>`maximumTimeout` | [`Int`](#int) | Maximum timeout (in seconds) for jobs processed by the runner. | | <a id="cirunnerownerproject"></a>`ownerProject` | [`Project`](#project) | Project that owns the runner. For project runners only. | | <a id="cirunnerpaused"></a>`paused` | [`Boolean!`](#boolean) | Indicates the runner is paused and not available to run jobs. | -| <a id="cirunnerplatformname"></a>`platformName` | [`String`](#string) | Platform provided by the runner. | +| <a id="cirunnerplatformname"></a>`platformName` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | | <a id="cirunnerprivateprojectsminutescostfactor"></a>`privateProjectsMinutesCostFactor` | [`Float`](#float) | Private projects' "compute cost factor" associated with the runner (GitLab.com only). | | <a id="cirunnerprojectcount"></a>`projectCount` | [`Int`](#int) | Number of projects that the runner is associated with. | | <a id="cirunnerpublicprojectsminutescostfactor"></a>`publicProjectsMinutesCostFactor` | [`Float`](#float) | Public projects' "compute cost factor" associated with the runner (GitLab.com only). | | <a id="cirunnerregisteradminurl"></a>`registerAdminUrl` | [`String`](#string) | URL of the temporary registration page of the runner. Only available before the runner is registered. Only available for administrators. | -| <a id="cirunnerrevision"></a>`revision` | [`String`](#string) | Revision of the runner. | +| <a id="cirunnerrevision"></a>`revision` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | | <a id="cirunnerrununtagged"></a>`runUntagged` | [`Boolean!`](#boolean) | Indicates the runner is able to run untagged jobs. | | <a id="cirunnerrunnertype"></a>`runnerType` | [`CiRunnerType!`](#cirunnertype) | Type of the runner. | | <a id="cirunnershortsha"></a>`shortSha` | [`String`](#string) | First eight characters of the runner's token used to authenticate new job requests. Used as the runner's unique ID. | @@ -13117,10 +13292,22 @@ CI/CD variables for a project. | <a id="cirunnertokenexpiresat"></a>`tokenExpiresAt` | [`Time`](#time) | Runner token expiration time. | | <a id="cirunnerupgradestatus"></a>`upgradeStatus` **{warning-solid}** | [`CiRunnerUpgradeStatus`](#cirunnerupgradestatus) | **Introduced** in 14.10. This feature is an Experiment. It can be changed or removed at any time. Availability of upgrades for the runner. | | <a id="cirunneruserpermissions"></a>`userPermissions` | [`RunnerPermissions!`](#runnerpermissions) | Permissions for the current user on the resource. | -| <a id="cirunnerversion"></a>`version` | [`String`](#string) | Version of the runner. | +| <a id="cirunnerversion"></a>`version` **{warning-solid}** | [`String`](#string) | **Deprecated** in 16.2. Use field in `manager` object instead. | #### Fields with arguments +##### `CiRunner.jobCount` + +Number of jobs processed by the runner (limited to 1000, plus one to indicate that more items exist). + +Returns [`Int`](#int). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="cirunnerjobcountstatuses"></a>`statuses` **{warning-solid}** | [`[CiJobStatus!]`](#cijobstatus) | **Introduced** in 16.2. This feature is an Experiment. It can be changed or removed at any time. Filter jobs by status. | + ##### `CiRunner.jobs` Jobs assigned to the runner. This field can only be resolved for one runner in any single request. @@ -13509,6 +13696,21 @@ Represents a ComplianceFramework associated with a Project. | <a id="complianceframeworkname"></a>`name` | [`String!`](#string) | Name of the compliance framework. | | <a id="complianceframeworkpipelineconfigurationfullpath"></a>`pipelineConfigurationFullPath` | [`String`](#string) | Full path of the compliance pipeline configuration stored in a project repository, such as `.gitlab/.compliance-gitlab-ci.yml@compliance/hipaa` **(ULTIMATE)**. | +### `ComplianceStandardsAdherence` + +Compliance standards adherence for a project. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="compliancestandardsadherencecheckname"></a>`checkName` | [`ComplianceStandardsAdherenceCheckName!`](#compliancestandardsadherencecheckname) | Name of the check for the compliance standard. | +| <a id="compliancestandardsadherenceid"></a>`id` | [`ID!`](#id) | Compliance standards adherence ID. | +| <a id="compliancestandardsadherenceproject"></a>`project` | [`Project!`](#project) | Project adhering to the compliance standard. | +| <a id="compliancestandardsadherencestandard"></a>`standard` | [`ComplianceStandardsAdherenceStandard!`](#compliancestandardsadherencestandard) | Name of the compliance standard. | +| <a id="compliancestandardsadherencestatus"></a>`status` | [`ComplianceStandardsAdherenceStatus!`](#compliancestandardsadherencestatus) | Status of the compliance standards adherence. | +| <a id="compliancestandardsadherenceupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp when the adherence was updated. | + ### `ComplianceViolation` Compliance violation associated with a merged merge request. @@ -14696,6 +14898,7 @@ Describes where code is deployed for a project. | <a id="environmentenvironmenttype"></a>`environmentType` | [`String`](#string) | Folder name of the environment. | | <a id="environmentexternalurl"></a>`externalUrl` | [`String`](#string) | External URL of the environment. | | <a id="environmentid"></a>`id` | [`ID!`](#id) | ID of the environment. | +| <a id="environmentkubernetesnamespace"></a>`kubernetesNamespace` | [`String`](#string) | Kubernetes namespace of the environment. | | <a id="environmentlatestopenedmostseverealert"></a>`latestOpenedMostSevereAlert` | [`AlertManagementAlert`](#alertmanagementalert) | Most severe open alert for the environment. If multiple alerts have equal severity, the most recent is returned. | | <a id="environmentname"></a>`name` | [`String!`](#string) | Human-readable name of the environment. | | <a id="environmentpath"></a>`path` | [`String!`](#string) | Path to the environment. | @@ -14737,22 +14940,6 @@ Returns [`Deployment`](#deployment). | ---- | ---- | ----------- | | <a id="environmentlastdeploymentstatus"></a>`status` | [`DeploymentStatus!`](#deploymentstatus) | Status of the Deployment. | -##### `Environment.metricsDashboard` - -Metrics dashboard schema for the environment. - -WARNING: -**Deprecated** in 16.0. -Returns no data. Underlying feature was removed in 16.0. - -Returns [`MetricsDashboard`](#metricsdashboard). - -###### Arguments - -| Name | Type | Description | -| ---- | ---- | ----------- | -| <a id="environmentmetricsdashboardpath"></a>`path` | [`String!`](#string) | Path to a file which defines a metrics dashboard eg: `"config/prometheus/common_metrics.yml"`. | - ### `EnvironmentPermissions` #### Fields @@ -15251,6 +15438,15 @@ Representing an event. | <a id="eventid"></a>`id` | [`ID!`](#id) | ID of the event. | | <a id="eventupdatedat"></a>`updatedAt` | [`Time!`](#time) | When this event was updated. | +### `ExplainVulnerabilityPrompt` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="explainvulnerabilitypromptpromptwithcode"></a>`promptWithCode` | [`String`](#string) | AI text prompt generated using the vulnerability's information, including the vulnerable code. | +| <a id="explainvulnerabilitypromptpromptwithoutcode"></a>`promptWithoutCode` | [`String`](#string) | AI text prompt generated using the vulnerability's information, excluding the vulnerable code. | + ### `ExternalAuditEventDestination` Represents an external resource to send audit events to. @@ -15264,6 +15460,7 @@ Represents an external resource to send audit events to. | <a id="externalauditeventdestinationgroup"></a>`group` | [`Group!`](#group) | Group the destination belongs to. | | <a id="externalauditeventdestinationheaders"></a>`headers` | [`AuditEventStreamingHeaderConnection!`](#auditeventstreamingheaderconnection) | List of additional HTTP headers sent with each event. (see [Connections](#connections)) | | <a id="externalauditeventdestinationid"></a>`id` | [`ID!`](#id) | ID of the destination. | +| <a id="externalauditeventdestinationname"></a>`name` | [`String!`](#string) | Name of the external destination to send audit events to. | | <a id="externalauditeventdestinationverificationtoken"></a>`verificationToken` | [`String!`](#string) | Verification token to validate source of event. | ### `ExternalIssue` @@ -15611,6 +15808,29 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="geonodepipelineartifactregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. | | <a id="geonodepipelineartifactregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. | +##### `GeoNode.projectRepositoryRegistries` + +Find Project registries on this Geo node. Ignored if `geo_project_repository_replication` feature flag is disabled. + +WARNING: +**Introduced** in 16.2. +This feature is an Experiment. It can be changed or removed at any time. + +Returns [`ProjectRepositoryRegistryConnection`](#projectrepositoryregistryconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="geonodeprojectrepositoryregistriesids"></a>`ids` | [`[ID!]`](#id) | Filters registries by their ID. | +| <a id="geonodeprojectrepositoryregistrieskeyword"></a>`keyword` | [`String`](#string) | Filters registries by their attributes using a keyword. | +| <a id="geonodeprojectrepositoryregistriesreplicationstate"></a>`replicationState` | [`ReplicationStateEnum`](#replicationstateenum) | Filters registries by their replication state. | +| <a id="geonodeprojectrepositoryregistriesverificationstate"></a>`verificationState` | [`VerificationStateEnum`](#verificationstateenum) | Filters registries by their verification state. | + ##### `GeoNode.projectWikiRepositoryRegistries` Find Project Wiki Repository registries on this Geo node. Ignored if `geo_project_wiki_repository_replication` feature flag is disabled. @@ -15736,13 +15956,14 @@ GPG signature for a signed commit. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="groupactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. | +| <a id="groupactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. This limit only applies to namespaces under Project limit enforcement. | +| <a id="groupactualsizelimit"></a>`actualSizeLimit` | [`Float`](#float) | Actual storage size limit for the namespace in bytes. This limit is agnostic of enforcement type. | | <a id="groupadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | | <a id="groupallowstalerunnerpruning"></a>`allowStaleRunnerPruning` | [`Boolean!`](#boolean) | Indicates whether to regularly prune stale group runners. Defaults to false. | | <a id="groupautodevopsenabled"></a>`autoDevopsEnabled` | [`Boolean`](#boolean) | Indicates whether Auto DevOps is enabled for all projects within this group. | | <a id="groupavatarurl"></a>`avatarUrl` | [`String`](#string) | Avatar URL of the group. | | <a id="groupcontainerrepositoriescount"></a>`containerRepositoriesCount` | [`Int!`](#int) | Number of container repositories in the group. | -| <a id="groupcontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | +| <a id="groupcontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. This only applies to namespaces under Project limit enforcement. | | <a id="groupcrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | | <a id="groupcustomemoji"></a>`customEmoji` **{warning-solid}** | [`CustomEmojiConnection`](#customemojiconnection) | **Introduced** in 13.6. This feature is an Experiment. It can be changed or removed at any time. Custom emoji within this namespace. | | <a id="groupdependencyproxyblobcount"></a>`dependencyProxyBlobCount` | [`Int!`](#int) | Number of dependency proxy blobs cached in the group. | @@ -15778,19 +15999,19 @@ GPG signature for a signed commit. | <a id="grouppath"></a>`path` | [`String!`](#string) | Path of the namespace. | | <a id="groupprojectcreationlevel"></a>`projectCreationLevel` | [`String`](#string) | Permission level required to create projects in the group. | | <a id="grouprecentissueboards"></a>`recentIssueBoards` | [`BoardConnection`](#boardconnection) | List of recently visited boards of the group. Maximum size is 4. (see [Connections](#connections)) | -| <a id="grouprepositorysizeexcessprojectcount"></a>`repositorySizeExcessProjectCount` | [`Int!`](#int) | Number of projects in the root namespace where the repository size exceeds the limit. | +| <a id="grouprepositorysizeexcessprojectcount"></a>`repositorySizeExcessProjectCount` | [`Int!`](#int) | Number of projects in the root namespace where the repository size exceeds the limit. This only applies to namespaces under Project limit enforcement. | | <a id="grouprequestaccessenabled"></a>`requestAccessEnabled` | [`Boolean`](#boolean) | Indicates if users can request access to namespace. | | <a id="grouprequiretwofactorauthentication"></a>`requireTwoFactorAuthentication` | [`Boolean`](#boolean) | Indicates if all users in this group are required to set up two-factor authentication. | | <a id="grouprootstoragestatistics"></a>`rootStorageStatistics` | [`RootStorageStatistics`](#rootstoragestatistics) | Aggregated storage statistics of the namespace. Only available for root namespaces. | | <a id="groupsharewithgrouplock"></a>`shareWithGroupLock` | [`Boolean`](#boolean) | Indicates if sharing a project with another group within this group is prevented. | | <a id="groupsharedrunnerssetting"></a>`sharedRunnersSetting` | [`SharedRunnersSetting`](#sharedrunnerssetting) | Shared runners availability for the namespace and its descendants. | | <a id="groupstats"></a>`stats` | [`GroupStats`](#groupstats) | Group statistics. | -| <a id="groupstoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Total storage limit of the root namespace in bytes. | +| <a id="groupstoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Storage limit included in the root namespace plan in bytes. This limit only applies to namespaces under Namespace limit enforcement. | | <a id="groupsubgroupcreationlevel"></a>`subgroupCreationLevel` | [`String`](#string) | Permission level required to create subgroups within the group. | | <a id="grouptemporarystorageincreaseendson"></a>`temporaryStorageIncreaseEndsOn` | [`Time`](#time) | Date until the temporary storage increase is active. | | <a id="grouptimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Timelog categories for the namespace. | | <a id="grouptotalrepositorysize"></a>`totalRepositorySize` | [`Float`](#float) | Total repository size of all projects in the root namespace in bytes. | -| <a id="grouptotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. | +| <a id="grouptotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. This only applies to namespaces under Project limit enforcement. | | <a id="grouptwofactorgraceperiod"></a>`twoFactorGracePeriod` | [`Int`](#int) | Time before two-factor authentication is enforced. | | <a id="groupuserpermissions"></a>`userPermissions` | [`GroupPermissions!`](#grouppermissions) | Permissions for the current user on the resource. | | <a id="groupvisibility"></a>`visibility` | [`String`](#string) | Visibility of the namespace. | @@ -15819,6 +16040,18 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="groupachievementsids"></a>`ids` | [`[AchievementsAchievementID!]`](#achievementsachievementid) | Filter achievements by IDs. | +##### `Group.addOnPurchase` + +AddOnPurchase associated with the namespace. + +Returns [`AddOnPurchase`](#addonpurchase). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupaddonpurchaseaddonname"></a>`addOnName` | [`String!`](#string) | AddOn name. | + ##### `Group.billableMembersCount` Number of billable users in the group. @@ -16171,6 +16404,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuesassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. | | <a id="groupissuesassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. | | <a id="groupissuesassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | +| <a id="groupissuesassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="groupissuesauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | | <a id="groupissuesclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. | | <a id="groupissuesclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. | @@ -16180,6 +16414,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="groupissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="groupissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="groupissuesepicwildcardid"></a>`epicWildcardId` | [`EpicWildcardId`](#epicwildcardid) | Filter by epic ID wildcard. Incompatible with epicId. | | <a id="groupissueshealthstatus"></a>`healthStatus` **{warning-solid}** | [`HealthStatus`](#healthstatus) | **Deprecated** in 15.4. Use `healthStatusFilter`. | | <a id="groupissueshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. | | <a id="groupissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | @@ -16188,7 +16423,9 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuesincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Return issues from archived projects. | | <a id="groupissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | | <a id="groupissuesincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include issues belonging to subgroups. | +| <a id="groupissuesiterationcadenceid"></a>`iterationCadenceId` | [`[IterationsCadenceID!]`](#iterationscadenceid) | Filter by a list of iteration cadence IDs. | | <a id="groupissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | +| <a id="groupissuesiterationtitle"></a>`iterationTitle` | [`String`](#string) | Filter by iteration title. | | <a id="groupissuesiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | | <a id="groupissueslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="groupissuesmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | @@ -16203,6 +16440,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. | | <a id="groupissuesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. | | <a id="groupissuesweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. | +| <a id="groupissuesweightwildcardid"></a>`weightWildcardId` | [`WeightWildcardId`](#weightwildcardid) | Filter by weight ID wildcard. Incompatible with weight. | ##### `Group.iterationCadences` @@ -16409,6 +16647,22 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="grouppackagessort"></a>`sort` | [`PackageGroupSort`](#packagegroupsort) | Sort packages by this criteria. | | <a id="grouppackagesstatus"></a>`status` | [`PackageStatus`](#packagestatus) | Filter a package by status. | +##### `Group.projectComplianceStandardsAdherence` + +Compliance standards adherence for the projects in a group and its subgroups. + +Returns [`ComplianceStandardsAdherenceConnection`](#compliancestandardsadherenceconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupprojectcompliancestandardsadherencefilters"></a>`filters` | [`ComplianceStandardsAdherenceInput`](#compliancestandardsadherenceinput) | Filters applied when retrieving compliance standards adherence. | + ##### `Group.projects` Projects within this namespace. @@ -16429,6 +16683,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupprojectsids"></a>`ids` | [`[ID!]`](#id) | Filter projects by IDs. | | <a id="groupprojectsincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include also subgroup projects. | | <a id="groupprojectsnotaimedfordeletion"></a>`notAimedForDeletion` | [`Boolean`](#boolean) | Include projects that are not aimed for deletion. | +| <a id="groupprojectssbomcomponentid"></a>`sbomComponentId` | [`ID`](#id) | Return only the projects related to the specified SBOM component. | | <a id="groupprojectssearch"></a>`search` | [`String`](#string) | Search project with most similar names or paths. | | <a id="groupprojectssort"></a>`sort` | [`NamespaceProjectSort`](#namespaceprojectsort) | Sort projects by this criteria. | | <a id="groupprojectswithissuesenabled"></a>`withIssuesEnabled` | [`Boolean`](#boolean) | Return only projects with issues enabled. | @@ -16830,6 +17085,16 @@ Helm file metadata. | <a id="helmfilemetadatametadata"></a>`metadata` | [`PackageHelmMetadataType!`](#packagehelmmetadatatype) | Metadata of the Helm chart. | | <a id="helmfilemetadataupdatedat"></a>`updatedAt` | [`Time!`](#time) | Date of most recent update. | +### `Ide` + +IDE settings and feature flags. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="idecodesuggestionsenabled"></a>`codeSuggestionsEnabled` | [`Boolean!`](#boolean) | Indicates whether AI assisted code suggestions are enabled. | + ### `IncidentManagementOncallRotation` Describes an incident management on-call rotation. @@ -16934,8 +17199,10 @@ Represents an external resource to send instance audit events to. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="instanceexternalauditeventdestinationdestinationurl"></a>`destinationUrl` | [`String!`](#string) | External destination to send audit events to. | +| <a id="instanceexternalauditeventdestinationeventtypefilters"></a>`eventTypeFilters` | [`[String!]!`](#string) | List of event type filters added for streaming. | | <a id="instanceexternalauditeventdestinationheaders"></a>`headers` | [`AuditEventsStreamingInstanceHeaderConnection!`](#auditeventsstreaminginstanceheaderconnection) | List of additional HTTP headers sent with each event. (see [Connections](#connections)) | | <a id="instanceexternalauditeventdestinationid"></a>`id` | [`ID!`](#id) | ID of the destination. | +| <a id="instanceexternalauditeventdestinationname"></a>`name` | [`String!`](#string) | Name of the external destination to send audit events to. | | <a id="instanceexternalauditeventdestinationverificationtoken"></a>`verificationToken` | [`String!`](#string) | Verification token to validate source of event. | ### `InstanceSecurityDashboard` @@ -17518,11 +17785,12 @@ Defines which user roles, users, or groups can merge into a protected branch. | <a id="mergerequestmergecommitsha"></a>`mergeCommitSha` | [`String`](#string) | SHA of the merge request commit (set once merged). | | <a id="mergerequestmergeerror"></a>`mergeError` | [`String`](#string) | Error message due to a merge error. | | <a id="mergerequestmergeongoing"></a>`mergeOngoing` | [`Boolean!`](#boolean) | Indicates if a merge is currently occurring. | +| <a id="mergerequestmergerequestdiffs"></a>`mergeRequestDiffs` **{warning-solid}** | [`MergeRequestDiffConnection`](#mergerequestdiffconnection) | **Introduced** in 16.2. This feature is an Experiment. It can be changed or removed at any time. Diff versions of a merge request. | | <a id="mergerequestmergestatus"></a>`mergeStatus` **{warning-solid}** | [`String`](#string) | **Deprecated** in 14.0. This was renamed. Use: [`MergeRequest.mergeStatusEnum`](#mergerequestmergestatusenum). | | <a id="mergerequestmergestatusenum"></a>`mergeStatusEnum` | [`MergeStatus`](#mergestatus) | Merge status of the merge request. | | <a id="mergerequestmergetrainscount"></a>`mergeTrainsCount` | [`Int`](#int) | Number of merge requests in the merge train. | -| <a id="mergerequestmergeuser"></a>`mergeUser` | [`UserCore`](#usercore) | User who merged this merge request or set it to merge when pipeline succeeds. | -| <a id="mergerequestmergewhenpipelinesucceeds"></a>`mergeWhenPipelineSucceeds` | [`Boolean`](#boolean) | Indicates if the merge has been set to be merged when its pipeline succeeds (MWPS). | +| <a id="mergerequestmergeuser"></a>`mergeUser` | [`UserCore`](#usercore) | User who merged this merge request or set it to auto-merge. | +| <a id="mergerequestmergewhenpipelinesucceeds"></a>`mergeWhenPipelineSucceeds` | [`Boolean`](#boolean) | Indicates if the merge has been set to auto-merge. | | <a id="mergerequestmergeable"></a>`mergeable` | [`Boolean!`](#boolean) | Indicates if the merge request is mergeable. | | <a id="mergerequestmergeablediscussionsstate"></a>`mergeableDiscussionsState` | [`Boolean`](#boolean) | Indicates if all discussions in the merge request have been resolved, allowing the merge request to be merged. | | <a id="mergerequestmergedat"></a>`mergedAt` | [`Time`](#time) | Timestamp of when the merge request was merged, null if not merged. | @@ -17681,6 +17949,7 @@ A user assigned to a merge request. | <a id="mergerequestassigneegroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestassigneegroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestassigneeid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestassigneeide"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="mergerequestassigneejobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="mergerequestassigneelinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="mergerequestassigneelocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -17692,6 +17961,7 @@ A user assigned to a merge request. | <a id="mergerequestassigneepreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="mergerequestassigneeprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestassigneeprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestassigneepronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="mergerequestassigneepublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="mergerequestassigneesavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestassigneestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -17724,6 +17994,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestassigneeassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestassigneeassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestassigneeassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestassigneeassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestassigneeassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestassigneeassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestassigneeassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -17759,6 +18030,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestassigneeauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestassigneeauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestassigneeauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestassigneeauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestassigneeauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestassigneeauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestassigneeauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -17812,6 +18084,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestassigneereviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestassigneereviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestassigneereviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestassigneereviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestassigneereviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestassigneereviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestassigneereviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -17932,6 +18205,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mergerequestassigneeworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="mergerequestassigneeworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="mergerequestassigneeworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | ### `MergeRequestAuthor` @@ -17954,6 +18229,7 @@ The author of the merge request. | <a id="mergerequestauthorgroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestauthorgroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestauthorid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestauthoride"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="mergerequestauthorjobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="mergerequestauthorlinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="mergerequestauthorlocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -17965,6 +18241,7 @@ The author of the merge request. | <a id="mergerequestauthorpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="mergerequestauthorprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestauthorprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestauthorpronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="mergerequestauthorpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="mergerequestauthorsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestauthorstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -17997,6 +18274,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestauthorassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestauthorassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestauthorassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestauthorassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestauthorassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestauthorassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18032,6 +18310,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestauthorauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestauthorauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestauthorauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestauthorauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestauthorauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestauthorauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18085,6 +18364,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestauthorreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestauthorreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestauthorreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorreviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestauthorreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestauthorreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestauthorreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18205,6 +18485,21 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mergerequestauthorworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="mergerequestauthorworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="mergerequestauthorworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | + +### `MergeRequestDiff` + +A diff version of a merge request. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestdiffcreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the diff was created. | +| <a id="mergerequestdiffdiffllmsummary"></a>`diffLlmSummary` | [`MergeRequestDiffLlmSummary`](#mergerequestdiffllmsummary) | Diff summary generated by AI. | +| <a id="mergerequestdiffreviewllmsummaries"></a>`reviewLlmSummaries` | [`MergeRequestReviewLlmSummaryConnection`](#mergerequestreviewllmsummaryconnection) | Review summaries generated by AI. (see [Connections](#connections)) | +| <a id="mergerequestdiffupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the diff was updated. | ### `MergeRequestDiffLlmSummary` @@ -18261,6 +18556,7 @@ A user participating in a merge request. | <a id="mergerequestparticipantgroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestparticipantgroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestparticipantid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestparticipantide"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="mergerequestparticipantjobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="mergerequestparticipantlinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="mergerequestparticipantlocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -18272,6 +18568,7 @@ A user participating in a merge request. | <a id="mergerequestparticipantpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="mergerequestparticipantprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestparticipantprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestparticipantpronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="mergerequestparticipantpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="mergerequestparticipantsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestparticipantstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -18304,6 +18601,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestparticipantassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestparticipantassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestparticipantassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestparticipantassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestparticipantassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestparticipantassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18339,6 +18637,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestparticipantauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestparticipantauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestparticipantauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestparticipantauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestparticipantauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestparticipantauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18392,6 +18691,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestparticipantreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestparticipantreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestparticipantreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestparticipantreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestparticipantreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestparticipantreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18512,6 +18812,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mergerequestparticipantworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="mergerequestparticipantworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="mergerequestparticipantworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | ### `MergeRequestPermissions` @@ -18532,6 +18834,22 @@ Check permissions for the current user on a merge request. | <a id="mergerequestpermissionsrevertoncurrentmergerequest"></a>`revertOnCurrentMergeRequest` | [`Boolean!`](#boolean) | Indicates the user can perform `revert_on_current_merge_request` on this resource. | | <a id="mergerequestpermissionsupdatemergerequest"></a>`updateMergeRequest` | [`Boolean!`](#boolean) | Indicates the user can perform `update_merge_request` on this resource. | +### `MergeRequestReviewLlmSummary` + +A review summary generated by AI. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestreviewllmsummarycontent"></a>`content` | [`String!`](#string) | Content of the review summary. | +| <a id="mergerequestreviewllmsummarycreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the review summary was created. | +| <a id="mergerequestreviewllmsummarymergerequestdiffid"></a>`mergeRequestDiffId` | [`ID!`](#id) | ID of the Merge Request diff associated with the review summary. | +| <a id="mergerequestreviewllmsummaryprovider"></a>`provider` | [`String!`](#string) | AI provider that generated the summary. | +| <a id="mergerequestreviewllmsummaryreviewer"></a>`reviewer` | [`UserCore`](#usercore) | User who authored the review associated with the review summary. | +| <a id="mergerequestreviewllmsummaryupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of when the review summary was updated. | +| <a id="mergerequestreviewllmsummaryuser"></a>`user` | [`UserCore`](#usercore) | User associated with the review summary. | + ### `MergeRequestReviewer` A user assigned to a merge request as a reviewer. @@ -18553,6 +18871,7 @@ A user assigned to a merge request as a reviewer. | <a id="mergerequestreviewergroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestreviewergroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestreviewerid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestrevieweride"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="mergerequestreviewerjobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="mergerequestreviewerlinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="mergerequestreviewerlocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -18564,6 +18883,7 @@ A user assigned to a merge request as a reviewer. | <a id="mergerequestreviewerpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="mergerequestreviewerprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestreviewerprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestreviewerpronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="mergerequestreviewerpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="mergerequestreviewersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="mergerequestreviewerstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -18596,6 +18916,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestreviewerassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestreviewerassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestreviewerassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestreviewerassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestreviewerassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestreviewerassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestreviewerassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18631,6 +18952,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestreviewerauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestreviewerauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestreviewerauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestreviewerauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestreviewerauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestreviewerauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestreviewerauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18684,6 +19006,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestreviewerreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="mergerequestreviewerreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="mergerequestreviewerreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestreviewerreviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="mergerequestreviewerreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="mergerequestreviewerreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="mergerequestreviewerreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -18804,6 +19127,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="mergerequestreviewerworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="mergerequestreviewerworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="mergerequestreviewerworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | ### `Metadata` @@ -18830,34 +19155,6 @@ Represents a metric image upload. | <a id="metricimageiid"></a>`iid` | [`ID!`](#id) | Internal ID of the metric upload. | | <a id="metricimageurl"></a>`url` | [`String!`](#string) | URL of the metric source. | -### `MetricsDashboard` - -#### Fields - -| Name | Type | Description | -| ---- | ---- | ----------- | -| <a id="metricsdashboardpath"></a>`path` | [`String`](#string) | Path to a file with the dashboard definition. | -| <a id="metricsdashboardschemavalidationwarnings"></a>`schemaValidationWarnings` | [`[String!]`](#string) | Dashboard schema validation warnings. | - -#### Fields with arguments - -##### `MetricsDashboard.annotations` - -Annotations added to the dashboard. - -Returns [`MetricsDashboardAnnotationConnection`](#metricsdashboardannotationconnection). - -This field returns a [connection](#connections). It accepts the -four standard [pagination arguments](#connection-pagination-arguments): -`before: String`, `after: String`, `first: Int`, `last: Int`. - -###### Arguments - -| Name | Type | Description | -| ---- | ---- | ----------- | -| <a id="metricsdashboardannotationsfrom"></a>`from` | [`Time!`](#time) | Timestamp marking date and time from which annotations need to be fetched. | -| <a id="metricsdashboardannotationsto"></a>`to` | [`Time`](#time) | Timestamp marking date and time to which annotations need to be fetched. | - ### `MetricsDashboardAnnotation` #### Fields @@ -18926,9 +19223,10 @@ Contains statistics about a milestone. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="namespaceactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. | +| <a id="namespaceactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. This limit only applies to namespaces under Project limit enforcement. | +| <a id="namespaceactualsizelimit"></a>`actualSizeLimit` | [`Float`](#float) | Actual storage size limit for the namespace in bytes. This limit is agnostic of enforcement type. | | <a id="namespaceadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | -| <a id="namespacecontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | +| <a id="namespacecontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. This only applies to namespaces under Project limit enforcement. | | <a id="namespacecrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | | <a id="namespacedescription"></a>`description` | [`String`](#string) | Description of the namespace. | | <a id="namespacedescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | GitLab Flavored Markdown rendering of `description`. | @@ -18940,15 +19238,15 @@ Contains statistics about a milestone. | <a id="namespacename"></a>`name` | [`String!`](#string) | Name of the namespace. | | <a id="namespacepackagesettings"></a>`packageSettings` | [`PackageSettings`](#packagesettings) | Package settings for the namespace. | | <a id="namespacepath"></a>`path` | [`String!`](#string) | Path of the namespace. | -| <a id="namespacerepositorysizeexcessprojectcount"></a>`repositorySizeExcessProjectCount` | [`Int!`](#int) | Number of projects in the root namespace where the repository size exceeds the limit. | +| <a id="namespacerepositorysizeexcessprojectcount"></a>`repositorySizeExcessProjectCount` | [`Int!`](#int) | Number of projects in the root namespace where the repository size exceeds the limit. This only applies to namespaces under Project limit enforcement. | | <a id="namespacerequestaccessenabled"></a>`requestAccessEnabled` | [`Boolean`](#boolean) | Indicates if users can request access to namespace. | | <a id="namespacerootstoragestatistics"></a>`rootStorageStatistics` | [`RootStorageStatistics`](#rootstoragestatistics) | Aggregated storage statistics of the namespace. Only available for root namespaces. | | <a id="namespacesharedrunnerssetting"></a>`sharedRunnersSetting` | [`SharedRunnersSetting`](#sharedrunnerssetting) | Shared runners availability for the namespace and its descendants. | -| <a id="namespacestoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Total storage limit of the root namespace in bytes. | +| <a id="namespacestoragesizelimit"></a>`storageSizeLimit` | [`Float`](#float) | Storage limit included in the root namespace plan in bytes. This limit only applies to namespaces under Namespace limit enforcement. | | <a id="namespacetemporarystorageincreaseendson"></a>`temporaryStorageIncreaseEndsOn` | [`Time`](#time) | Date until the temporary storage increase is active. | | <a id="namespacetimelogcategories"></a>`timelogCategories` **{warning-solid}** | [`TimeTrackingTimelogCategoryConnection`](#timetrackingtimelogcategoryconnection) | **Introduced** in 15.3. This feature is an Experiment. It can be changed or removed at any time. Timelog categories for the namespace. | | <a id="namespacetotalrepositorysize"></a>`totalRepositorySize` | [`Float`](#float) | Total repository size of all projects in the root namespace in bytes. | -| <a id="namespacetotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. | +| <a id="namespacetotalrepositorysizeexcess"></a>`totalRepositorySizeExcess` | [`Float`](#float) | Total excess repository size of all projects in the root namespace in bytes. This only applies to namespaces under Project limit enforcement. | | <a id="namespacevisibility"></a>`visibility` | [`String`](#string) | Visibility of the namespace. | #### Fields with arguments @@ -18973,6 +19271,18 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="namespaceachievementsids"></a>`ids` | [`[AchievementsAchievementID!]`](#achievementsachievementid) | Filter achievements by IDs. | +##### `Namespace.addOnPurchase` + +AddOnPurchase associated with the namespace. + +Returns [`AddOnPurchase`](#addonpurchase). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="namespaceaddonpurchaseaddonname"></a>`addOnName` | [`String!`](#string) | AddOn name. | + ##### `Namespace.complianceFrameworks` Compliance frameworks available to projects in this namespace. @@ -19009,6 +19319,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="namespaceprojectsids"></a>`ids` | [`[ID!]`](#id) | Filter projects by IDs. | | <a id="namespaceprojectsincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include also subgroup projects. | | <a id="namespaceprojectsnotaimedfordeletion"></a>`notAimedForDeletion` | [`Boolean`](#boolean) | Include projects that are not aimed for deletion. | +| <a id="namespaceprojectssbomcomponentid"></a>`sbomComponentId` | [`ID`](#id) | Return only the projects related to the specified SBOM component. | | <a id="namespaceprojectssearch"></a>`search` | [`String`](#string) | Search project with most similar names or paths. | | <a id="namespaceprojectssort"></a>`sort` | [`NamespaceProjectSort`](#namespaceprojectsort) | Sort projects by this criteria. | | <a id="namespaceprojectswithissuesenabled"></a>`withIssuesEnabled` | [`Boolean`](#boolean) | Return only projects with issues enabled. | @@ -19536,7 +19847,7 @@ Represents a file or directory in the project repository that has been locked. | <a id="pipelinecommitpath"></a>`commitPath` | [`String`](#string) | Path to the commit that triggered the pipeline. | | <a id="pipelinecommittedat"></a>`committedAt` | [`Time`](#time) | Timestamp of the pipeline's commit. | | <a id="pipelinecomplete"></a>`complete` | [`Boolean!`](#boolean) | Indicates if a pipeline is complete. | -| <a id="pipelineconfigsource"></a>`configSource` | [`PipelineConfigSourceEnum`](#pipelineconfigsourceenum) | Configuration source of the pipeline (UNKNOWN_SOURCE, REPOSITORY_SOURCE, AUTO_DEVOPS_SOURCE, WEBIDE_SOURCE, REMOTE_SOURCE, EXTERNAL_PROJECT_SOURCE, BRIDGE_SOURCE, PARAMETER_SOURCE, COMPLIANCE_SOURCE). | +| <a id="pipelineconfigsource"></a>`configSource` | [`PipelineConfigSourceEnum`](#pipelineconfigsourceenum) | Configuration source of the pipeline (UNKNOWN_SOURCE, REPOSITORY_SOURCE, AUTO_DEVOPS_SOURCE, WEBIDE_SOURCE, REMOTE_SOURCE, EXTERNAL_PROJECT_SOURCE, BRIDGE_SOURCE, PARAMETER_SOURCE, COMPLIANCE_SOURCE, SECURITY_POLICIES_DEFAULT_SOURCE). | | <a id="pipelinecoverage"></a>`coverage` | [`Float`](#float) | Coverage percentage. | | <a id="pipelinecreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of the pipeline's creation. | | <a id="pipelinedastprofile"></a>`dastProfile` | [`DastProfile`](#dastprofile) | DAST profile associated with the pipeline. | @@ -19846,6 +20157,7 @@ Represents a product analytics dashboard panel. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="productanalyticsdashboardpanelgridattributes"></a>`gridAttributes` | [`JSON`](#json) | Description of the position and size of the panel. | +| <a id="productanalyticsdashboardpanelqueryoverrides"></a>`queryOverrides` | [`JSON`](#json) | Overrides for the visualization query object. | | <a id="productanalyticsdashboardpaneltitle"></a>`title` | [`String!`](#string) | Title of the panel. | | <a id="productanalyticsdashboardpanelvisualization"></a>`visualization` | [`ProductAnalyticsDashboardVisualization!`](#productanalyticsdashboardvisualization) | Visualization of the panel. | @@ -19903,7 +20215,6 @@ Represents a product analytics dashboard visualization. | <a id="projectid"></a>`id` | [`ID!`](#id) | ID of the project. | | <a id="projectimportstatus"></a>`importStatus` | [`String`](#string) | Status of import background job of the project. | | <a id="projectincidentmanagementtimelineeventtags"></a>`incidentManagementTimelineEventTags` | [`[TimelineEventTagType!]`](#timelineeventtagtype) | Timeline event tags for the project. | -| <a id="projectinheritedcivariables"></a>`inheritedCiVariables` | [`InheritedCiVariableConnection`](#inheritedcivariableconnection) | List of CI/CD variables the project inherited from its parent group and ancestors. (see [Connections](#connections)) | | <a id="projectiscatalogresource"></a>`isCatalogResource` **{warning-solid}** | [`Boolean`](#boolean) | **Introduced** in 15.11. This feature is an Experiment. It can be changed or removed at any time. Indicates if a project is a catalog resource. | | <a id="projectissuesenabled"></a>`issuesEnabled` | [`Boolean`](#boolean) | Indicates if Issues are enabled for the current user. | | <a id="projectjiraimportstatus"></a>`jiraImportStatus` | [`String`](#string) | Status of Jira import background job of the project. | @@ -20459,6 +20770,22 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="projectincidentmanagementtimelineeventsincidentid"></a>`incidentId` | [`IssueID!`](#issueid) | ID of the incident. | +##### `Project.inheritedCiVariables` + +List of CI/CD variables the project inherited from its parent group and ancestors. + +Returns [`InheritedCiVariableConnection`](#inheritedcivariableconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectinheritedcivariablessort"></a>`sort` | [`CiGroupVariablesSort`](#cigroupvariablessort) | Sort variables by the criteria. | + ##### `Project.issue` A single issue of the project. @@ -20472,6 +20799,7 @@ Returns [`Issue`](#issue). | <a id="projectissueassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. | | <a id="projectissueassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. | | <a id="projectissueassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | +| <a id="projectissueassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="projectissueauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | | <a id="projectissueclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. | | <a id="projectissueclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. | @@ -20481,13 +20809,16 @@ Returns [`Issue`](#issue). | <a id="projectissuecrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="projectissuecrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissueepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="projectissueepicwildcardid"></a>`epicWildcardId` | [`EpicWildcardId`](#epicwildcardid) | Filter by epic ID wildcard. Incompatible with epicId. | | <a id="projectissuehealthstatus"></a>`healthStatus` **{warning-solid}** | [`HealthStatus`](#healthstatus) | **Deprecated** in 15.4. Use `healthStatusFilter`. | | <a id="projectissuehealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. | | <a id="projectissueiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissueiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | | <a id="projectissuein"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissueincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | +| <a id="projectissueiterationcadenceid"></a>`iterationCadenceId` | [`[IterationsCadenceID!]`](#iterationscadenceid) | Filter by a list of iteration cadence IDs. | | <a id="projectissueiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | +| <a id="projectissueiterationtitle"></a>`iterationTitle` | [`String`](#string) | Filter by iteration title. | | <a id="projectissueiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | | <a id="projectissuelabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="projectissuemilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | @@ -20504,6 +20835,7 @@ Returns [`Issue`](#issue). | <a id="projectissueupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. | | <a id="projectissueupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. | | <a id="projectissueweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. | +| <a id="projectissueweightwildcardid"></a>`weightWildcardId` | [`WeightWildcardId`](#weightwildcardid) | Filter by weight ID wildcard. Incompatible with weight. | ##### `Project.issueStatusCounts` @@ -20518,6 +20850,7 @@ Returns [`IssueStatusCountsType`](#issuestatuscountstype). | <a id="projectissuestatuscountsassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. | | <a id="projectissuestatuscountsassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. | | <a id="projectissuestatuscountsassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | +| <a id="projectissuestatuscountsassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="projectissuestatuscountsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | | <a id="projectissuestatuscountsclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. | | <a id="projectissuestatuscountsclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. | @@ -20527,12 +20860,15 @@ Returns [`IssueStatusCountsType`](#issuestatuscountstype). | <a id="projectissuestatuscountscrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="projectissuestatuscountscrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissuestatuscountsepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="projectissuestatuscountsepicwildcardid"></a>`epicWildcardId` | [`EpicWildcardId`](#epicwildcardid) | Filter by epic ID wildcard. Incompatible with epicId. | | <a id="projectissuestatuscountshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. | | <a id="projectissuestatuscountsiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissuestatuscountsiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | | <a id="projectissuestatuscountsin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissuestatuscountsincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | +| <a id="projectissuestatuscountsiterationcadenceid"></a>`iterationCadenceId` | [`[IterationsCadenceID!]`](#iterationscadenceid) | Filter by a list of iteration cadence IDs. | | <a id="projectissuestatuscountsiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | +| <a id="projectissuestatuscountsiterationtitle"></a>`iterationTitle` | [`String`](#string) | Filter by iteration title. | | <a id="projectissuestatuscountsiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | | <a id="projectissuestatuscountslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="projectissuestatuscountsmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | @@ -20547,6 +20883,7 @@ Returns [`IssueStatusCountsType`](#issuestatuscountstype). | <a id="projectissuestatuscountsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. | | <a id="projectissuestatuscountsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. | | <a id="projectissuestatuscountsweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. | +| <a id="projectissuestatuscountsweightwildcardid"></a>`weightWildcardId` | [`WeightWildcardId`](#weightwildcardid) | Filter by weight ID wildcard. Incompatible with weight. | ##### `Project.issues` @@ -20565,6 +20902,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectissuesassigneeid"></a>`assigneeId` | [`String`](#string) | ID of a user assigned to the issues. Wildcard values "NONE" and "ANY" are supported. | | <a id="projectissuesassigneeusername"></a>`assigneeUsername` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.11. Use `assigneeUsernames`. | | <a id="projectissuesassigneeusernames"></a>`assigneeUsernames` | [`[String!]`](#string) | Usernames of users assigned to the issue. | +| <a id="projectissuesassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="projectissuesauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author of the issue. | | <a id="projectissuesclosedafter"></a>`closedAfter` | [`Time`](#time) | Issues closed after this date. | | <a id="projectissuesclosedbefore"></a>`closedBefore` | [`Time`](#time) | Issues closed before this date. | @@ -20574,13 +20912,16 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectissuescrmcontactid"></a>`crmContactId` | [`String`](#string) | ID of a contact assigned to the issues. | | <a id="projectissuescrmorganizationid"></a>`crmOrganizationId` | [`String`](#string) | ID of an organization assigned to the issues. | | <a id="projectissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | +| <a id="projectissuesepicwildcardid"></a>`epicWildcardId` | [`EpicWildcardId`](#epicwildcardid) | Filter by epic ID wildcard. Incompatible with epicId. | | <a id="projectissueshealthstatus"></a>`healthStatus` **{warning-solid}** | [`HealthStatus`](#healthstatus) | **Deprecated** in 15.4. Use `healthStatusFilter`. | | <a id="projectissueshealthstatusfilter"></a>`healthStatusFilter` | [`HealthStatusFilter`](#healthstatusfilter) | Health status of the issue, "none" and "any" values are supported. | | <a id="projectissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="projectissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | | <a id="projectissuesin"></a>`in` | [`[IssuableSearchableField!]`](#issuablesearchablefield) | Specify the fields to perform the search in. Defaults to `[TITLE, DESCRIPTION]`. Requires the `search` argument.'. | | <a id="projectissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | +| <a id="projectissuesiterationcadenceid"></a>`iterationCadenceId` | [`[IterationsCadenceID!]`](#iterationscadenceid) | Filter by a list of iteration cadence IDs. | | <a id="projectissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | +| <a id="projectissuesiterationtitle"></a>`iterationTitle` | [`String`](#string) | Filter by iteration title. | | <a id="projectissuesiterationwildcardid"></a>`iterationWildcardId` | [`IterationWildcardId`](#iterationwildcardid) | Filter by iteration ID wildcard. | | <a id="projectissueslabelname"></a>`labelName` | [`[String]`](#string) | Labels applied to this issue. | | <a id="projectissuesmilestonetitle"></a>`milestoneTitle` | [`[String]`](#string) | Milestone applied to this issue. | @@ -20597,6 +20938,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectissuesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Issues updated after this date. | | <a id="projectissuesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Issues updated before this date. | | <a id="projectissuesweight"></a>`weight` | [`String`](#string) | Weight applied to the issue, "none" and "any" values are supported. | +| <a id="projectissuesweightwildcardid"></a>`weightWildcardId` | [`WeightWildcardId`](#weightwildcardid) | Filter by weight ID wildcard. Incompatible with weight. | ##### `Project.iterationCadences` @@ -21436,6 +21778,25 @@ Returns [`UserMergeRequestInteraction`](#usermergerequestinteraction). | <a id="projectpermissionsupdatewiki"></a>`updateWiki` | [`Boolean!`](#boolean) | Indicates the user can perform `update_wiki` on this resource. | | <a id="projectpermissionsuploadfile"></a>`uploadFile` | [`Boolean!`](#boolean) | Indicates the user can perform `upload_file` on this resource. | +### `ProjectRepositoryRegistry` + +Represents the Geo replication and verification state of a project repository. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectrepositoryregistrycreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp when the ProjectRepositoryRegistry was created. | +| <a id="projectrepositoryregistryid"></a>`id` | [`ID!`](#id) | ID of the ProjectRepositoryRegistry. | +| <a id="projectrepositoryregistrylastsyncfailure"></a>`lastSyncFailure` | [`String`](#string) | Error message during sync of the ProjectRepositoryRegistry. | +| <a id="projectrepositoryregistrylastsyncedat"></a>`lastSyncedAt` | [`Time`](#time) | Timestamp of the most recent successful sync of the ProjectRepositoryRegistry. | +| <a id="projectrepositoryregistryprojectid"></a>`projectId` | [`ID!`](#id) | ID of the Project. | +| <a id="projectrepositoryregistryretryat"></a>`retryAt` | [`Time`](#time) | Timestamp after which the ProjectRepositoryRegistry is resynced. | +| <a id="projectrepositoryregistryretrycount"></a>`retryCount` | [`Int`](#int) | Number of consecutive failed sync attempts of the ProjectRepositoryRegistry. | +| <a id="projectrepositoryregistrystate"></a>`state` | [`RegistryState`](#registrystate) | Sync state of the ProjectRepositoryRegistry. | +| <a id="projectrepositoryregistryverificationretryat"></a>`verificationRetryAt` | [`Time`](#time) | Timestamp after which the ProjectRepositoryRegistry is reverified. | +| <a id="projectrepositoryregistryverifiedat"></a>`verifiedAt` | [`Time`](#time) | Timestamp of the most recent successful verification of the ProjectRepositoryRegistry. | + ### `ProjectSecurityPolicySource` Represents the source of a security policy belonging to a project. @@ -21469,6 +21830,7 @@ Represents the source of a security policy belonging to a project. | <a id="projectstatisticsbuildartifactssize"></a>`buildArtifactsSize` | [`Float!`](#float) | Build artifacts size of the project in bytes. | | <a id="projectstatisticscommitcount"></a>`commitCount` | [`Float!`](#float) | Commit count of the project. | | <a id="projectstatisticscontainerregistrysize"></a>`containerRegistrySize` | [`Float`](#float) | Container Registry size of the project in bytes. | +| <a id="projectstatisticscostfactoredstoragesize"></a>`costFactoredStorageSize` **{warning-solid}** | [`Float!`](#float) | **Introduced** in 16.2. This feature is an Experiment. It can be changed or removed at any time. Storage size in bytes with any applicable cost factor for forks applied. This will equal storage_size if there is no applicable cost factor. | | <a id="projectstatisticslfsobjectssize"></a>`lfsObjectsSize` | [`Float!`](#float) | Large File Storage (LFS) object size of the project in bytes. | | <a id="projectstatisticspackagessize"></a>`packagesSize` | [`Float!`](#float) | Packages size of the project in bytes. | | <a id="projectstatisticspipelineartifactssize"></a>`pipelineArtifactsSize` | [`Float`](#float) | CI Pipeline artifacts size in bytes. | @@ -21900,6 +22262,18 @@ Returns [`Tree`](#tree). | <a id="repositorytreeref"></a>`ref` | [`String`](#string) | Commit ref to get the tree for. Default value is HEAD. | | <a id="repositorytreereftype"></a>`refType` | [`RefType`](#reftype) | Type of ref. | +##### `Repository.validateCodeownerFile` + +Shows linting errors in the CODEOWNER file of the repository. + +Returns [`RepositoryCodeownerValidation`](#repositorycodeownervalidation). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="repositoryvalidatecodeownerfileref"></a>`ref` | [`String`](#string) | Ref where code owners file needs to be checked. Defaults to the repository's default branch. | + ### `RepositoryBlob` #### Fields @@ -21947,6 +22321,24 @@ Returns [`Tree`](#tree). | <a id="repositoryblobstoredexternally"></a>`storedExternally` | [`Boolean`](#boolean) | Whether the blob's content is stored externally (for instance, in LFS). | | <a id="repositoryblobwebpath"></a>`webPath` | [`String`](#string) | Web path of the blob. | +### `RepositoryCodeownerError` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="repositorycodeownererrorcode"></a>`code` | [`String!`](#string) | Linting error code. | +| <a id="repositorycodeownererrorlines"></a>`lines` | [`[Int!]!`](#int) | Lines where the error occurred. | + +### `RepositoryCodeownerValidation` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="repositorycodeownervalidationtotal"></a>`total` | [`Int!`](#int) | Total number of validation error in the file. | +| <a id="repositorycodeownervalidationvalidationerrors"></a>`validationErrors` | [`[RepositoryCodeownerError!]!`](#repositorycodeownererror) | Specific lint error code. | + ### `RepositoryLanguage` #### Fields @@ -22032,11 +22424,13 @@ Counts of requirements by their state. | ---- | ---- | ----------- | | <a id="rootstoragestatisticsbuildartifactssize"></a>`buildArtifactsSize` | [`Float!`](#float) | CI artifacts size in bytes. | | <a id="rootstoragestatisticscontainerregistrysize"></a>`containerRegistrySize` | [`Float!`](#float) | Container Registry size in bytes. | +| <a id="rootstoragestatisticscontainerregistrysizeisestimated"></a>`containerRegistrySizeIsEstimated` | [`Boolean!`](#boolean) | Indicates whether the deduplicated Container Registry size for the namespace is an estimated value or not. | +| <a id="rootstoragestatisticscostfactoredstoragesize"></a>`costFactoredStorageSize` **{warning-solid}** | [`Float!`](#float) | **Introduced** in 16.2. This feature is an Experiment. It can be changed or removed at any time. Total storage in bytes with any applicable cost factor for forks applied. This will equal storage_size if there is no applicable cost factor. | | <a id="rootstoragestatisticsdependencyproxysize"></a>`dependencyProxySize` | [`Float!`](#float) | Dependency Proxy sizes in bytes. | | <a id="rootstoragestatisticslfsobjectssize"></a>`lfsObjectsSize` | [`Float!`](#float) | LFS objects size in bytes. | | <a id="rootstoragestatisticspackagessize"></a>`packagesSize` | [`Float!`](#float) | Packages size in bytes. | | <a id="rootstoragestatisticspipelineartifactssize"></a>`pipelineArtifactsSize` | [`Float!`](#float) | CI pipeline artifacts size in bytes. | -| <a id="rootstoragestatisticsregistrysizeestimated"></a>`registrySizeEstimated` | [`Boolean!`](#boolean) | Indicates whether the deduplicated Container Registry size for the namespace is an estimated value or not. | +| <a id="rootstoragestatisticsregistrysizeestimated"></a>`registrySizeEstimated` **{warning-solid}** | [`Boolean!`](#boolean) | **Deprecated** in 16.2. Use `container_registry_size_is_estimated`. | | <a id="rootstoragestatisticsrepositorysize"></a>`repositorySize` | [`Float!`](#float) | Git repository size in bytes. | | <a id="rootstoragestatisticssnippetssize"></a>`snippetsSize` | [`Float!`](#float) | Snippets size in bytes. | | <a id="rootstoragestatisticsstoragesize"></a>`storageSize` | [`Float!`](#float) | Total storage in bytes. | @@ -23068,6 +23462,7 @@ Core represention of a GitLab user. | <a id="usercoregroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="usercoregroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="usercoreid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="usercoreide"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="usercorejobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="usercorelinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="usercorelocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -23078,6 +23473,7 @@ Core represention of a GitLab user. | <a id="usercorepreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="usercoreprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="usercoreprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="usercorepronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="usercorepublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="usercoresavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="usercorestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -23110,6 +23506,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="usercoreassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="usercoreassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="usercoreassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="usercoreassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="usercoreassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="usercoreassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="usercoreassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -23145,6 +23542,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="usercoreauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="usercoreauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="usercoreauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="usercoreauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="usercoreauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="usercoreauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="usercoreauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -23198,6 +23596,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="usercorereviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="usercorereviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="usercorereviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="usercorereviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="usercorereviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="usercorereviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="usercorereviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -23318,6 +23717,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="usercoreworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="usercoreworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="usercoreworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | ### `UserMergeRequestInteraction` @@ -23421,6 +23822,7 @@ Represents a vulnerability. | <a id="vulnerabilitydetails"></a>`details` | [`[VulnerabilityDetail!]!`](#vulnerabilitydetail) | Details of the vulnerability. | | <a id="vulnerabilitydetectedat"></a>`detectedAt` | [`Time!`](#time) | Timestamp of when the vulnerability was first detected. | | <a id="vulnerabilitydiscussions"></a>`discussions` | [`DiscussionConnection!`](#discussionconnection) | All discussions on this noteable. (see [Connections](#connections)) | +| <a id="vulnerabilitydismissalreason"></a>`dismissalReason` | [`VulnerabilityDismissalReason`](#vulnerabilitydismissalreason) | Reason for dismissal. Returns `null` if `expose_dismissal_reason`feature flag is disabled. | | <a id="vulnerabilitydismissedat"></a>`dismissedAt` | [`Time`](#time) | Timestamp of when the vulnerability state was changed to dismissed. | | <a id="vulnerabilitydismissedby"></a>`dismissedBy` | [`UserCore`](#usercore) | User that dismissed the vulnerability. | | <a id="vulnerabilityexternalissuelinks"></a>`externalIssueLinks` | [`VulnerabilityExternalIssueLinkConnection!`](#vulnerabilityexternalissuelinkconnection) | List of external issue links related to the vulnerability. (see [Connections](#connections)) | @@ -24289,6 +24691,7 @@ Represents a progress widget. | ---- | ---- | ----------- | | <a id="workitemwidgetprogressprogress"></a>`progress` | [`Int`](#int) | Progress of the work item. | | <a id="workitemwidgetprogresstype"></a>`type` | [`WorkItemWidgetType`](#workitemwidgettype) | Widget type. | +| <a id="workitemwidgetprogressupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp of last progress update. | ### `WorkItemWidgetRequirementLegacy` @@ -24672,6 +25075,17 @@ Deploy freeze period status. | <a id="cifreezeperiodstatusactive"></a>`ACTIVE` | Freeze period is active. | | <a id="cifreezeperiodstatusinactive"></a>`INACTIVE` | Freeze period is inactive. | +### `CiGroupVariablesSort` + +Values for sorting inherited variables. + +| Value | Description | +| ----- | ----------- | +| <a id="cigroupvariablessortcreated_asc"></a>`CREATED_ASC` | Created at ascending order. | +| <a id="cigroupvariablessortcreated_desc"></a>`CREATED_DESC` | Created at descending order. | +| <a id="cigroupvariablessortkey_asc"></a>`KEY_ASC` | Key by ascending order. | +| <a id="cigroupvariablessortkey_desc"></a>`KEY_DESC` | Key by descending order. | + ### `CiJobKind` | Value | Description | @@ -24738,6 +25152,7 @@ Values for sorting runners. | <a id="cirunnersortcontacted_desc"></a>`CONTACTED_DESC` | Ordered by contacted_at in descending order. | | <a id="cirunnersortcreated_asc"></a>`CREATED_ASC` | Ordered by created_at in ascending order. | | <a id="cirunnersortcreated_desc"></a>`CREATED_DESC` | Ordered by created_at in descending order. | +| <a id="cirunnersortmost_active_desc"></a>`MOST_ACTIVE_DESC` **{warning-solid}** | **Introduced** in 16.2. This feature is an Experiment. It can be changed or removed at any time. Ordered by number of running jobs in descending order (only available on Ultimate plans). | | <a id="cirunnersorttoken_expires_at_asc"></a>`TOKEN_EXPIRES_AT_ASC` | Ordered by token_expires_at in ascending order. | | <a id="cirunnersorttoken_expires_at_desc"></a>`TOKEN_EXPIRES_AT_DESC` | Ordered by token_expires_at in descending order. | @@ -24838,6 +25253,31 @@ ComplianceFramework of a project for filtering. | <a id="complianceframeworkpresencefilterany"></a>`ANY` | Any compliance framework is assigned. | | <a id="complianceframeworkpresencefilternone"></a>`NONE` | No compliance framework is assigned. | +### `ComplianceStandardsAdherenceCheckName` + +Name of the check for the compliance standard. + +| Value | Description | +| ----- | ----------- | +| <a id="compliancestandardsadherencechecknameprevent_approval_by_merge_request_author"></a>`PREVENT_APPROVAL_BY_MERGE_REQUEST_AUTHOR` | Prevent approval by merge request author. | + +### `ComplianceStandardsAdherenceStandard` + +Name of the compliance standard. + +| Value | Description | +| ----- | ----------- | +| <a id="compliancestandardsadherencestandardgitlab"></a>`GITLAB` | Gitlab. | + +### `ComplianceStandardsAdherenceStatus` + +Status of the compliance standards adherence. + +| Value | Description | +| ----- | ----------- | +| <a id="compliancestandardsadherencestatusfail"></a>`FAIL` | Fail. | +| <a id="compliancestandardsadherencestatussuccess"></a>`SUCCESS` | Success. | + ### `ComplianceViolationReason` Reason for the compliance violation. @@ -25389,6 +25829,7 @@ Geo registry class. | <a id="georegistryclasspackage_file_registry"></a>`PACKAGE_FILE_REGISTRY` | Geo::PackageFileRegistry registry class. | | <a id="georegistryclasspages_deployment_registry"></a>`PAGES_DEPLOYMENT_REGISTRY` | Geo::PagesDeploymentRegistry registry class. | | <a id="georegistryclasspipeline_artifact_registry"></a>`PIPELINE_ARTIFACT_REGISTRY` | Geo::PipelineArtifactRegistry registry class. | +| <a id="georegistryclassproject_repository_registry"></a>`PROJECT_REPOSITORY_REGISTRY` | Geo::ProjectRepositoryRegistry registry class. | | <a id="georegistryclassproject_wiki_repository_registry"></a>`PROJECT_WIKI_REPOSITORY_REGISTRY` | Geo::ProjectWikiRepositoryRegistry registry class. | | <a id="georegistryclasssnippet_repository_registry"></a>`SNIPPET_REPOSITORY_REGISTRY` | Geo::SnippetRepositoryRegistry registry class. | | <a id="georegistryclassterraform_state_version_registry"></a>`TERRAFORM_STATE_VERSION_REGISTRY` | Geo::TerraformStateVersionRegistry registry class. | @@ -25837,6 +26278,7 @@ Representation of whether a GitLab merge request can be merged. | ----- | ----------- | | <a id="mergestrategyenumadd_to_merge_train_when_pipeline_succeeds"></a>`ADD_TO_MERGE_TRAIN_WHEN_PIPELINE_SUCCEEDS` | Use the add_to_merge_train_when_pipeline_succeeds merge strategy. | | <a id="mergestrategyenummerge_train"></a>`MERGE_TRAIN` | Use the merge_train merge strategy. | +| <a id="mergestrategyenummerge_when_checks_pass"></a>`MERGE_WHEN_CHECKS_PASS` | Use the merge_when_checks_pass merge strategy. | | <a id="mergestrategyenummerge_when_pipeline_succeeds"></a>`MERGE_WHEN_PIPELINE_SUCCEEDS` | Use the merge_when_pipeline_succeeds merge strategy. | ### `MilestoneSort` @@ -26090,6 +26532,7 @@ Values for sorting package. | <a id="pipelineconfigsourceenumparameter_source"></a>`PARAMETER_SOURCE` | Parameter source. | | <a id="pipelineconfigsourceenumremote_source"></a>`REMOTE_SOURCE` | Remote source. | | <a id="pipelineconfigsourceenumrepository_source"></a>`REPOSITORY_SOURCE` | Repository source. | +| <a id="pipelineconfigsourceenumsecurity_policies_default_source"></a>`SECURITY_POLICIES_DEFAULT_SOURCE` | Security policies default source. | | <a id="pipelineconfigsourceenumunknown_source"></a>`UNKNOWN_SOURCE` | Unknown source. | | <a id="pipelineconfigsourceenumwebide_source"></a>`WEBIDE_SOURCE` | Webide source. | @@ -26561,10 +27004,9 @@ Name of the feature that the callout is for. | <a id="usercalloutfeaturenameenummerge_request_settings_moved_callout"></a>`MERGE_REQUEST_SETTINGS_MOVED_CALLOUT` | Callout feature name for merge_request_settings_moved_callout. | | <a id="usercalloutfeaturenameenummr_experience_survey"></a>`MR_EXPERIENCE_SURVEY` | Callout feature name for mr_experience_survey. | | <a id="usercalloutfeaturenameenumnamespace_over_storage_users_combined_alert"></a>`NAMESPACE_OVER_STORAGE_USERS_COMBINED_ALERT` | Callout feature name for namespace_over_storage_users_combined_alert. | -| <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_alert_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_ALERT_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_alert_threshold. | -| <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_error_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_ERROR_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_error_threshold. | -| <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_info_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_INFO_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_info_threshold. | -| <a id="usercalloutfeaturenameenumnamespace_storage_limit_banner_warning_threshold"></a>`NAMESPACE_STORAGE_LIMIT_BANNER_WARNING_THRESHOLD` | Callout feature name for namespace_storage_limit_banner_warning_threshold. | +| <a id="usercalloutfeaturenameenumnamespace_storage_limit_alert_alert_threshold"></a>`NAMESPACE_STORAGE_LIMIT_ALERT_ALERT_THRESHOLD` | Callout feature name for namespace_storage_limit_alert_alert_threshold. | +| <a id="usercalloutfeaturenameenumnamespace_storage_limit_alert_error_threshold"></a>`NAMESPACE_STORAGE_LIMIT_ALERT_ERROR_THRESHOLD` | Callout feature name for namespace_storage_limit_alert_error_threshold. | +| <a id="usercalloutfeaturenameenumnamespace_storage_limit_alert_warning_threshold"></a>`NAMESPACE_STORAGE_LIMIT_ALERT_WARNING_THRESHOLD` | Callout feature name for namespace_storage_limit_alert_warning_threshold. | | <a id="usercalloutfeaturenameenumnamespace_storage_pre_enforcement_banner"></a>`NAMESPACE_STORAGE_PRE_ENFORCEMENT_BANNER` | Callout feature name for namespace_storage_pre_enforcement_banner. | | <a id="usercalloutfeaturenameenumnew_navigation_callout"></a>`NEW_NAVIGATION_CALLOUT` | Callout feature name for new_navigation_callout. | | <a id="usercalloutfeaturenameenumnew_top_level_group_alert"></a>`NEW_TOP_LEVEL_GROUP_ALERT` | Callout feature name for new_top_level_group_alert. | @@ -26576,11 +27018,11 @@ Name of the feature that the callout is for. | <a id="usercalloutfeaturenameenumpreview_user_over_limit_free_plan_alert"></a>`PREVIEW_USER_OVER_LIMIT_FREE_PLAN_ALERT` | Callout feature name for preview_user_over_limit_free_plan_alert. | | <a id="usercalloutfeaturenameenumprofile_personal_access_token_expiry"></a>`PROFILE_PERSONAL_ACCESS_TOKEN_EXPIRY` | Callout feature name for profile_personal_access_token_expiry. | | <a id="usercalloutfeaturenameenumproject_quality_summary_feedback"></a>`PROJECT_QUALITY_SUMMARY_FEEDBACK` | Callout feature name for project_quality_summary_feedback. | +| <a id="usercalloutfeaturenameenumproject_repository_limit_alert_alert_threshold"></a>`PROJECT_REPOSITORY_LIMIT_ALERT_ALERT_THRESHOLD` | Callout feature name for project_repository_limit_alert_alert_threshold. | +| <a id="usercalloutfeaturenameenumproject_repository_limit_alert_error_threshold"></a>`PROJECT_REPOSITORY_LIMIT_ALERT_ERROR_THRESHOLD` | Callout feature name for project_repository_limit_alert_error_threshold. | +| <a id="usercalloutfeaturenameenumproject_repository_limit_alert_warning_threshold"></a>`PROJECT_REPOSITORY_LIMIT_ALERT_WARNING_THRESHOLD` | Callout feature name for project_repository_limit_alert_warning_threshold. | | <a id="usercalloutfeaturenameenumregistration_enabled_callout"></a>`REGISTRATION_ENABLED_CALLOUT` | Callout feature name for registration_enabled_callout. | -| <a id="usercalloutfeaturenameenumrepository_storage_limit_banner_alert_threshold"></a>`REPOSITORY_STORAGE_LIMIT_BANNER_ALERT_THRESHOLD` | Callout feature name for repository_storage_limit_banner_alert_threshold. | -| <a id="usercalloutfeaturenameenumrepository_storage_limit_banner_error_threshold"></a>`REPOSITORY_STORAGE_LIMIT_BANNER_ERROR_THRESHOLD` | Callout feature name for repository_storage_limit_banner_error_threshold. | -| <a id="usercalloutfeaturenameenumrepository_storage_limit_banner_info_threshold"></a>`REPOSITORY_STORAGE_LIMIT_BANNER_INFO_THRESHOLD` | Callout feature name for repository_storage_limit_banner_info_threshold. | -| <a id="usercalloutfeaturenameenumrepository_storage_limit_banner_warning_threshold"></a>`REPOSITORY_STORAGE_LIMIT_BANNER_WARNING_THRESHOLD` | Callout feature name for repository_storage_limit_banner_warning_threshold. | +| <a id="usercalloutfeaturenameenumrich_text_editor"></a>`RICH_TEXT_EDITOR` | Callout feature name for rich_text_editor. | | <a id="usercalloutfeaturenameenumsecurity_configuration_devops_alert"></a>`SECURITY_CONFIGURATION_DEVOPS_ALERT` | Callout feature name for security_configuration_devops_alert. | | <a id="usercalloutfeaturenameenumsecurity_configuration_upgrade_banner"></a>`SECURITY_CONFIGURATION_UPGRADE_BANNER` | Callout feature name for security_configuration_upgrade_banner. | | <a id="usercalloutfeaturenameenumsecurity_newsletter_callout"></a>`SECURITY_NEWSLETTER_CALLOUT` | Callout feature name for security_newsletter_callout. | @@ -26635,6 +27077,7 @@ Verification status of a GPG or X.509 signature for a commit. | <a id="verificationstatusunverified"></a>`UNVERIFIED` | unverified verification status. | | <a id="verificationstatusunverified_key"></a>`UNVERIFIED_KEY` | unverified_key verification status. | | <a id="verificationstatusverified"></a>`VERIFIED` | verified verification status. | +| <a id="verificationstatusverified_system"></a>`VERIFIED_SYSTEM` | verified_system verification status. | ### `VisibilityLevelsEnum` @@ -27006,6 +27449,12 @@ A `CiPipelineScheduleID` is a global ID. It is encoded as a string. An example `CiPipelineScheduleID` is: `"gid://gitlab/Ci::PipelineSchedule/1"`. +### `CiPipelineScheduleVariableID` + +A `CiPipelineScheduleVariableID` is a global ID. It is encoded as a string. + +An example `CiPipelineScheduleVariableID` is: `"gid://gitlab/Ci::PipelineScheduleVariable/1"`. + ### `CiRunnerID` A `CiRunnerID` is a global ID. It is encoded as a string. @@ -27212,6 +27661,12 @@ A `GitlabErrorTrackingDetailedErrorID` is a global ID. It is encoded as a string An example `GitlabErrorTrackingDetailedErrorID` is: `"gid://gitlab/Gitlab::ErrorTracking::DetailedError/1"`. +### `GitlabSubscriptionsAddOnPurchaseID` + +A `GitlabSubscriptionsAddOnPurchaseID` is a global ID. It is encoded as a string. + +An example `GitlabSubscriptionsAddOnPurchaseID` is: `"gid://gitlab/GitlabSubscriptions::AddOnPurchase/1"`. + ### `GlobalID` A global identifier. @@ -27896,7 +28351,9 @@ Implementations: | Name | Type | Description | | ---- | ---- | ----------- | | <a id="externalauditeventdestinationinterfacedestinationurl"></a>`destinationUrl` | [`String!`](#string) | External destination to send audit events to. | +| <a id="externalauditeventdestinationinterfaceeventtypefilters"></a>`eventTypeFilters` | [`[String!]!`](#string) | List of event type filters added for streaming. | | <a id="externalauditeventdestinationinterfaceid"></a>`id` | [`ID!`](#id) | ID of the destination. | +| <a id="externalauditeventdestinationinterfacename"></a>`name` | [`String!`](#string) | Name of the external destination to send audit events to. | | <a id="externalauditeventdestinationinterfaceverificationtoken"></a>`verificationToken` | [`String!`](#string) | Verification token to validate source of event. | #### `MemberInterface` @@ -28088,6 +28545,7 @@ Implementations: | <a id="usergroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="usergroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="userid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="useride"></a>`ide` | [`Ide`](#ide) | IDE settings. | | <a id="userjobtitle"></a>`jobTitle` | [`String`](#string) | Job title of the user. | | <a id="userlinkedin"></a>`linkedin` | [`String`](#string) | LinkedIn profile name of the user. | | <a id="userlocation"></a>`location` | [`String`](#string) | Location of the user. | @@ -28098,6 +28556,7 @@ Implementations: | <a id="userpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | | <a id="userprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="userprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="userpronouns"></a>`pronouns` | [`String`](#string) | Pronouns of the user. | | <a id="userpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | | <a id="usersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. Will not return saved replies if `saved_replies` feature flag is disabled. (see [Connections](#connections)) | | <a id="userstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | @@ -28130,6 +28589,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="userassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="userassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="userassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="userassignedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="userassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="userassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="userassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -28165,6 +28625,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="userauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="userauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="userauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="userauthoredmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="userauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="userauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="userauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -28218,6 +28679,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="userreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | | <a id="userreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="userreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="userreviewrequestedmergerequestsgroupid"></a>`groupId` | [`GroupID`](#groupid) | The global ID of the group the authored merge requests should be in. Merge requests in subgroups are included. | | <a id="userreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | | <a id="userreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="userreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -28338,6 +28800,8 @@ four standard [pagination arguments](#connection-pagination-arguments): | Name | Type | Description | | ---- | ---- | ----------- | | <a id="userworkspacesids"></a>`ids` | [`[RemoteDevelopmentWorkspaceID!]`](#remotedevelopmentworkspaceid) | Array of global workspace IDs. For example, `["gid://gitlab/RemoteDevelopment::Workspace/1"]`. | +| <a id="userworkspacesincludeactualstates"></a>`includeActualStates` | [`[String!]`](#string) | Includes all workspaces that match any of the actual states. | +| <a id="userworkspacesprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter workspaces by project id. | #### `WorkItemWidget` @@ -28382,7 +28846,8 @@ see the associated mutation type above. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="aichatinputcontent"></a>`content` | [`String!`](#string) | Content of the message. | -| <a id="aichatinputresourceid"></a>`resourceId` | [`AiModelID!`](#aimodelid) | Global ID of the resource to mutate. | +| <a id="aichatinputnamespaceid"></a>`namespaceId` | [`NamespaceID`](#namespaceid) | Global ID of the namespace the user is acting on. | +| <a id="aichatinputresourceid"></a>`resourceId` | [`AiModelID`](#aimodelid) | Global ID of the resource to mutate. | ### `AiExplainCodeInput` @@ -28408,6 +28873,7 @@ see the associated mutation type above. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="aiexplainvulnerabilityinputincludesourcecode"></a>`includeSourceCode` | [`Boolean`](#boolean) | Include vulnerablility source code in the AI prompt. | | <a id="aiexplainvulnerabilityinputresourceid"></a>`resourceId` | [`AiModelID!`](#aimodelid) | Global ID of the resource to mutate. | ### `AiFillInMergeRequestTemplateInput` @@ -28494,7 +28960,7 @@ Field that are available while modifying the custom mapping attributes for an HT | Name | Type | Description | | ---- | ---- | ----------- | | <a id="boardissueinputassigneeusername"></a>`assigneeUsername` | [`[String]`](#string) | Filter by assignee username. | -| <a id="boardissueinputassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername. | +| <a id="boardissueinputassigneewildcardid"></a>`assigneeWildcardId` | [`AssigneeWildcardId`](#assigneewildcardid) | Filter by assignee wildcard. Incompatible with assigneeUsername and assigneeUsernames. | | <a id="boardissueinputauthorusername"></a>`authorUsername` | [`String`](#string) | Filter by author username. | | <a id="boardissueinputconfidential"></a>`confidential` | [`Boolean`](#boolean) | Filter by confidentiality. | | <a id="boardissueinputepicid"></a>`epicId` | [`EpicID`](#epicid) | Filter by epic ID. Incompatible with epicWildcardId. | @@ -28564,6 +29030,16 @@ Attributes for defining a CI/CD variable. | <a id="complianceframeworkinputname"></a>`name` | [`String`](#string) | New name for the compliance framework. | | <a id="complianceframeworkinputpipelineconfigurationfullpath"></a>`pipelineConfigurationFullPath` | [`String`](#string) | Full path of the compliance pipeline configuration stored in a project repository, such as `.gitlab/.compliance-gitlab-ci.yml@compliance/hipaa` **(ULTIMATE)**. | +### `ComplianceStandardsAdherenceInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="compliancestandardsadherenceinputcheckname"></a>`checkName` | [`ComplianceStandardsAdherenceCheckName`](#compliancestandardsadherencecheckname) | Name of the check for the compliance standard. | +| <a id="compliancestandardsadherenceinputprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter compliance standards adherence by project. | +| <a id="compliancestandardsadherenceinputstandard"></a>`standard` | [`ComplianceStandardsAdherenceStandard`](#compliancestandardsadherencestandard) | Name of the compliance standard. | + ### `ComplianceViolationInput` #### Arguments @@ -28855,6 +29331,8 @@ Attributes for the pipeline schedule variable. | Name | Type | Description | | ---- | ---- | ----------- | +| <a id="pipelineschedulevariableinputdestroy"></a>`destroy` | [`Boolean`](#boolean) | Boolean option to destroy the variable. | +| <a id="pipelineschedulevariableinputid"></a>`id` | [`CiPipelineScheduleVariableID`](#cipipelineschedulevariableid) | ID of the variable to mutate. | | <a id="pipelineschedulevariableinputkey"></a>`key` | [`String!`](#string) | Name of the variable. | | <a id="pipelineschedulevariableinputvalue"></a>`value` | [`String!`](#string) | Value of the variable. | | <a id="pipelineschedulevariableinputvariabletype"></a>`variableType` | [`CiVariableType!`](#civariabletype) | Type of the variable. | @@ -29169,7 +29647,7 @@ A time-frame defined as a closed inclusive range of two dates. | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="workitemwidgetprogressinputprogress"></a>`progress` | [`Int!`](#int) | Progress of the work item. | +| <a id="workitemwidgetprogressinputcurrentvalue"></a>`currentValue` | [`Int!`](#int) | Current progress value of the work item. | ### `WorkItemWidgetStartAndDueDateUpdateInput` diff --git a/doc/api/group_access_tokens.md b/doc/api/group_access_tokens.md index 446aa3668d8..2493dfcc039 100644 --- a/doc/api/group_access_tokens.md +++ b/doc/api/group_access_tokens.md @@ -81,7 +81,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a ## Create a group access token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77236) in GitLab 14.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77236) in GitLab 14.7. +> - The `expires_at` attribute default was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120213) in GitLab 16.0. Create a [group access token](../user/group/settings/group_access_tokens.md). You must have the Owner role for the group to create group access tokens. @@ -96,7 +97,7 @@ POST groups/:id/access_tokens | `name` | String | yes | Name of the group access token | | `scopes` | `Array[String]` | yes | [List of scopes](../user/group/settings/group_access_tokens.md#scopes-for-a-group-access-token) | | `access_level` | Integer | no | Access level. Valid values are `10` (Guest), `20` (Reporter), `30` (Developer), `40` (Maintainer), and `50` (Owner). | -| `expires_at` | Date | no | Token expires at midnight UTC on that date | +| `expires_at` | Date | yes | Expiration date of the access token in ISO format (`YYYY-MM-DD`). The date cannot be set later than the [maximum allowable lifetime of an access token](../user/profile/personal_access_tokens.md#when-personal-access-tokens-expire). | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ diff --git a/doc/api/group_import_export.md b/doc/api/group_import_export.md index a409b2e3a11..314fb63058c 100644 --- a/doc/api/group_import_export.md +++ b/doc/api/group_import_export.md @@ -93,8 +93,8 @@ returns either: The maximum import file size can be set by the Administrator on self-managed instances (default is `0` (unlimited)). As an administrator, you can modify the maximum import file size either: -- The admin [Admin Area](../user/admin_area/settings/account_and_limit_settings.md). -- The `max_import_size` option in the [Application settings API](settings.md#change-application-settings). +- In the [Admin Area](../administration/settings/account_and_limit_settings.md). +- By using the `max_import_size` option in the [Application settings API](settings.md#change-application-settings). For information on the maximum import file size on GitLab.com, see [Account and limit settings](../user/gitlab_com/index.md#account-and-limit-settings). diff --git a/doc/api/group_level_variables.md b/doc/api/group_level_variables.md index 921a9922c9b..08fb9e55f32 100644 --- a/doc/api/group_level_variables.md +++ b/doc/api/group_level_variables.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Group-level Variables API **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34519) in GitLab 9.5 - ## List group variables Get list of a group's variables. @@ -16,9 +14,9 @@ Get list of a group's variables. GET /groups/:id/variables ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/variables" @@ -33,7 +31,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null }, { "key": "TEST_VARIABLE_2", @@ -42,7 +41,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ] ``` @@ -55,10 +55,10 @@ Get the details of a group's specific variable. GET /groups/:id/variables/:key ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|-----------------------| -| `id` | integer/string | yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/variables/TEST_VARIABLE_1" @@ -72,7 +72,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ``` @@ -84,16 +85,17 @@ Create a new variable. POST /groups/:id/variables ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-----------------------| -| `id` | integer/string | yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected | -| `masked` | boolean | no | Whether the variable is masked | -| `raw` | boolean | no | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | -| `environment_scope` **(PREMIUM)** | string | no | The [environment scope](../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable | +| Attribute | Type | Required | Description | +|-----------------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | No | Whether the variable is protected | +| `masked` | boolean | No | Whether the variable is masked | +| `raw` | boolean | No | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | +| `environment_scope` **(PREMIUM)** | string | No | The [environment scope](../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable | +| `description` | string | No | The `description` of the variable. Default: `null` | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -108,7 +110,8 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ``` @@ -120,16 +123,17 @@ Update a group's variable. PUT /groups/:id/variables/:key ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-------------------------| -| `id` | integer/string | yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected | -| `masked` | boolean | no | Whether the variable is masked | -| `raw` | boolean | no | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | -| `environment_scope` **(PREMIUM)** | string | no | The [environment scope](../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable | +| Attribute | Type | Required | Description | +|-----------------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | No | Whether the variable is protected | +| `masked` | boolean | No | Whether the variable is masked | +| `raw` | boolean | No | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | +| `environment_scope` **(PREMIUM)** | string | No | The [environment scope](../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) of a variable | +| `description` | string | No | The description of the variable. Default: `null`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409641) in GitLab 16.2. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -144,7 +148,8 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ "protected": true, "masked": true, "raw": true, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ``` @@ -156,10 +161,10 @@ Remove a group's variable. DELETE /groups/:id/variables/:key ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|-------------------------| -| `id` | integer/string | yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID of a group or [URL-encoded path of the group](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" \ diff --git a/doc/api/group_protected_branches.md b/doc/api/group_protected_branches.md index db42ca98f0e..c017d0741ce 100644 --- a/doc/api/group_protected_branches.md +++ b/doc/api/group_protected_branches.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110603) in GitLab 15.9 [with a flag](../administration/feature_flags.md) named `group_protected_branches`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `group_protected_branches`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `group_protected_branches`. On GitLab.com, this feature is not available. ## Valid access levels diff --git a/doc/api/group_relations_export.md b/doc/api/group_relations_export.md index 3721e712dd9..8e8eb05b6b9 100644 --- a/doc/api/group_relations_export.md +++ b/doc/api/group_relations_export.md @@ -8,11 +8,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/59978) in GitLab 13.12. -The group relations export API partially exports a group's structure as separate files for each top-level +The group relations export API partially exports a group's structure as separate files for each +top-level relation (for example, milestones, boards, and labels). The group relations export API is primarily used in -[group migration by direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) and +[group migration by direct transfer](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) +and can't be used with the [group import and export API](group_import_export.md). ## Schedule new export @@ -23,9 +25,10 @@ Start a new group relations export: POST /groups/:id/export_relations ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|--------------------------------------------------| | `id` | integer/string | yes | ID of the group owned by the authenticated user. | +| `batched` | boolean | no | Whether to export in batches. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/export_relations" @@ -45,9 +48,10 @@ View the status of the relations export: GET /groups/:id/export_relations/status ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | ID of the group owned by the authenticated user. | +| Attribute | Type | Required | Description | +|------------|----------------|----------|--------------------------------------------------| +| `id` | integer/string | yes | ID of the group owned by the authenticated user. | +| `relation` | string | no | Name of the project top-level relation to view. | ```shell curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -70,13 +74,24 @@ The status can be one of the following: "relation": "badges", "status": 1, "error": null, - "updated_at": "2021-05-04T11:25:20.423Z" + "updated_at": "2021-05-04T11:25:20.423Z", + "batched": true, + "batches": [ + { + "status": 1, + "batch_number": 1, + "objects_count": 1, + "error": null, + "updated_at": "2021-05-04T11:25:20.423Z" + } + ] }, { "relation": "boards", "status": 1, "error": null, - "updated_at": "2021-05-04T11:25:20.085Z" + "updated_at": "2021-05-04T11:25:20.085Z", + "batched": false } ] ``` @@ -89,10 +104,12 @@ Download the finished relations export: GET /groups/:id/export_relations/download ``` -| Attribute | Type | Required | Description | -| --------------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | ID of the group owned by the authenticated user. | -| `relation` | string | yes | Name of the group top-level relation to download. | +| Attribute | Type | Required | Description | +|----------------|----------------|----------|---------------------------------------------------| +| `id` | integer/string | yes | ID of the group owned by the authenticated user. | +| `relation` | string | yes | Name of the group top-level relation to download. | +| `batched` | boolean | no | Whether the export is batched. | +| `batch_number` | integer | no | Number of export batch to download. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" --remote-header-name \ diff --git a/doc/api/group_releases.md b/doc/api/group_releases.md index a735f36eb82..b8d15afcf06 100644 --- a/doc/api/group_releases.md +++ b/doc/api/group_releases.md @@ -12,7 +12,7 @@ NOTE: For more information about the project releases API, see [Releases API](releases/index.md). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `group_releases_finder_inoperator`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `group_releases_finder_inoperator`. ## List group releases diff --git a/doc/api/groups.md b/doc/api/groups.md index 7b73fe9066e..235ef119e7b 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -303,6 +303,7 @@ Parameters: | `simple` | boolean | no | Return only limited fields for each project. This is a no-op without authentication where only simple fields are returned. | | `owned` | boolean | no | Limit by projects owned by the current user | | `starred` | boolean | no | Limit by projects starred by the current user | +| `topic` | string | no | Return projects matching the topic | | `with_issues_enabled` | boolean | no | Limit by projects with issues feature enabled. Default is `false` | | `with_merge_requests_enabled` | boolean | no | Limit by projects with merge requests feature enabled. Default is `false` | | `with_shared` | boolean | no | Include projects shared to this group. Default is `true` | @@ -822,8 +823,8 @@ Parameters: | `two_factor_grace_period` | integer | no | Time before Two-factor authentication is enforced (in hours). | | `visibility` | string | no | The group's visibility. Can be `private`, `internal`, or `public`. | | `membership_lock` **(PREMIUM)** | boolean | no | Users cannot be added to projects in this group. | -| `extra_shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Additional units of compute for this group. | -| `shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Maximum number of monthly units of compute for this group. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | +| `extra_shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Additional compute minutes for this group. | +| `shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Maximum number of monthly compute minutes for this group. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | | `wiki_access_level` **(PREMIUM)** | string | no | The wiki access level. Can be `disabled`, `private`, or `enabled`. | ### Options for `default_branch_protection` @@ -836,7 +837,7 @@ The `default_branch_protection` attribute determines whether users with the Deve | `1` | Partial protection. Users with the Developer or Maintainer role can: <br>- Push new commits | | `2` | Full protection. Only users with the Maintainer role can: <br>- Push new commits | | `3` | Protected against pushes. Users with the Maintainer role can: <br>- Push new commits<br>- Force push changes<br>- Accept merge requests<br>Users with the Developer role can:<br>- Accept merge requests| -| `4` | Protected against pushes except initial push. User with the Developer rope can: <br>- Push commit to empty repository.<br> Users with the Maintainer role can: <br>- Push new commits<br>- Force push changes<br>- Accept merge requests<br>Users with the Developer role can:<br>- Accept merge requests| +| `4` | Protected against pushes except initial push. User with the Developer role can: <br>- Push commit to empty repository.<br> Users with the Maintainer role can: <br>- Push new commits<br>- Force push changes<br>- Accept merge requests<br>Users with the Developer role can:<br>- Accept merge requests| ## New Subgroup @@ -949,7 +950,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ FLAG: On self-managed GitLab, by default `unique_project_download_limit`, `unique_project_download_limit_interval_in_seconds`, `unique_project_download_limit_allowlist` and `auto_ban_user_on_excessive_projects_download` are not available. -To make them available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +To make them available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. Updates the project group. Only available to group owners and administrators. @@ -979,11 +980,11 @@ PUT /groups/:id | `subgroup_creation_level` | string | no | Allowed to [create subgroups](../user/group/subgroups/index.md#create-a-subgroup). Can be `owner` (Owners), or `maintainer` (users with the Maintainer role). | | `two_factor_grace_period` | integer | no | Time before Two-factor authentication is enforced (in hours). | | `visibility` | string | no | The visibility level of the group. Can be `private`, `internal`, or `public`. | -| `extra_shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Additional units of compute for this group. | +| `extra_shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Additional compute minutes for this group. | | `file_template_project_id` **(PREMIUM)** | integer | no | The ID of a project to load custom file templates from. | | `membership_lock` **(PREMIUM)** | boolean | no | Users cannot be added to projects in this group. | | `prevent_forking_outside_group` **(PREMIUM)** | boolean | no | When enabled, users can **not** fork projects from this group to external namespaces. | -| `shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Maximum number of monthly units of compute for this group. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | +| `shared_runners_minutes_limit` **(PREMIUM SELF)** | integer | no | Can be set by administrators only. Maximum number of monthly compute minutes for this group. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | | `unique_project_download_limit` **(ULTIMATE)** | integer | no | Maximum number of unique projects a user can download in the specified time period before they are banned. Available only on top-level groups. Default: 0, Maximum: 10,000. | | `unique_project_download_limit_interval_in_seconds` **(ULTIMATE)** | integer | no | Time period during which a user can download a maximum amount of projects before they are banned. Available only on top-level groups. Default: 0, Maximum: 864,000 seconds (10 days). | | `unique_project_download_limit_allowlist` **(ULTIMATE)** | array of strings | no | List of usernames excluded from the unique project download limit. Available only on top-level groups. Default: `[]`, Maximum: 100 usernames. | @@ -1139,7 +1140,7 @@ Only available to group owners and administrators. This endpoint: -- On Premium and Ultimate tiers, marks the group for deletion. The deletion happens 7 days later by default, but you can change the retention period in the [instance settings](../user/admin_area/settings/visibility_and_access_controls.md#deletion-protection). +- On Premium and Ultimate tiers, marks the group for deletion. The deletion happens 7 days later by default, but you can change the retention period in the [instance settings](../administration/settings/visibility_and_access_controls.md#deletion-protection). - On Free tier, removes the group immediately and queues a background job to delete all projects in the group. - Deletes a subgroup immediately if the subgroup is marked for deletion (GitLab 15.4 and later). The endpoint does not immediately delete top-level groups. @@ -1304,7 +1305,7 @@ POST /groups/:id/service_accounts/:user_id/personal_access_tokens ``` ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gdk.test:3443/api/v4/groups/35/service_accounts/71/personal_access_tokens" --data "scopes[]=api" --data "name=service_accounts_token" +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/35/service_accounts/71/personal_access_tokens" --data "scopes[]=api" --data "name=service_accounts_token" ``` Example response: @@ -1333,7 +1334,7 @@ POST /groups/:id/service_accounts/:user_id/personal_access_tokens/:token_id/rota ``` ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gdk.test:3443/api/v4/groups/35/service_accounts/71/personal_access_tokens/6/rotate" +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/35/service_accounts/71/personal_access_tokens/6/rotate" ``` Example response: diff --git a/doc/api/import.md b/doc/api/import.md index be70868cca5..7bbc19cb36a 100644 --- a/doc/api/import.md +++ b/doc/api/import.md @@ -26,14 +26,15 @@ Prerequisites: POST /import/github ``` -| Attribute | Type | Required | Description | -|-------------------------|---------|----------|-------------------------------------------------------------------------------------| -| `personal_access_token` | string | yes | GitHub personal access token | -| `repo_id` | integer | yes | GitHub repository ID | -| `new_name` | string | no | New repository name | -| `target_namespace` | string | yes | Namespace to import repository into. Supports subgroups like `/namespace/subgroup`. In GitLab 15.8 and later, must not be blank | -| `github_hostname` | string | no | Custom GitHub Enterprise hostname. Do not set for GitHub.com. | -| `optional_stages` | object | no | [Additional items to import](../user/project/import/github.md#select-additional-items-to-import). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/373705) in GitLab 15.5 | +| Attribute | Type | Required | Description | +|----------------------------|---------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `personal_access_token` | string | yes | GitHub personal access token | +| `repo_id` | integer | yes | GitHub repository ID | +| `new_name` | string | no | New repository name | +| `target_namespace` | string | yes | Namespace to import repository into. Supports subgroups like `/namespace/subgroup`. In GitLab 15.8 and later, must not be blank | +| `github_hostname` | string | no | Custom GitHub Enterprise hostname. Do not set for GitHub.com. | +| `optional_stages` | object | no | [Additional items to import](../user/project/import/github.md#select-additional-items-to-import). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/373705) in GitLab 15.5 | +| `additional_access_tokens` | string | no | Additional list of comma-separated personal access tokens. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337232) in GitLab 16.2 | ```shell curl --request POST \ @@ -51,7 +52,8 @@ curl --request POST \ "single_endpoint_notes_import": true, "attachments_import": true, "collaborators_import": true - } + }, + "additional_access_tokens": "foo,bar" }' ``` @@ -64,6 +66,8 @@ The following keys are available for `optional_stages`: For more information, see [Select additional items to import](../user/project/import/github.md#select-additional-items-to-import). +You can supply multiple personal access tokens in `additional_access_tokens` from different user accounts to import projects faster. + Example response: ```json diff --git a/doc/api/instance_level_ci_variables.md b/doc/api/instance_level_ci_variables.md index 840744bcae1..2bf9b209e20 100644 --- a/doc/api/instance_level_ci_variables.md +++ b/doc/api/instance_level_ci_variables.md @@ -6,9 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Instance-level CI/CD variables API **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0 -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/218249) in GitLab 13.2. - ## List all instance variables Get the list of all instance-level variables. @@ -50,9 +47,9 @@ Get the details of a specific instance-level variable. GET /admin/ci/variables/:key ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|-----------------------| -| `key` | string | yes | The `key` of a variable | +| Attribute | Type | Required | Description | +|-----------|---------|----------|-------------| +| `key` | string | Yes | The `key` of a variable | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables/TEST_VARIABLE_1" @@ -73,20 +70,20 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a Create a new instance-level variable. -[In GitLab 13.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/216097), the maximum number of allowed instance-level variables can be changed. +The [maximum number of instance-level variables](../administration/instance_limits.md#number-of-instance-level-variables) can be changed. ```plaintext POST /admin/ci/variables ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-----------------------| -| `key` | string | yes | The `key` of a variable. Max 255 characters, only `A-Z`, `a-z`, `0-9`, and `_` are allowed. | -| `value` | string | yes | The `value` of a variable. 10,000 characters allowed ([GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028)). | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file`. | -| `protected` | boolean | no | Whether the variable is protected. | -| `masked` | boolean | no | Whether the variable is masked. | -| `raw` | boolean | no | Whether the variable is expandable. | +| Attribute | Type | Required | Description | +|-----------------|---------|----------|-------------| +| `key` | string | Yes | The `key` of a variable. Maximum of 255 characters, only `A-Z`, `a-z`, `0-9`, and `_` are allowed. | +| `value` | string | Yes | The `value` of a variable. Maximum of 10,000 characters. | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file`. | +| `protected` | boolean | No | Whether the variable is protected. | +| `masked` | boolean | No | Whether the variable is masked. | +| `raw` | boolean | No | Whether the variable is expandable. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -112,14 +109,14 @@ Update an instance-level variable. PUT /admin/ci/variables/:key ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-------------------------| -| `key` | string | yes | The `key` of a variable. | -| `value` | string | yes | The `value` of a variable. 10,000 characters allowed ([GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028)). | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file`. | -| `protected` | boolean | no | Whether the variable is protected. | -| `masked` | boolean | no | Whether the variable is masked. | -| `raw` | boolean | no | Whether the variable is expandable. | +| Attribute | Type | Required | Description | +|-----------------|---------|----------|-------------| +| `key` | string | Yes | The `key` of a variable. Maximum of 255 characters, only `A-Z`, `a-z`, `0-9`, and `_` are allowed. | +| `value` | string | Yes | The `value` of a variable. Maximum of 10,000 characters. | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file`. | +| `protected` | boolean | No | Whether the variable is protected. | +| `masked` | boolean | No | Whether the variable is masked. | +| `raw` | boolean | No | Whether the variable is expandable. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -145,9 +142,9 @@ Remove an instance-level variable. DELETE /admin/ci/variables/:key ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|-------------------------| -| `key` | string | yes | The `key` of a variable | +| Attribute | Type | Required | Description | +|-----------|--------|----------|-------------| +| `key` | string | Yes | The `key` of a variable | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/admin/ci/variables/VARIABLE_1" diff --git a/doc/api/issues.md b/doc/api/issues.md index 03cc107415f..9f9775c1746 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -1142,7 +1142,7 @@ Please use `iid` of the `epic` attribute instead. ## Rate limits To help avoid abuse, users can be limited to a specific number of `Create` requests per minute. -See [Issues rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md). +See [Issues rate limits](../administration/settings/rate_limit_on_issues_creation.md). ## Edit issue diff --git a/doc/api/job_artifacts.md b/doc/api/job_artifacts.md index afb8f294a18..d8343cb7c44 100644 --- a/doc/api/job_artifacts.md +++ b/doc/api/job_artifacts.md @@ -18,9 +18,9 @@ GET /projects/:id/jobs/:job_id/artifacts | Attribute | Type | Required | Description | |---------------------------|----------------|----------|-------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `job_id` | integer | yes | ID of a job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | +| `job_token` **(PREMIUM)** | string | No | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | Example request using the `PRIVATE-TOKEN` header: @@ -82,10 +82,10 @@ Parameters | Attribute | Type | Required | Description | |---------------------------|----------------|----------|-------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `ref_name` | string | yes | Branch or tag name in repository. HEAD or SHA references are not supported. | -| `job` | string | yes | The name of the job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `ref_name` | string | Yes | Branch or tag name in repository. HEAD or SHA references are not supported. | +| `job` | string | Yes | The name of the job. | +| `job_token` **(PREMIUM)** | string | No | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | Example request using the `PRIVATE-TOKEN` header: @@ -128,9 +128,6 @@ Possible response status codes: ## Download a single artifact file by job ID -> - Introduced in GitLab 10.0. -> - The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55042) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.10. - Download a single artifact file from a job with a specified ID from inside the job's artifacts zipped archive. The file is extracted from the archive and streamed to the client. @@ -143,10 +140,10 @@ Parameters | Attribute | Type | Required | Description | |---------------------------|----------------|----------|-------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `job_id` | integer | yes | The unique job identifier. | -| `artifact_path` | string | yes | Path to a file inside the artifacts archive. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | The unique job identifier. | +| `artifact_path` | string | Yes | Path to a file inside the artifacts archive. | +| `job_token` **(PREMIUM)** | string | No | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | Example request: @@ -164,9 +161,6 @@ Possible response status codes: ## Download a single artifact file from specific tag or branch -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/23538) in GitLab 11.5. -> - The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55042) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.10. - Download a single artifact file for a specific job of the latest **successful** pipeline for the given reference name from inside the job's artifacts archive. The file is extracted from the archive and streamed to the client. @@ -174,10 +168,9 @@ The file is extracted from the archive and streamed to the client. The artifact file provides more detail than what is available in the [CSV export](../user/application_security/vulnerability_report/index.md#export-vulnerability-details). -In [GitLab 13.5](https://gitlab.com/gitlab-org/gitlab/-/issues/201784) and later, artifacts -for [parent and child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) are searched in hierarchical -order from parent to child. For example, if both parent and child pipelines have a -job with the same name, the artifact from the parent pipeline is returned. +Artifacts for [parent and child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines) +are searched in hierarchical order from parent to child. For example, if both parent and child pipelines +have a job with the same name, the artifact from the parent pipeline is returned. ```plaintext GET /projects/:id/jobs/artifacts/:ref_name/raw/*artifact_path?job=name @@ -187,11 +180,11 @@ Parameters: | Attribute | Type | Required | Description | |---------------------------|----------------|----------|-------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `ref_name` | string | yes | Branch or tag name in repository. `HEAD` or `SHA` references are not supported. | -| `artifact_path` | string | yes | Path to a file inside the artifacts archive. | -| `job` | string | yes | The name of the job. | -| `job_token` **(PREMIUM)** | string | no | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `ref_name` | string | Yes | Branch or tag name in repository. `HEAD` or `SHA` references are not supported. | +| `artifact_path` | string | Yes | Path to a file inside the artifacts archive. | +| `job` | string | Yes | The name of the job. | +| `job_token` **(PREMIUM)** | string | No | To be used with [triggers](../ci/jobs/ci_job_token.md#download-an-artifact-from-a-different-pipeline) for multi-project pipelines. It should be invoked only in a CI/CD job defined in the `.gitlab-ci.yml` file. The value is always `$CI_JOB_TOKEN`. The job associated with the `$CI_JOB_TOKEN` must be running when this token is used. | Example request: @@ -217,10 +210,10 @@ POST /projects/:id/jobs/:job_id/artifacts/keep Parameters -| Attribute | Type | Required | Description | -|-----------|----------------|----------|--------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | +| `job_id` | integer | Yes | ID of a job. | Example request: @@ -264,8 +257,6 @@ Example response: ## Delete job artifacts -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/25522) in GitLab 11.9. - Delete artifacts of a job. Prerequisites: @@ -276,10 +267,10 @@ Prerequisites: DELETE /projects/:id/jobs/:job_id/artifacts ``` -| Attribute | Type | Required | Description | -|-----------|----------------|----------|-----------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `job_id` | integer | yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | Example request: @@ -305,9 +296,9 @@ By default, [artifacts from the most recent successful pipeline of each ref are DELETE /projects/:id/artifacts ``` -| Attribute | Type | Required | Description | -|-----------|----------------|----------|-----------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | Example request: diff --git a/doc/api/jobs.md b/doc/api/jobs.md index a28acfbd1a1..a48168c8362 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -21,10 +21,10 @@ pagination is recommended when requesting consecutive pages of results. GET /projects/:id/jobs ``` -| Attribute | Type | Required | Description | -|-----------|--------------------------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `scope` | string **or** array of strings | **{dotted-circle}** No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | +| Attribute | Type | Required | Description | +|-----------|--------------------------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `scope` | string **or** array of strings | No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | ```shell curl --globoff --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs?scope[]=pending&scope[]=running" @@ -172,16 +172,22 @@ Get a list of jobs for a pipeline. By default, this request returns 20 results at a time because the API results [are paginated](rest/index.md#pagination) +This endpoint: + +- [Returns data for any pipeline](pipelines.md#get-a-single-pipeline) including [child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). +- Does not return retried jobs in the response by default. +- Sorts jobs by ID in descending order (newest first). + ```plaintext GET /projects/:id/pipelines/:pipeline_id/jobs ``` -| Attribute | Type | Required | Description | -|-------------------|--------------------------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `pipeline_id` | integer | **{check-circle}** Yes | ID of a pipeline. Can also be obtained in CI jobs via the [predefined CI variable](../ci/variables/predefined_variables.md) `CI_PIPELINE_ID`. | -| `scope` | string **or** array of strings | **{dotted-circle}** No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | -| `include_retried` | boolean | **{dotted-circle}** No | Include retried jobs in the response. Defaults to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/272627) in GitLab 13.9. | +| Attribute | Type | Required | Description | +|-------------------|--------------------------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `pipeline_id` | integer | Yes | ID of a pipeline. Can also be obtained in CI jobs via the [predefined CI variable](../ci/variables/predefined_variables.md) `CI_PIPELINE_ID`. | +| `scope` | string **or** array of strings | No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | +| `include_retried` | boolean | No | Include retried jobs in the response. Defaults to `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/272627) in GitLab 13.9. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/6/jobs?scope[]=pending&scope[]=running" @@ -323,16 +329,6 @@ Example of response ] ``` -In GitLab 13.3 and later, this endpoint [returns data for any pipeline](pipelines.md#get-a-single-pipeline) -including [child pipelines](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). - -In GitLab 13.5 and later, this endpoint does not return retried jobs in the response -by default. Additionally, jobs are sorted by ID in descending order (newest first). -In earlier GitLab versions, jobs are sorted by ID in ascending order (oldest first). - -In GitLab 13.9 and later, this endpoint can include retried jobs in the response -with `include_retried` set to `true`. - ## List pipeline trigger jobs Get a list of trigger jobs for a pipeline. @@ -341,11 +337,11 @@ Get a list of trigger jobs for a pipeline. GET /projects/:id/pipelines/:pipeline_id/bridges ``` -| Attribute | Type | Required | Description | -|---------------|--------------------------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `pipeline_id` | integer | **{check-circle}** Yes | ID of a pipeline. | -| `scope` | string **or** array of strings | **{dotted-circle}** No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | +| Attribute | Type | Required | Description | +|---------------|--------------------------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `pipeline_id` | integer | Yes | ID of a pipeline. | +| `scope` | string **or** array of strings | No | Scope of jobs to show. Either one of or an array of the following: `created`, `pending`, `running`, `failed`, `success`, `canceled`, `skipped`, `waiting_for_resource`, or `manual`. All jobs are returned if `scope` is not provided. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/6/bridges?scope[]=pending&scope[]=running" @@ -425,8 +421,6 @@ Example of response ## Get job token's job -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/51727) in GitLab 13.10. - Retrieve the job that generated a job token. ```plaintext @@ -505,8 +499,6 @@ Example of response ## Get GitLab agent by `CI_JOB_TOKEN` **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324269) in GitLab 13.11. - Retrieve the job that generated the `CI_JOB_TOKEN`, along with a list of allowed [agents](../user/clusters/agent/index.md). @@ -516,9 +508,9 @@ GET /job/allowed_agents Supported attributes: -| Attribute | Type | Required | Description | -|----------------|----------|------------------------|-------------| -| `CI_JOB_TOKEN` | string | **{check-circle}** Yes | Token value associated with the GitLab-provided `CI_JOB_TOKEN` variable. | +| Attribute | Type | Required | Description | +|----------------|--------|----------|-------------| +| `CI_JOB_TOKEN` | string | Yes | Token value associated with the GitLab-provided `CI_JOB_TOKEN` variable. | Example request: @@ -584,10 +576,10 @@ Get a single job of a project GET /projects/:id/jobs/:job_id ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/8" @@ -665,10 +657,10 @@ Get a log (trace) of a specific job of a project: GET /projects/:id/jobs/:job_id/trace ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | ```shell curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/8/trace" @@ -689,10 +681,10 @@ Cancel a single job of a project POST /projects/:id/jobs/:job_id/cancel ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/1/cancel" @@ -743,10 +735,10 @@ Retry a single job of a project POST /projects/:id/jobs/:job_id/retry ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/1/retry" @@ -799,10 +791,10 @@ POST /projects/:id/jobs/:job_id/erase Parameters -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | Example of request @@ -860,11 +852,11 @@ For a job in manual status, trigger an action to start the job. POST /projects/:id/jobs/:job_id/play ``` -| Attribute | Type | Required | Description | -|----------------------------|-----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | -| `job_variables_attributes` | array of hashes | **{dotted-circle}** No | An array containing the custom variables available to the job. [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/37267) GitLab 14.9. | +| Attribute | Type | Required | Description | +|----------------------------|-----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `job_id` | integer | Yes | ID of a job. | +| `job_variables_attributes` | array of hashes | No | An array containing the custom variables available to the job. [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/37267) GitLab 14.9. | Example request: diff --git a/doc/api/lint.md b/doc/api/lint.md index cfd34f6a40c..b0282b89f6e 100644 --- a/doc/api/lint.md +++ b/doc/api/lint.md @@ -6,9 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # CI Lint API **(FREE)** -## Validate a CI YAML configuration with a namespace - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231352) in GitLab 13.6. +## Validate the CI/CD configuration for a namespace Checks if CI/CD YAML configuration is valid. This endpoint has namespace specific context. @@ -17,12 +15,12 @@ specific context. POST /projects/:id/ci/lint ``` -| Attribute | Type | Required | Description | -| ---------- | ------- | -------- | -------- | -| `content` | string | yes | The CI/CD configuration content. | -| `dry_run` | boolean | no | Run [pipeline creation simulation](../ci/lint.md#simulate-a-pipeline), or only do static check. This is false by default. | -| `include_jobs` | boolean | no | If the list of jobs that would exist in a static check or pipeline simulation should be included in the response. This is false by default. | -| `ref` | string | no | When `dry_run` is `true`, sets the branch or tag to use. Defaults to the project's default branch when not set. | +| Attribute | Type | Required | Description | +|----------------|---------|----------|-------------| +| `content` | string | Yes | The CI/CD configuration content. | +| `dry_run` | boolean | No | Run [pipeline creation simulation](../ci/lint.md#simulate-a-pipeline), or only do static check. Default: `false`. | +| `include_jobs` | boolean | No | If the list of jobs that would exist in a static check or pipeline simulation should be included in the response. Default: `false`. | +| `ref` | string | No | When `dry_run` is `true`, sets the branch or tag to use. Defaults to the project's default branch when not set. | Example request: @@ -58,8 +56,6 @@ Example responses: ## Validate a project's CI configuration -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231352) in GitLab 13.5. - Checks if a project's latest (`HEAD` of the project's default branch) `.gitlab-ci.yml` configuration is valid. This endpoint uses all namespace specific data available, including variables and local includes. @@ -68,11 +64,11 @@ specific data available, including variables and local includes. GET /projects/:id/ci/lint ``` -| Attribute | Type | Required | Description | -| ---------- | ------- | -------- | -------- | -| `dry_run` | boolean | no | Run pipeline creation simulation, or only do static check. | -| `include_jobs` | boolean | no | If the list of jobs that would exist in a static check or pipeline simulation should be included in the response. This is false by default. | -| `ref` | string | no | When `dry_run` is `true`, sets the branch or tag to use. Defaults to the project's default branch when not set. | +| Attribute | Type | Required | Description | +|----------------|---------|----------|-------------| +| `dry_run` | boolean | No | Run pipeline creation simulation, or only do static check. | +| `include_jobs` | boolean | No | If the list of jobs that would exist in a static check or pipeline simulation should be included in the response. Default: `false`. | +| `ref` | string | No | When `dry_run` is `true`, sets the branch or tag to use. Defaults to the project's default branch when not set. | Example request: @@ -112,18 +108,18 @@ Example responses: WARNING: This endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/381669) in GitLab 15.7 -and was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/403256) in 16.0. Use [`POST /projects/:id/ci/lint`](#validate-a-ci-yaml-configuration-with-a-namespace) instead. +and was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/403256) in 16.0. Use [`POST /projects/:id/ci/lint`](#validate-the-cicd-configuration-for-a-namespace) instead. Checks if CI/CD YAML configuration is valid. This endpoint validates basic CI/CD configuration syntax. It doesn't have any namespace-specific context. Access to this endpoint does not require authentication when the instance -[allows new sign ups](../user/admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups) +[allows new sign ups](../administration/settings/sign_up_restrictions.md#disable-new-sign-ups) and: -- Does not have an [allowlist or denylist](../user/admin_area/settings/sign_up_restrictions.md#allow-or-deny-sign-ups-using-specific-email-domains). -- Does not [require administrator approval for new sign ups](../user/admin_area/settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups). -- Does not have additional [sign up restrictions](../user/admin_area/settings/sign_up_restrictions.md). +- Does not have an [allowlist or denylist](../administration/settings/sign_up_restrictions.md#allow-or-deny-sign-ups-using-specific-email-domains). +- Does not [require administrator approval for new sign ups](../administration/settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups). +- Does not have additional [sign up restrictions](../administration/settings/sign_up_restrictions.md). Otherwise, authentication is required. @@ -131,11 +127,11 @@ Otherwise, authentication is required. POST /ci/lint ``` -| Attribute | Type | Required | Description | -| ---------- | ------- | -------- | -------- | -| `content` | string | yes | The CI/CD configuration content. | -| `include_merged_yaml` | boolean | no | If the [expanded CI/CD configuration](#yaml-expansion) should be included in the response. | -| `include_jobs` | boolean | no | If the list of jobs should be included in the response. This is false by default. | +| Attribute | Type | Required | Description | +|-----------------------|---------|----------|-------------| +| `content` | string | Yes | The CI/CD configuration content. | +| `include_merged_yaml` | boolean | No | If the [expanded CI/CD configuration](#yaml-expansion) should be included in the response. | +| `include_jobs` | boolean | No | If the list of jobs should be included in the response. Default: `false`. | ```shell curl --header "Content-Type: application/json" --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/ci/lint" --data '{"content": "{ \"image\": \"ruby:2.6\", \"services\": [\"postgres\"], \"before_script\": [\"bundle install\", \"bundle exec rake db:create\"], \"variables\": {\"DB_NAME\": \"postgres\"}, \"types\": [\"test\", \"deploy\", \"notify\"], \"rspec\": { \"script\": \"rake spec\", \"tags\": [\"ruby\", \"postgres\"], \"only\": [\"branches\"]}}"}' @@ -190,8 +186,6 @@ Example responses: ### YAML expansion -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29568) in GitLab 13.5. - The CI lint returns an expanded version of the configuration. The expansion does not work for CI configuration added with [`include: local`](../ci/yaml/index.md#includelocal), and the [`extends:`](../ci/yaml/index.md#extends) keyword is [not fully supported](https://gitlab.com/gitlab-org/gitlab/-/issues/258843). diff --git a/doc/api/markdown.md b/doc/api/markdown.md index e79b05ac012..42948661cbb 100644 --- a/doc/api/markdown.md +++ b/doc/api/markdown.md @@ -16,7 +16,7 @@ Available only in APIv4. FLAG: On self-managed GitLab, by default this feature is enabled and authentication is required. -To remove the requirement to authenticate, ask an administrator to +To remove the requirement to authenticate, an administrator can [disable the feature flag](../administration/feature_flags.md) named `authenticate_markdown_api`. On GitLab.com, this feature is available. diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index 19179bddb00..9123fe0dc1e 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -37,7 +37,7 @@ Supported attributes: { "approvers": [], // Deprecated in GitLab 12.3, always returns empty "approver_groups": [], // Deprecated in GitLab 12.3, always returns empty - "approvals_before_merge": 2, + "approvals_before_merge": 2, // Deprecated in GitLab 12.3, use Approval Rules instead "reset_approvals_on_push": true, "selective_code_owner_removals": false, "disable_overriding_approvers_per_merge_request": false, @@ -63,7 +63,7 @@ Supported attributes: | Attribute | Type | Required | Description | |--------------------------------------------------|-------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------| | `id` | integer or string | **{check-circle}** Yes | The ID or [URL-encoded path of a project](rest/index.md#namespaced-path-encoding). | -| `approvals_before_merge` (deprecated) | integer | **{dotted-circle}** No | How many approvals are required before a merge request can be merged. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3. | +| `approvals_before_merge` (deprecated) | integer | **{dotted-circle}** No | How many approvals are required before a merge request can be merged. [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3. Use [Approval Rules](#create-project-level-rule) instead. | | `disable_overriding_approvers_per_merge_request` | boolean | **{dotted-circle}** No | Allow or prevent overriding approvers per merge request. | | `merge_requests_author_approval` | boolean | **{dotted-circle}** No | Allow or prevent authors from self approving merge requests; `true` means authors can self approve. | | `merge_requests_disable_committers_approval` | boolean | **{dotted-circle}** No | Allow or prevent committers from self approving merge requests. | @@ -73,7 +73,7 @@ Supported attributes: ```json { - "approvals_before_merge": 2, + "approvals_before_merge": 2, // Deprecated in GitLab 12.3, use Approval Rules instead "reset_approvals_on_push": true, "selective_code_owner_removals": false, "disable_overriding_approvers_per_merge_request": false, diff --git a/doc/api/merge_trains.md b/doc/api/merge_trains.md index c041b01da04..536cf616a88 100644 --- a/doc/api/merge_trains.md +++ b/doc/api/merge_trains.md @@ -6,10 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Merge Trains API **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/36146) in GitLab 12.9. -> - Using this API you can consume [Merge Train](../ci/pipelines/merge_trains.md) entries. - -Every API call to merge trains must be authenticated with at lease the Developer [role](../user/permissions.md). +Every API call for [merge train](../ci/pipelines/merge_trains.md) must be authenticated with at lease the Developer [role](../user/permissions.md). If a user is not a member of a project and the project is private, a `GET` request on that project returns a `404` status code. @@ -31,11 +28,11 @@ GET /projects/:id/merge_trains GET /projects/:id/merge_trains?scope=complete ``` -| Attribute | Type | Required | Description | -| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `scope` | string | no | Return Merge Trains filtered by the given scope. Available scopes are `active` (to be merged) and `complete` (have been merged). | -| `sort` | string | no | Return Merge Trains sorted in `asc` or `desc` order. Default is `desc`. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `scope` | string | No | Return Merge Trains filtered by the given scope. Available scopes are `active` (to be merged) and `complete` (have been merged). | +| `sort` | string | No | Return Merge Trains sorted in `asc` or `desc` order. Default: `desc`. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/merge_trains" @@ -95,12 +92,12 @@ GET /projects/:id/merge_trains/:target_branch Supported attributes: -| Attribute | Type | Required | Description | -| --------------- | ---------------| -------- | ------------ | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `target_branch` | string | yes | The target branch of the merge train. | -| `scope` | string | no | Return Merge Trains filtered by the given scope. Available scopes are `active` (to be merged) and `complete` (have been merged). | -| `sort` | string | no | Return Merge Trains sorted in `asc` or `desc` order. Default is `desc`. | +| Attribute | Type | Required | Description | +|-----------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `target_branch` | string | Yes | The target branch of the merge train. | +| `scope` | string | No | Return Merge Trains filtered by the given scope. Available scopes are `active` (to be merged) and `complete` (have been merged). | +| `sort` | string | No | Return Merge Trains sorted in `asc` or `desc` order. Default: `desc`. | Example request: @@ -165,10 +162,10 @@ GET /projects/:id/merge_trains/merge_requests/:merge_request_iid Supported attributes: -| Attribute | Type | Required | Description | -| ------------------- | -------------- | -------- | ------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | Yes | The internal ID of the merge request. | Example request: @@ -231,13 +228,13 @@ POST /projects/:id/merge_trains/merge_requests/:merge_request_iid Supported attributes: -| Attribute | Type | Required | Description | -| ------------------------------ | -------------- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `when_pipeline_succeeds` | boolean | no | If true, the merge request is added to the merge train when the pipeline succeeds. When false or unspecified, the merge request is added directly to the merge train. | -| `sha` | string | no | If present, the SHA must match the `HEAD` of the source branch, otherwise the merge fails. | -| `squash` | boolean | no | If true, the commits are squashed into a single commit on merge. | +| Attribute | Type | Required | Description | +|--------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `merge_request_iid` | integer | Yes | The internal ID of the merge request. | +| `when_pipeline_succeeds` | boolean | No | If true, the merge request is added to the merge train when the pipeline succeeds. When false or unspecified, the merge request is added directly to the merge train. | +| `sha` | string | No | If present, the SHA must match the `HEAD` of the source branch, otherwise the merge fails. | +| `squash` | boolean | No | If true, the commits are squashed into a single commit on merge. | Example request: diff --git a/doc/api/notes.md b/doc/api/notes.md index 41b4ab7fd9a..ab8abc0973f 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -39,7 +39,7 @@ Read more on [pagination](rest/index.md#pagination). ## Rate limits To help avoid abuse, you can limit your users to a specific number of `Create` request per minute. -See [Notes rate limits](../user/admin_area/settings/rate_limit_on_notes_creation.md). +See [Notes rate limits](../administration/settings/rate_limit_on_notes_creation.md). ## Issues diff --git a/doc/api/openapi/openapi.yaml b/doc/api/openapi/openapi.yaml index e090eab1e5d..d89e33a916b 100644 --- a/doc/api/openapi/openapi.yaml +++ b/doc/api/openapi/openapi.yaml @@ -1,14 +1,7 @@ -openapi: 3.0.0 -tags: - - name: metadata - description: Metadata of the GitLab instance - - name: version - description: Version - - name: access_requests - description: Access requests for projects and groups - - name: access_tokens - description: Access tokens for projects +openapi: 3.0.1 info: + title: GitLab API + version: v4 description: | An OpenAPI definition for the GitLab REST API. Few API resources or endpoints are currently included. @@ -20,680 +13,3599 @@ info: The feature uses the current [GitLab session cookie](https://docs.gitlab.com/ee/api/index.html#session-cookie), so each request is made using your account. - Instructions for using this tool can be found in [Interactive API Documentation](https://docs.gitlab.com/ee/api/openapi/openapi_interactive.html). - - version: v4 - title: GitLab API + Instructions for using this tool can be found in [Interactive API Documentation](https://docs.gitlab.com/ee/api/openapi/openapi_interactive.html) termsOfService: 'https://about.gitlab.com/terms/' license: name: CC BY-SA 4.0 url: 'https://gitlab.com/gitlab-org/gitlab/-/blob/master/LICENSE' servers: - - url: 'https://gitlab.com/api/' +- url: https://www.gitlab.com/api/ security: - ApiKeyAuth: [] - -components: - securitySchemes: - ApiKeyAuth: - type: apiKey - in: header - name: Private-Token - +tags: +- name: badges + description: Operations about badges +- name: branches + description: Operations about branches +- name: alert_management + description: Operations about alert_managements +- name: batched_background_migrations + description: Operations about batched_background_migrations +- name: admin + description: Operations about admins +- name: migrations + description: Operations about migrations +- name: applications + description: Operations about applications +- name: avatar + description: Operations about avatars +- name: broadcast_messages + description: Operations about broadcast_messages +- name: bulk_imports + description: Operations about bulk_imports +- name: application + description: Operations about applications +- name: access_requests + description: Operations related to access requests +- name: ci_lint + description: Operations related to linting a CI config file +- name: ci_resource_groups + description: Operations to manage job concurrency with resource groups +- name: ci_variables + description: Operations related to CI/CD variables +- name: cluster_agents + description: Operations related to the GitLab agent for Kubernetes +- name: clusters + description: Operations related to clusters +- name: composer_packages + description: Operations related to Composer packages +- name: conan_packages + description: Operations related to Conan packages +- name: container_registry + description: Operations related to container registry +- name: container_registry_event + description: Operations related to container registry events +- name: dashboard_annotations + description: Operations related to dashboard annotations +- name: debian_distribution + description: Operations related to Debian Linux distributions +- name: debian_packages + description: Operations related to Debian Linux packages +- name: dependency_proxy + description: Operations to manage dependency proxy for a groups +- name: deploy_keys + description: Operations related to deploy keys +- name: deploy_tokens + description: Operations related to deploy tokens +- name: deployments + description: Operations related to deployments +- name: dora_metrics + description: Operations related to DevOps Research and Assessment (DORA) key metrics +- name: environments + description: Operations related to environments +- name: error_tracking_client_keys + description: Operations related to error tracking client keys +- name: error_tracking_project_settings + description: Operations related to error tracking project settings +- name: feature_flags_user_lists + description: Operations related to accessing GitLab feature flag user lists +- name: feature_flags + description: Operations related to feature flags +- name: features + description: Operations related to managing Flipper-based feature flags +- name: freeze_periods + description: Operations related to deploy freeze periods +- name: generic_packages + description: Operations related to Generic packages +- name: geo + description: Operations related to Geo +- name: geo_nodes + description: Operations related Geo Nodes +- name: go_proxy + description: Operations related to Go Proxy +- name: group_export + description: Operations related to exporting groups +- name: group_import + description: Operations related to importing groups +- name: group_packages + description: Operations related to group packages +- name: helm_packages + description: Operations related to Helm packages +- name: integrations + description: Operations related to integrations +- name: issue_links + description: Operations related to issue links +- name: jira_connect_subscriptions + description: Operations related to JiraConnect subscriptions +- name: maven_packages + description: Operations related to Maven packages +- name: merge_requests + description: Operations related to merge requests +- name: metadata + description: Operations related to metadata of the GitLab instance +- name: metrics_user_starred_dashboards + description: Operations related to User-starred metrics dashboards +- name: ml_model_registry + description: Operations related to Model registry +- name: npm_packages + description: Operations related to NPM packages +- name: nuget_packages + description: Operations related to Nuget packages +- name: package_files + description: Operations about package files +- name: plan_limits + description: Operations related to plan limits +- name: project_export + description: Operations related to exporting projects +- name: project_hooks + description: Operations related to project hooks +- name: project_import + description: Operations related to importing projects +- name: project_import_bitbucket + description: Operations related to importing BitBucket projects +- name: project_import_github + description: Operations related to importing GitHub projects +- name: project_packages + description: Operations related to project packages +- name: projects + description: Operations related to projects +- name: protected environments + description: Operations related to protected environments +- name: pypi_packages + description: Operations related to PyPI packages +- name: release_links + description: Operations related to release assets (links) +- name: releases + description: Operations related to releases +- name: resource_milestone_events + description: Operations about resource milestone events +- name: rpm_packages + description: Operations related to RPM packages +- name: rubygem_packages + description: Operations related to RubyGems +- name: suggestions + description: Operations related to suggestions +- name: system_hooks + description: Operations related to system hooks +- name: terraform_state + description: Operations related to Terraform state files +- name: terraform_registry + description: Operations related to the Terraform module registry +- name: unleash_api + description: Operations related to Unleash API paths: - # METADATA - /v4/metadata: - $ref: '#/metadata' - - # VERSION - /v4/version: - $ref: '#/version' - - # ACCESS REQUESTS (PROJECTS) - /v4/projects/{id}/access_requests: - $ref: '#/accessRequestsProjects' - - /v4/projects/{id}/access_requests/{user_id}/approve: - $ref: '#/accessRequestsProjectsApprove' - - /v4/projects/{id}/access_requests/{user_id}: - $ref: '#/accessRequestsProjectsDeny' - - # ACCESS REQUESTS (GROUPS) - /v4/groups/{id}/access_requests: - $ref: '#/accessRequestsGroups' - - /v4/groups/{id}/access_requests/{user_id}/approve: - $ref: '#/accessRequestsGroupsApprove' - - /v4/groups/{id}/access_requests/{user_id}: - $ref: '#/accessRequestsGroupsDeny' - - # ACCESS REQUESTS (PROJECTS) - /v4/projects/{id}/access_tokens: - $ref: '#/accessTokens' - - /v4/projects/{id}/access_tokens/{token_id}: - $ref: '#/accessTokensRevoke' - -metadata: - get: - tags: - - metadata - summary: 'Retrieve metadata information for this GitLab instance.' - operationId: 'getMetadata' - responses: - '401': - description: 'unauthorized operation' - '200': - description: 'successful operation' + /api/v4/groups/{id}/badges/{badge_id}: + get: + tags: + - badges + summary: Gets a badge of a group. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4GroupsIdBadgesBadgeId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + - name: badge_id + in: path + description: The badge ID + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Gets a badge of a group. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + put: + tags: + - badges + summary: Updates a badge of a group. + description: This feature was introduced in GitLab 10.6. + operationId: putApiV4GroupsIdBadgesBadgeId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + - name: badge_id + in: path + required: true + schema: + type: integer + format: int32 + requestBody: content: - 'application/json': + application/json: schema: - title: 'MetadataResponse' - type: 'object' properties: - version: - type: 'string' - revision: - type: 'string' - kas: - type: 'object' - properties: - enabled: - type: 'boolean' - externalUrl: - type: 'string' - nullable: true - version: - type: 'string' - nullable: true - examples: - Example: - value: - version: '15.0-pre' - revision: 'c401a659d0c' - kas: - enabled: true - externalUrl: 'grpc://gitlab.example.com:8150' - version: '15.0.0' - -version: - get: - tags: - - version - summary: 'Retrieve version information for this GitLab instance.' - operationId: 'getVersion' - responses: - '401': - description: 'unauthorized operation' - '200': - description: 'successful operation' + link_url: + type: string + description: URL of the badge link + image_url: + type: string + description: URL of the badge image + name: + type: string + description: Name for the badge + responses: + 200: + description: Updates a badge of a group. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + delete: + tags: + - badges + summary: Removes a badge from the group. + description: This feature was introduced in GitLab 10.6. + operationId: deleteApiV4GroupsIdBadgesBadgeId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + - name: badge_id + in: path + description: The badge ID + required: true + schema: + type: integer + format: int32 + responses: + 204: + description: Removes a badge from the group. + content: {} + /api/v4/groups/{id}/badges: + get: + tags: + - badges + summary: Gets a list of group badges viewable by the authenticated user. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4GroupsIdBadges + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + - name: name + in: query + description: Name for the badge + schema: + type: string + responses: + 200: + description: Gets a list of group badges viewable by the authenticated user. + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_Badge' + post: + tags: + - badges + summary: Adds a badge to a group. + description: This feature was introduced in GitLab 10.6. + operationId: postApiV4GroupsIdBadges + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + requestBody: content: - 'application/json': + application/json: schema: - title: 'VersionResponse' - type: 'object' + required: + - image_url + - link_url properties: - version: - type: 'string' - revision: - type: 'string' - examples: - Example: - value: - version: '13.3.0-pre' - revision: 'f2b05afebb0' - -#/v4/projects/{id}/access_requests -accessRequestsProjects: - get: - description: Lists access requests for a project - summary: List access requests for a project - operationId: accessRequestsProjects_get - tags: + link_url: + type: string + description: URL of the badge link + image_url: + type: string + description: URL of the badge image + name: + type: string + description: Name for the badge + required: true + responses: + 201: + description: Adds a badge to a group. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + /api/v4/groups/{id}/badges/render: + get: + tags: + - badges + summary: Preview a badge from a group. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4GroupsIdBadgesRender + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user. + required: true + schema: + type: string + - name: link_url + in: query + description: URL of the badge link + required: true + schema: + type: string + - name: image_url + in: query + description: URL of the badge image + required: true + schema: + type: string + responses: + 200: + description: Preview a badge from a group. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BasicBadgeDetails' + /api/v4/groups/{id}/access_requests/{user_id}: + delete: + tags: + - access_requests + summary: Denies an access request for the given user. + description: This feature was introduced in GitLab 8.11. + operationId: deleteApiV4GroupsIdAccessRequestsUserId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user + required: true + schema: + type: string + - name: user_id + in: path + description: The user ID of the access requester + required: true + schema: + type: integer + format: int32 + responses: + 204: + description: Denies an access request for the given user. + content: {} + /api/v4/groups/{id}/access_requests/{user_id}/approve: + put: + tags: - access_requests - parameters: + summary: Approves an access request for the given user. + description: This feature was introduced in GitLab 8.11. + operationId: putApiV4GroupsIdAccessRequestsUserIdApprove + parameters: - name: id in: path - description: The ID or URL-encoded path of the project owned by the authenticated user. + description: The ID or URL-encoded path of the group owned by the authenticated + user + required: true + schema: + type: string + - name: user_id + in: path + description: The user ID of the access requester required: true schema: - oneOf: - - type: integer - - type: string - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: integer + format: int32 + requestBody: content: application/json: schema: - title: ProjectAccessResponse - type: object properties: - id: + access_level: type: integer - usename: - type: string - name: - type: string - state: - type: string - created_at: - type: string - requested_at: - type: string - example: - - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - - 'id': 2 - 'username': 'john_doe' - 'name': 'John Doe' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - post: - description: Requests access for the authenticated user to a project - summary: Requests access for the authenticated user to a project - operationId: accessRequestsProjects_post - tags: + description: 'A valid access level (defaults: `30`, the Developer + role)' + format: int32 + default: 30 + responses: + 200: + description: successful operation + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + successfull_response: + example: + id: 1 + username: raymond_smith + name: Raymond Smith + state: active + created_at: 2012-10-22T14:13:35Z + access_level: 20 + /api/v4/groups/{id}/access_requests: + get: + tags: + - access_requests + summary: Gets a list of access requests for a group. + description: This feature was introduced in GitLab 8.11. + operationId: getApiV4GroupsIdAccessRequests + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user + required: true + schema: + type: string + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + responses: + 200: + description: Gets a list of access requests for a group. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + post: + tags: - access_requests - parameters: + summary: Requests access for the authenticated user to a group. + description: This feature was introduced in GitLab 8.11. + operationId: postApiV4GroupsIdAccessRequests + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the group owned by the authenticated + user + required: true + schema: + type: string + responses: + 200: + description: successful operation + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + successfull_response: + example: + id: 1 + username: raymond_smith + name: Raymond Smith + state: active + created_at: 2012-10-22T14:13:35Z + access_level: 20 + /api/v4/projects/{id}/repository/merged_branches: + delete: + tags: + - branches + description: Delete all merged branches + operationId: deleteApiV4ProjectsIdRepositoryMergedBranches + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + responses: + 202: + description: 202 Accepted + content: {} + 404: + description: 404 Project Not Found + content: {} + /api/v4/projects/{id}/repository/branches/{branch}: + get: + tags: + - branches + description: Get a single repository branch + operationId: getApiV4ProjectsIdRepositoryBranchesBranch + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: branch + in: path + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Get a single repository branch + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Branch' + 404: + description: Branch Not Found + content: {} + delete: + tags: + - branches + description: Delete a branch + operationId: deleteApiV4ProjectsIdRepositoryBranchesBranch + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: branch + in: path + description: The name of the branch + required: true + schema: + type: string + responses: + 204: + description: Delete a branch + content: {} + 404: + description: Branch Not Found + content: {} + head: + tags: + - branches + description: Check if a branch exists + operationId: headApiV4ProjectsIdRepositoryBranchesBranch + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: branch + in: path + description: The name of the branch + required: true + schema: + type: string + responses: + 204: + description: No Content + content: {} + 404: + description: Not Found + content: {} + /api/v4/projects/{id}/repository/branches: + get: + tags: + - branches + description: Get a project repository branches + operationId: getApiV4ProjectsIdRepositoryBranches + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + - name: search + in: query + description: Return list of branches matching the search criteria + schema: + type: string + - name: regex + in: query + description: Return list of branches matching the regex + schema: + type: string + - name: sort + in: query + description: Return list of branches sorted by the given field + schema: + type: string + enum: + - name_asc + - updated_asc + - updated_desc + - name: page_token + in: query + description: Name of branch to start the pagination from + schema: + type: string + responses: + 200: + description: Get a project repository branches + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_Branch' + 404: + description: 404 Project Not Found + content: {} + post: + tags: + - branches + description: Create branch + operationId: postApiV4ProjectsIdRepositoryBranches + parameters: - name: id in: path - description: The ID or URL-encoded path of the project owned by the authenticated user. + description: The ID or URL-encoded path of the project required: true schema: - oneOf: - - type: integer - - type: string - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: string + requestBody: content: application/json: schema: - title: ProjectAccessRequest - type: object + required: + - branch + - ref properties: - id: - type: integer - usename: + branch: type: string - name: + description: The name of the branch + ref: type: string - state: + description: Create branch from commit sha or existing branch + required: true + responses: + 201: + description: Create branch + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Branch' + 400: + description: Failed to create branch + content: {} + /api/v4/projects/{id}/repository/branches/{branch}/unprotect: + put: + tags: + - branches + description: Unprotect a single branch + operationId: putApiV4ProjectsIdRepositoryBranchesBranchUnprotect + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: branch + in: path + description: The name of the branch + required: true + schema: + type: string + responses: + 200: + description: Unprotect a single branch + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Branch' + 404: + description: 404 Project Not Found + content: {} + /api/v4/projects/{id}/repository/branches/{branch}/protect: + put: + tags: + - branches + description: Protect a single branch + operationId: putApiV4ProjectsIdRepositoryBranchesBranchProtect + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: branch + in: path + description: The name of the branch + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + properties: + developers_can_push: + type: boolean + description: Flag if developers can push to that branch + developers_can_merge: + type: boolean + description: Flag if developers can merge to that branch + responses: + 200: + description: Protect a single branch + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Branch' + 404: + description: 404 Branch Not Found + content: {} + /api/v4/projects/{id}/badges/{badge_id}: + get: + tags: + - badges + summary: Gets a badge of a project. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4ProjectsIdBadgesBadgeId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user. + required: true + schema: + type: string + - name: badge_id + in: path + description: The badge ID + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Gets a badge of a project. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + put: + tags: + - badges + summary: Updates a badge of a project. + description: This feature was introduced in GitLab 10.6. + operationId: putApiV4ProjectsIdBadgesBadgeId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user. + required: true + schema: + type: string + - name: badge_id + in: path + required: true + schema: + type: integer + format: int32 + requestBody: + content: + application/json: + schema: + properties: + link_url: type: string - created_at: + description: URL of the badge link + image_url: type: string - requested_at: + description: URL of the badge image + name: type: string - example: - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - -#/v4/projects/{id}/access_requests/{user_id}/approve -accessRequestsProjectsApprove: - put: - description: Approves access for the authenticated user to a project - summary: Approves access for the authenticated user to a project - operationId: accessRequestsProjectsApprove_put - tags: - - access_requests - parameters: + description: Name for the badge + responses: + 200: + description: Updates a badge of a project. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + delete: + tags: + - badges + summary: Removes a badge from the project. + description: This feature was introduced in GitLab 10.6. + operationId: deleteApiV4ProjectsIdBadgesBadgeId + parameters: - name: id in: path - description: The ID or URL-encoded path of the project owned by the authenticated user. + description: The ID or URL-encoded path of the project owned by the authenticated + user. required: true schema: - oneOf: - - type: integer - - type: string - - name: user_id + type: string + - name: badge_id in: path - description: The userID of the access requester + description: The badge ID required: true schema: type: integer - - name: access_level + format: int32 + responses: + 204: + description: Removes a badge from the project. + content: {} + /api/v4/projects/{id}/badges: + get: + tags: + - badges + summary: Gets a list of project badges viewable by the authenticated user. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4ProjectsIdBadges + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user. + required: true + schema: + type: string + - name: page in: query - description: A valid project access level. 0 = no access , 10 = guest, 20 = reporter, 30 = developer, 40 = Maintainer. Default is 30.' - required: false + description: Current page number schema: - enum: [0, 10, 20, 30, 40] - default: 30 type: integer - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + - name: name + in: query + description: Name for the badge + schema: + type: string + responses: + 200: + description: Gets a list of project badges viewable by the authenticated + user. + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_Badge' + post: + tags: + - badges + summary: Adds a badge to a project. + description: This feature was introduced in GitLab 10.6. + operationId: postApiV4ProjectsIdBadges + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user. + required: true + schema: + type: string + requestBody: content: application/json: schema: - title: ProjectAccessApprove - type: object + required: + - image_url + - link_url properties: - id: - type: integer - usename: + link_url: type: string - name: - type: string - state: + description: URL of the badge link + image_url: type: string - created_at: + description: URL of the badge image + name: type: string - access_level: - type: integer - example: - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'access_level': 20 - -#/v4/projects/{id}/access_requests/{user_id} -accessRequestsProjectsDeny: - delete: - description: Denies a project access request for the given user - summary: Denies a project access request for the given user - operationId: accessRequestProjectsDeny_delete - tags: + description: Name for the badge + required: true + responses: + 201: + description: Adds a badge to a project. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Badge' + /api/v4/projects/{id}/badges/render: + get: + tags: + - badges + summary: Preview a badge from a project. + description: This feature was introduced in GitLab 10.6. + operationId: getApiV4ProjectsIdBadgesRender + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user. + required: true + schema: + type: string + - name: link_url + in: query + description: URL of the badge link + required: true + schema: + type: string + - name: image_url + in: query + description: URL of the badge image + required: true + schema: + type: string + responses: + 200: + description: Preview a badge from a project. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BasicBadgeDetails' + /api/v4/projects/{id}/access_requests/{user_id}: + delete: + tags: - access_requests - parameters: + summary: Denies an access request for the given user. + description: This feature was introduced in GitLab 8.11. + operationId: deleteApiV4ProjectsIdAccessRequestsUserId + parameters: - name: id in: path - description: The ID or URL-encoded path of the project owned by the authenticated user. + description: The ID or URL-encoded path of the project owned by the authenticated + user required: true schema: - oneOf: - - type: integer - - type: string + type: string - name: user_id in: path description: The user ID of the access requester required: true schema: type: integer - responses: # Does anything go here? Markdown doc does not list a response. - '401': - description: Unauthorized operation - '200': - description: Successful operation - -#/v4/groups/{id}/access_requests -accessRequestsGroups: - get: - description: List access requests for a group - summary: List access requests for a group - operationId: accessRequestsGroups_get - tags: + format: int32 + responses: + 204: + description: Denies an access request for the given user. + content: {} + /api/v4/projects/{id}/access_requests/{user_id}/approve: + put: + tags: - access_requests - parameters: + summary: Approves an access request for the given user. + description: This feature was introduced in GitLab 8.11. + operationId: putApiV4ProjectsIdAccessRequestsUserIdApprove + parameters: - name: id in: path - description: The ID or URL-encoded path of the group owned by the authenticated user. + description: The ID or URL-encoded path of the project owned by the authenticated + user + required: true + schema: + type: string + - name: user_id + in: path + description: The user ID of the access requester required: true schema: - oneOf: - - type: integer - - type: string - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: integer + format: int32 + requestBody: content: application/json: schema: - title: GroupAccessResponse - type: object properties: - id: + access_level: type: integer - usename: - type: string - name: - type: string - state: - type: string - created_at: - type: string - requested_at: - type: string - example: - - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - - 'id': 2 - 'username': 'john_doe' - 'name': 'John Doe' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - post: - description: Requests access for the authenticated user to a group - summary: Requests access for the authenticated user to a group - operationId: accessRequestsGroups_post - tags: + description: 'A valid access level (defaults: `30`, the Developer + role)' + format: int32 + default: 30 + responses: + 200: + description: successful operation + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + successfull_response: + example: + id: 1 + username: raymond_smith + name: Raymond Smith + state: active + created_at: 2012-10-22T14:13:35Z + access_level: 20 + /api/v4/projects/{id}/access_requests: + get: + tags: - access_requests - parameters: + summary: Gets a list of access requests for a project. + description: This feature was introduced in GitLab 8.11. + operationId: getApiV4ProjectsIdAccessRequests + parameters: - name: id in: path - description: The ID or URL-encoded path of the group owned by the authenticated user. + description: The ID or URL-encoded path of the project owned by the authenticated + user required: true schema: - oneOf: - - type: integer - - type: string - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: string + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + responses: + 200: + description: Gets a list of access requests for a project. + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + post: + tags: + - access_requests + summary: Requests access for the authenticated user to a project. + description: This feature was introduced in GitLab 8.11. + operationId: postApiV4ProjectsIdAccessRequests + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project owned by the authenticated + user + required: true + schema: + type: string + responses: + 200: + description: successful operation + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_AccessRequester' + successfull_response: + example: + id: 1 + username: raymond_smith + name: Raymond Smith + state: active + created_at: 2012-10-22T14:13:35Z + access_level: 20 + /api/v4/projects/{id}/alert_management_alerts/{alert_iid}/metric_images/{metric_image_id}: + put: + tags: + - alert_management + description: Update a metric image for an alert + operationId: putApiV4ProjectsIdAlertManagementAlertsAlertIidMetricImagesMetricImageId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: alert_iid + in: path + description: The IID of the Alert + required: true + schema: + type: integer + format: int32 + - name: metric_image_id + in: path + description: The ID of metric image + required: true + schema: + type: integer + format: int32 + requestBody: content: - application/json: + multipart/form-data: schema: - title: GroupAccessRequest - type: object properties: - id: - type: integer - usename: + url: type: string - name: + description: The url to view more metric info + url_text: type: string - state: + description: A description of the image or URL + responses: + 200: + description: Update a metric image for an alert + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_MetricImage' + 403: + description: Forbidden + content: {} + 422: + description: Unprocessable entity + content: {} + delete: + tags: + - alert_management + description: Remove a metric image for an alert + operationId: deleteApiV4ProjectsIdAlertManagementAlertsAlertIidMetricImagesMetricImageId + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: alert_iid + in: path + description: The IID of the Alert + required: true + schema: + type: integer + format: int32 + - name: metric_image_id + in: path + description: The ID of metric image + required: true + schema: + type: integer + format: int32 + responses: + 204: + description: Remove a metric image for an alert + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_MetricImage' + 403: + description: Forbidden + content: {} + 422: + description: Unprocessable entity + content: {} + /api/v4/projects/{id}/alert_management_alerts/{alert_iid}/metric_images: + get: + tags: + - alert_management + description: Metric Images for alert + operationId: getApiV4ProjectsIdAlertManagementAlertsAlertIidMetricImages + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: alert_iid + in: path + description: The IID of the Alert + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Metric Images for alert + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_MetricImage' + 404: + description: Not found + content: {} + post: + tags: + - alert_management + description: Upload a metric image for an alert + operationId: postApiV4ProjectsIdAlertManagementAlertsAlertIidMetricImages + parameters: + - name: id + in: path + description: The ID or URL-encoded path of the project + required: true + schema: + type: string + - name: alert_iid + in: path + description: The IID of the Alert + required: true + schema: + type: integer + format: int32 + requestBody: + content: + multipart/form-data: + schema: + required: + - file + properties: + file: type: string - created_at: + description: The image file to be uploaded + format: binary + url: type: string - requested_at: + description: The url to view more metric info + url_text: type: string - example: - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'requested_at': '2012-10-22T14:13:35Z' - -#/v4/groups/{id}/access_requests/{user_id}/approve -accessRequestsGroupsApprove: - put: - description: Approves access for the authenticated user to a group - summary: Approves access for the authenticated user to a group - operationId: accessRequestsGroupsApprove_put - tags: - - access_requests - parameters: + description: A description of the image or URL + required: true + responses: + 200: + description: Upload a metric image for an alert + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_MetricImage' + 403: + description: Forbidden + content: {} + /api/v4/projects/{id}/alert_management_alerts/{alert_iid}/metric_images/authorize: + post: + tags: + - alert_management + description: Workhorse authorize metric image file upload + operationId: postApiV4ProjectsIdAlertManagementAlertsAlertIidMetricImagesAuthorize + parameters: - name: id in: path - description: The ID or URL-encoded path of the group owned by the authenticated user. + description: The ID or URL-encoded path of the project required: true schema: - oneOf: - - type: integer - - type: string - - name: user_id + type: string + - name: alert_iid in: path - description: The userID of the access requester + description: The IID of the Alert required: true schema: type: integer - - name: access_level + format: int32 + responses: + 200: + description: Workhorse authorize metric image file upload + content: {} + 403: + description: Forbidden + content: {} + /api/v4/admin/batched_background_migrations/{id}: + get: + tags: + - batched_background_migrations + description: Retrieve a batched background migration + operationId: getApiV4AdminBatchedBackgroundMigrationsId + parameters: + - name: database in: query - description: A valid group access level. 0 = no access , 10 = Guest, 20 = Reporter, 30 = Developer, 40 = Maintainer, 50 = Owner. Default is 30. - required: false + description: The name of the database + schema: + type: string + default: main + enum: + - main + - ci + - embedding + - main_clusterwide + - geo + - name: id + in: path + description: The batched background migration id + required: true schema: - enum: [0, 10, 20, 30, 40, 50] - default: 30 type: integer - responses: - '401': - description: Unauthorized operation - '200': - description: Successful operation + format: int32 + responses: + 200: + description: Retrieve a batched background migration + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BatchedBackgroundMigration' + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + 404: + description: 404 Not found + content: {} + /api/v4/admin/batched_background_migrations: + get: + tags: + - batched_background_migrations + description: Get the list of batched background migrations + operationId: getApiV4AdminBatchedBackgroundMigrations + parameters: + - name: database + in: query + description: The name of the database, the default `main` + schema: + type: string + default: main + enum: + - main + - ci + - embedding + - main_clusterwide + - geo + responses: + 200: + description: Get the list of batched background migrations + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_BatchedBackgroundMigration' + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + /api/v4/admin/batched_background_migrations/{id}/resume: + put: + tags: + - batched_background_migrations + description: Resume a batched background migration + operationId: putApiV4AdminBatchedBackgroundMigrationsIdResume + parameters: + - name: id + in: path + description: The batched background migration id + required: true + schema: + type: integer + format: int32 + requestBody: content: application/json: schema: - title: GroupAccessApprove - type: object properties: - id: - type: integer - usename: + database: type: string - name: + description: The name of the database + default: main + enum: + - main + - ci + - embedding + - main_clusterwide + - geo + responses: + 200: + description: Resume a batched background migration + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BatchedBackgroundMigration' + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + 404: + description: 404 Not found + content: {} + 422: + description: You can resume only `paused` batched background migrations. + content: {} + /api/v4/admin/batched_background_migrations/{id}/pause: + put: + tags: + - batched_background_migrations + description: Pause a batched background migration + operationId: putApiV4AdminBatchedBackgroundMigrationsIdPause + parameters: + - name: id + in: path + description: The batched background migration id + required: true + schema: + type: integer + format: int32 + requestBody: + content: + application/json: + schema: + properties: + database: type: string - state: + description: The name of the database + default: main + enum: + - main + - ci + - embedding + - main_clusterwide + - geo + responses: + 200: + description: Pause a batched background migration + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BatchedBackgroundMigration' + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + 404: + description: 404 Not found + content: {} + 422: + description: You can pause only `active` batched background migrations. + content: {} + /api/v4/admin/ci/variables/{key}: + get: + tags: + - ci_variables + description: Get the details of a specific instance-level variable + operationId: getApiV4AdminCiVariablesKey + parameters: + - name: key + in: path + description: The key of a variable + required: true + schema: + type: string + responses: + 200: + description: Get the details of a specific instance-level variable + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Ci_Variable' + 404: + description: Instance Variable Not Found + content: {} + put: + tags: + - ci_variables + description: Update an instance-level variable + operationId: putApiV4AdminCiVariablesKey + parameters: + - name: key + in: path + description: The key of a variable + required: true + schema: + type: string + requestBody: + content: + application/json: + schema: + properties: + value: type: string - created_at: + description: The value of a variable + protected: + type: boolean + description: Whether the variable is protected + masked: + type: boolean + description: Whether the variable is masked + raw: + type: boolean + description: Whether the variable will be expanded + variable_type: type: string - access_level: - type: integer - example: - 'id': 1 - 'username': 'raymond_smith' - 'name': 'Raymond Smith' - 'state': 'active' - 'created_at': '2012-10-22T14:13:35Z' - 'access_level': 20 - -#/v4/groups/{id}/access_requests/{user_id} -accessRequestsGroupsDeny: - delete: - description: Denies a group access request for the given user - summary: Denies a group access request for the given user - operationId: accessRequestsGroupsDeny_delete - tags: - - access_requests - parameters: - - name: id + description: 'The type of a variable. Available types are: env_var + (default) and file' + enum: + - env_var + - file + responses: + 200: + description: Update an instance-level variable + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Ci_Variable' + 404: + description: Instance Variable Not Found + content: {} + delete: + tags: + - ci_variables + description: Delete an existing instance-level variable + operationId: deleteApiV4AdminCiVariablesKey + parameters: + - name: key in: path - description: The ID or URL-encoded path of the group owned by the authenticated user. + description: The key of a variable required: true schema: - oneOf: - - type: integer - - type: string - - name: user_id + type: string + responses: + 204: + description: Delete an existing instance-level variable + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Ci_Variable' + 404: + description: Instance Variable Not Found + content: {} + /api/v4/admin/ci/variables: + get: + tags: + - ci_variables + description: List all instance-level variables + operationId: getApiV4AdminCiVariables + parameters: + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + responses: + 200: + description: List all instance-level variables + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Ci_Variable' + post: + tags: + - ci_variables + description: Create a new instance-level variable + operationId: postApiV4AdminCiVariables + requestBody: + content: + application/json: + schema: + required: + - key + - value + properties: + key: + type: string + description: The key of the variable. Max 255 characters + value: + type: string + description: The value of a variable + protected: + type: boolean + description: Whether the variable is protected + masked: + type: boolean + description: Whether the variable is masked + raw: + type: boolean + description: Whether the variable will be expanded + variable_type: + type: string + description: 'The type of a variable. Available types are: env_var + (default) and file' + enum: + - env_var + - file + required: true + responses: + 201: + description: Create a new instance-level variable + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Ci_Variable' + 400: + description: 400 Bad Request + content: {} + /api/v4/admin/databases/{database_name}/dictionary/tables/{table_name}: + get: + tags: + - admin + description: Retrieve dictionary details + operationId: getApiV4AdminDatabasesDatabaseNameDictionaryTablesTableName + parameters: + - name: database_name + in: path + description: The database name + required: true + schema: + type: string + enum: + - main + - ci + - name: table_name in: path - description: The userID of the access requester + description: The table name + required: true + schema: + type: string + responses: + 200: + description: Retrieve dictionary details + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Dictionary_Table' + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + 404: + description: 404 Not found + content: {} + /api/v4/admin/clusters/{cluster_id}: + get: + tags: + - clusters + summary: Get a single instance cluster + description: This feature was introduced in GitLab 13.2. Returns a single instance + cluster. + operationId: getApiV4AdminClustersClusterId + parameters: + - name: cluster_id + in: path + description: The cluster ID required: true schema: type: integer - responses: # Does anything go here? Markdown doc does not list a response. - '401': - description: Unauthorized operation - '200': - description: Successful operation -#/v4/projects/{id}/access_tokens -accessTokens: - get: - description: Lists access tokens for a project - summary: List access tokens for a project - operationId: accessTokens_get - tags: - - access_tokens - parameters: - - name: id + format: int32 + responses: + 200: + description: Get a single instance cluster + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Cluster' + 403: + description: Forbidden + content: {} + 404: + description: Not found + content: {} + put: + tags: + - clusters + summary: Edit instance cluster + description: This feature was introduced in GitLab 13.2. Updates an existing + instance cluster. + operationId: putApiV4AdminClustersClusterId + parameters: + - name: cluster_id in: path - description: The ID or URL-encoded path of the project + description: The cluster ID required: true schema: - oneOf: - - type: integer - - type: string - responses: - '404': - description: Not Found - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: integer + format: int32 + requestBody: content: application/json: schema: - title: AccessTokenList - type: object properties: - user_id: - type: integer - scopes: - type: array name: type: string - expires_at: - type: date - id: + description: Cluster name + enabled: + type: boolean + description: Enable or disable Gitlab's connection to your Kubernetes + cluster + environment_scope: + type: string + description: The associated environment to the cluster + namespace_per_environment: + type: boolean + description: Deploy each environment to a separate Kubernetes namespace + default: true + domain: + type: string + description: Cluster base domain + management_project_id: type: integer - active: + description: The ID of the management project + format: int32 + managed: type: boolean - created_at: - type: date - revoked: + description: Determines if GitLab will manage namespaces and service + accounts for this cluster + platform_kubernetes_attributes[api_url]: + type: string + description: URL to access the Kubernetes API + platform_kubernetes_attributes[token]: + type: string + description: Token to authenticate against Kubernetes + platform_kubernetes_attributes[ca_cert]: + type: string + description: TLS certificate (needed if API is using a self-signed + TLS certificate) + platform_kubernetes_attributes[namespace]: + type: string + description: Unique namespace related to Project + responses: + 200: + description: Edit instance cluster + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Cluster' + 400: + description: Validation error + content: {} + 403: + description: Forbidden + content: {} + 404: + description: Not found + content: {} + delete: + tags: + - clusters + summary: Delete instance cluster + description: This feature was introduced in GitLab 13.2. Deletes an existing + instance cluster. Does not remove existing resources within the connected + Kubernetes cluster. + operationId: deleteApiV4AdminClustersClusterId + parameters: + - name: cluster_id + in: path + description: The cluster ID + required: true + schema: + type: integer + format: int32 + responses: + 204: + description: Delete instance cluster + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Cluster' + 403: + description: Forbidden + content: {} + 404: + description: Not found + content: {} + /api/v4/admin/clusters/add: + post: + tags: + - clusters + summary: Add existing instance cluster + description: This feature was introduced in GitLab 13.2. Adds an existing Kubernetes + instance cluster. + operationId: postApiV4AdminClustersAdd + requestBody: + content: + application/json: + schema: + required: + - name + - platform_kubernetes_attributes[api_url] + - platform_kubernetes_attributes[token] + properties: + name: + type: string + description: Cluster name + enabled: + type: boolean + description: Determines if cluster is active or not, defaults to + true + default: true + environment_scope: + type: string + description: The associated environment to the cluster + default: '*' + namespace_per_environment: + type: boolean + description: Deploy each environment to a separate Kubernetes namespace + default: true + domain: + type: string + description: Cluster base domain + management_project_id: + type: integer + description: The ID of the management project + format: int32 + managed: type: boolean - example: - 'user_id': 141 - 'scopes': ['api'] - 'name': 'token' - 'expires_at': '2022-01-31' - 'id': 42 - 'active': true - 'created_at': '2021-01-20T14:13:35Z' - 'revoked': false - post: - description: Creates an access token for a project - summary: Creates an access token for a project - operationId: accessTokens_post - tags: - - access_tokens - parameters: + description: Determines if GitLab will manage namespaces and service + accounts for this cluster, defaults to true + default: true + platform_kubernetes_attributes[api_url]: + type: string + description: URL to access the Kubernetes API + platform_kubernetes_attributes[token]: + type: string + description: Token to authenticate against Kubernetes + platform_kubernetes_attributes[ca_cert]: + type: string + description: TLS certificate (needed if API is using a self-signed + TLS certificate) + platform_kubernetes_attributes[namespace]: + type: string + description: Unique namespace related to Project + platform_kubernetes_attributes[authorization_type]: + type: string + description: Cluster authorization type, defaults to RBAC + default: rbac + enum: + - unknown_authorization + - rbac + - abac + required: true + responses: + 201: + description: Add existing instance cluster + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Cluster' + 400: + description: Validation error + content: {} + 403: + description: Forbidden + content: {} + 404: + description: Not found + content: {} + /api/v4/admin/clusters: + get: + tags: + - clusters + summary: List instance clusters + description: This feature was introduced in GitLab 13.2. Returns a list of instance + clusters. + operationId: getApiV4AdminClusters + responses: + 200: + description: List instance clusters + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_Cluster' + 403: + description: Forbidden + content: {} + /api/v4/admin/migrations/{timestamp}/mark: + post: + tags: + - migrations + description: Mark the migration as successfully executed + operationId: postApiV4AdminMigrationsTimestampMark + parameters: + - name: timestamp + in: path + description: The migration version timestamp + required: true + schema: + type: integer + format: int32 + requestBody: + content: + application/json: + schema: + properties: + database: + type: string + description: The name of the database + default: main + enum: + - main + - ci + - embedding + - main_clusterwide + - geo + responses: + 201: + description: 201 Created + content: {} + 401: + description: 401 Unauthorized + content: {} + 403: + description: 403 Forbidden + content: {} + 404: + description: 404 Not found + content: {} + 422: + description: You can mark only pending migrations + content: {} + /api/v4/applications/{id}: + delete: + tags: + - applications + summary: Delete an application + description: Delete a specific application + operationId: deleteApiV4ApplicationsId + parameters: - name: id in: path - description: The ID or URL-encoded path of the project + description: The ID of the application (not the application_id) required: true schema: - oneOf: - - type: integer - - type: string - - name: name + type: integer + format: int32 + responses: + 204: + description: Delete an application + content: {} + /api/v4/applications: + get: + tags: + - applications + summary: Get applications + description: List all registered applications + operationId: getApiV4Applications + responses: + 200: + description: Get applications + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_Application' + post: + tags: + - applications + summary: Create a new application + description: This feature was introduced in GitLab 10.5 + operationId: postApiV4Applications + requestBody: + content: + application/json: + schema: + required: + - name + - redirect_uri + - scopes + properties: + name: + type: string + description: Name of the application. + redirect_uri: + type: string + description: Redirect URI of the application. + scopes: + type: string + description: |- + Scopes of the application. You can specify multiple scopes by separating\ + each scope using a space + confidential: + type: boolean + description: |- + The application is used where the client secret can be kept confidential. Native mobile apps \ + and Single Page Apps are considered non-confidential. Defaults to true if not supplied + default: true + required: true + responses: + 200: + description: Create a new application + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_ApplicationWithSecret' + /api/v4/avatar: + get: + tags: + - avatar + description: Return avatar url for a user + operationId: getApiV4Avatar + parameters: + - name: email in: query - description: The name of the project access token + description: Public email address of the user required: true schema: type: string - - name: scopes + - name: size in: query - description: Defines read and write permissions for the token + description: Single pixel dimension for Gravatar images + schema: + type: integer + format: int32 + responses: + 200: + description: Return avatar url for a user + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Avatar' + /api/v4/broadcast_messages/{id}: + get: + tags: + - broadcast_messages + summary: Get a specific broadcast message + description: This feature was introduced in GitLab 8.12. + operationId: getApiV4BroadcastMessagesId + parameters: + - name: id + in: path + description: Broadcast message ID required: true schema: - type: array - items: - type: string - enum: - [ - 'api', - 'read_api', - 'read_registry', - 'write_registry', - 'read_repository', - 'write_repository', - ] - - name: expires_at - in: query - description: Date when the token expires. Time of day is Midnight UTC of that date. - required: false - schema: - type: date - responses: - '404': - description: Not Found - '401': - description: Unauthorized operation - '200': - description: Successful operation + type: integer + format: int32 + responses: + 200: + description: Get a specific broadcast message + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BroadcastMessage' + put: + tags: + - broadcast_messages + summary: Update a broadcast message + description: This feature was introduced in GitLab 8.12. + operationId: putApiV4BroadcastMessagesId + parameters: + - name: id + in: path + description: Broadcast message ID + required: true + schema: + type: integer + format: int32 + requestBody: content: application/json: schema: - title: AccessTokenList - type: object properties: - user_id: - type: integer - scopes: + message: + type: string + description: Message to display + starts_at: + type: string + description: Starting time + format: date-time + ends_at: + type: string + description: Ending time + format: date-time + color: + type: string + description: Background color + font: + type: string + description: Foreground color + target_access_levels: type: array - name: + description: Target user roles + items: + type: integer + format: int32 + enum: + - 10 + - 20 + - 30 + - 40 + - 50 + target_path: type: string - expires_at: - type: date - id: - type: integer - active: - type: boolean - created_at: - type: date - revoked: + description: Target path + broadcast_type: + type: string + description: Broadcast Type + enum: + - banner + - notification + dismissable: type: boolean - token: - type: string - example: - 'user_id': 166 - 'scopes': ['api', 'read_repository'] - 'name': 'test' - 'expires_at': '2022-01-31' - 'id': 58 - 'active': true - 'created_at': '2021-01-20T14:13:35Z' - 'revoked': false - 'token': 'D4y...Wzr' - -#/v4/projects/{id}/access_tokens/{token_id} -accessTokensRevoke: - delete: - description: Revokes an access token - summary: Revokes an access token - operationId: accessTokens_delete - tags: - - access_tokens - parameters: + description: Is dismissable + responses: + 200: + description: Update a broadcast message + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BroadcastMessage' + delete: + tags: + - broadcast_messages + summary: Delete a broadcast message + description: This feature was introduced in GitLab 8.12. + operationId: deleteApiV4BroadcastMessagesId + parameters: - name: id in: path - description: The ID or URL-encoded path of the project + description: Broadcast message ID + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Delete a broadcast message + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BroadcastMessage' + /api/v4/broadcast_messages: + get: + tags: + - broadcast_messages + summary: Get all broadcast messages + description: This feature was introduced in GitLab 8.12. + operationId: getApiV4BroadcastMessages + parameters: + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + responses: + 200: + description: Get all broadcast messages + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BroadcastMessage' + post: + tags: + - broadcast_messages + summary: Create a broadcast message + description: This feature was introduced in GitLab 8.12. + operationId: postApiV4BroadcastMessages + requestBody: + content: + application/json: + schema: + required: + - message + properties: + message: + type: string + description: Message to display + starts_at: + type: string + description: Starting time + format: date-time + ends_at: + type: string + description: Ending time + format: date-time + color: + type: string + description: Background color + font: + type: string + description: Foreground color + target_access_levels: + type: array + description: Target user roles + items: + type: integer + format: int32 + enum: + - 10 + - 20 + - 30 + - 40 + - 50 + target_path: + type: string + description: Target path + broadcast_type: + type: string + description: Broadcast type. Defaults to banner + enum: + - banner + - notification + dismissable: + type: boolean + description: Is dismissable + required: true + responses: + 201: + description: Create a broadcast message + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BroadcastMessage' + /api/v4/bulk_imports/{import_id}/entities/{entity_id}: + get: + tags: + - bulk_imports + summary: Get GitLab Migration entity details + description: This feature was introduced in GitLab 14.1. + operationId: getApiV4BulkImportsImportIdEntitiesEntityId + parameters: + - name: import_id + in: path + description: The ID of user's GitLab Migration + required: true + schema: + type: integer + format: int32 + - name: entity_id + in: path + description: The ID of GitLab Migration entity required: true schema: - oneOf: - - type: integer - - type: string - - name: token_id + type: integer + format: int32 + responses: + 200: + description: Get GitLab Migration entity details + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BulkImports' + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 503: + description: Service unavailable + content: {} + /api/v4/bulk_imports/{import_id}/entities: + get: + tags: + - bulk_imports + summary: List GitLab Migration entities + description: This feature was introduced in GitLab 14.1. + operationId: getApiV4BulkImportsImportIdEntities + parameters: + - name: import_id in: path - description: The ID of the project access token + description: The ID of user's GitLab Migration required: true schema: - oneOf: - - type: integer - - type: string - responses: - '400': - description: Bad Request - '404': - description: Not Found - '204': - description: No content if successfully revoked + type: integer + format: int32 + - name: status + in: query + description: Return import entities with specified status + schema: + type: string + enum: + - created + - started + - finished + - timeout + - failed + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + responses: + 200: + description: List GitLab Migration entities + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_BulkImports' + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 503: + description: Service unavailable + content: {} + /api/v4/bulk_imports/{import_id}: + get: + tags: + - bulk_imports + summary: Get GitLab Migration details + description: This feature was introduced in GitLab 14.1. + operationId: getApiV4BulkImportsImportId + parameters: + - name: import_id + in: path + description: The ID of user's GitLab Migration + required: true + schema: + type: integer + format: int32 + responses: + 200: + description: Get GitLab Migration details + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BulkImport' + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 503: + description: Service unavailable + content: {} + /api/v4/bulk_imports/entities: + get: + tags: + - bulk_imports + summary: List all GitLab Migrations' entities + description: This feature was introduced in GitLab 14.1. + operationId: getApiV4BulkImportsEntities + parameters: + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + - name: sort + in: query + description: Return GitLab Migrations sorted in created by `asc` or `desc` + order. + schema: + type: string + default: desc + enum: + - asc + - desc + - name: status + in: query + description: Return all GitLab Migrations' entities with specified status + schema: + type: string + enum: + - created + - started + - finished + - timeout + - failed + responses: + 200: + description: List all GitLab Migrations' entities + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_BulkImports' + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 503: + description: Service unavailable + content: {} + /api/v4/bulk_imports: + get: + tags: + - bulk_imports + summary: List all GitLab Migrations + description: This feature was introduced in GitLab 14.1. + operationId: getApiV4BulkImports + parameters: + - name: page + in: query + description: Current page number + schema: + type: integer + format: int32 + default: 1 + - name: per_page + in: query + description: Number of items per page + schema: + type: integer + format: int32 + default: 20 + - name: sort + in: query + description: Return GitLab Migrations sorted in created by `asc` or `desc` + order. + schema: + type: string + default: desc + enum: + - asc + - desc + - name: status + in: query + description: Return GitLab Migrations with specified status + schema: + type: string + enum: + - created + - started + - finished + - timeout + - failed + responses: + 200: + description: List all GitLab Migrations + content: + application/json: + schema: + type: array + items: + $ref: '#/components/schemas/API_Entities_BulkImport' + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 503: + description: Service unavailable + content: {} + post: + tags: + - bulk_imports + summary: Start a new GitLab Migration + description: This feature was introduced in GitLab 14.2. + operationId: postApiV4BulkImports + requestBody: + content: + application/x-www-form-urlencoded: + schema: + required: + - configuration[access_token] + - configuration[url] + - entities[destination_namespace] + - entities[source_full_path] + - entities[source_type] + properties: + configuration[url]: + type: string + description: Source GitLab instance URL + configuration[access_token]: + type: string + description: Access token to the source GitLab instance + entities[source_type]: + type: array + description: Source entity type + items: + type: string + enum: + - group_entity + - project_entity + entities[source_full_path]: + type: array + description: Relative path of the source entity to import + items: + type: string + entities[destination_namespace]: + type: array + description: Destination namespace for the entity + items: + type: string + entities[destination_slug]: + type: array + description: Destination slug for the entity + items: + type: string + entities[destination_name]: + type: array + description: 'Deprecated: Use :destination_slug instead. Destination + slug for the entity' + items: + type: string + entities[migrate_projects]: + type: array + description: Indicates group migration should include nested projects + items: + type: boolean + required: true + responses: + 200: + description: Start a new GitLab Migration + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_BulkImport' + 400: + description: Bad request + content: {} + 401: + description: Unauthorized + content: {} + 404: + description: Not found + content: {} + 422: + description: Unprocessable entity + content: {} + 503: + description: Service unavailable + content: {} + /api/v4/application/appearance: + get: + tags: + - application + description: Get the current appearance + operationId: getApiV4ApplicationAppearance + responses: + 200: + description: Get the current appearance + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Appearance' + put: + tags: + - application + description: Modify appearance + operationId: putApiV4ApplicationAppearance + requestBody: + content: + multipart/form-data: + schema: + properties: + title: + type: string + description: Instance title on the sign in / sign up page + description: + type: string + description: Markdown text shown on the sign in / sign up page + pwa_name: + type: string + description: Name of the Progressive Web App + pwa_short_name: + type: string + description: Optional, short name for Progressive Web App + pwa_description: + type: string + description: An explanation of what the Progressive Web App does + logo: + type: string + description: Instance image used on the sign in / sign up page + format: binary + pwa_icon: + type: string + description: Icon used for Progressive Web App + format: binary + header_logo: + type: string + description: Instance image used for the main navigation bar + format: binary + favicon: + type: string + description: Instance favicon in .ico/.png format + format: binary + new_project_guidelines: + type: string + description: Markdown text shown on the new project page + profile_image_guidelines: + type: string + description: Markdown text shown on the profile page below Public + Avatar + header_message: + type: string + description: Message within the system header bar + footer_message: + type: string + description: Message within the system footer bar + message_background_color: + type: string + description: Background color for the system header / footer bar + message_font_color: + type: string + description: Font color for the system header / footer bar + email_header_and_footer_enabled: + type: boolean + description: Add header and footer to all outgoing emails if enabled + responses: + 200: + description: Modify appearance + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Appearance' + /api/v4/application/plan_limits: + get: + tags: + - plan_limits + summary: Get current plan limits + description: List the current limits of a plan on the GitLab instance. + operationId: getApiV4ApplicationPlanLimits + parameters: + - name: plan_name + in: query + description: 'Name of the plan to get the limits from. Default: default.' + schema: + type: string + default: default + enum: + - default + - free + - bronze + - silver + - premium + - gold + - ultimate + - ultimate_trial + - premium_trial + - opensource + responses: + 200: + description: Get current plan limits + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_PlanLimit' + 401: + description: Unauthorized + content: {} + 403: + description: Forbidden + content: {} + put: + tags: + - plan_limits + summary: Change plan limits + description: Modify the limits of a plan on the GitLab instance. + operationId: putApiV4ApplicationPlanLimits + requestBody: + content: + application/json: + schema: + required: + - plan_name + properties: + plan_name: + type: string + description: Name of the plan to update + enum: + - default + - free + - bronze + - silver + - premium + - gold + - ultimate + - ultimate_trial + - premium_trial + - opensource + ci_pipeline_size: + type: integer + description: Maximum number of jobs in a single pipeline + format: int32 + ci_active_jobs: + type: integer + description: Total number of jobs in currently active pipelines + format: int32 + ci_project_subscriptions: + type: integer + description: Maximum number of pipeline subscriptions to and from + a project + format: int32 + ci_pipeline_schedules: + type: integer + description: Maximum number of pipeline schedules + format: int32 + ci_needs_size_limit: + type: integer + description: Maximum number of DAG dependencies that a job can have + format: int32 + ci_registered_group_runners: + type: integer + description: Maximum number of runners registered per group + format: int32 + ci_registered_project_runners: + type: integer + description: Maximum number of runners registered per project + format: int32 + conan_max_file_size: + type: integer + description: Maximum Conan package file size in bytes + format: int32 + enforcement_limit: + type: integer + description: Maximum storage size for the root namespace enforcement + in MiB + format: int32 + generic_packages_max_file_size: + type: integer + description: Maximum generic package file size in bytes + format: int32 + helm_max_file_size: + type: integer + description: Maximum Helm chart file size in bytes + format: int32 + maven_max_file_size: + type: integer + description: Maximum Maven package file size in bytes + format: int32 + notification_limit: + type: integer + description: Maximum storage size for the root namespace notifications + in MiB + format: int32 + npm_max_file_size: + type: integer + description: Maximum NPM package file size in bytes + format: int32 + nuget_max_file_size: + type: integer + description: Maximum NuGet package file size in bytes + format: int32 + pypi_max_file_size: + type: integer + description: Maximum PyPI package file size in bytes + format: int32 + terraform_module_max_file_size: + type: integer + description: Maximum Terraform Module package file size in bytes + format: int32 + storage_size_limit: + type: integer + description: Maximum storage size for the root namespace in MiB + format: int32 + pipeline_hierarchy_size: + type: integer + description: Maximum number of downstream pipelines in a pipeline's + hierarchy tree + format: int32 + required: true + responses: + 200: + description: Change plan limits + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_PlanLimit' + 400: + description: Bad request + content: {} + 401: + description: Unauthorized + content: {} + 403: + description: Forbidden + content: {} + /api/v4/metadata: + get: + tags: + - metadata + summary: Retrieve metadata information for this GitLab instance + description: This feature was introduced in GitLab 15.2. + operationId: getApiV4Metadata + responses: + 200: + description: Retrieve metadata information for this GitLab instance + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Metadata' + 401: + description: Unauthorized + content: {} + /api/v4/version: + get: + tags: + - metadata + summary: Retrieves version information for the GitLab instance + description: This feature was introduced in GitLab 8.13 and deprecated in 15.5. + We recommend you instead use the Metadata API. + operationId: getApiV4Version + responses: + 200: + description: Retrieves version information for the GitLab instance + content: + application/json: + schema: + $ref: '#/components/schemas/API_Entities_Metadata' + 401: + description: Unauthorized + content: {} +components: + schemas: + API_Entities_Badge: + type: object + properties: + name: + type: string + link_url: + type: string + image_url: + type: string + rendered_link_url: + type: string + rendered_image_url: + type: string + id: + type: string + kind: + type: string + description: API_Entities_Badge model + API_Entities_BasicBadgeDetails: + type: object + properties: + name: + type: string + link_url: + type: string + image_url: + type: string + rendered_link_url: + type: string + rendered_image_url: + type: string + description: API_Entities_BasicBadgeDetails model + API_Entities_AccessRequester: + type: object + properties: + id: + type: integer + format: int32 + example: 1 + username: + type: string + example: admin + name: + type: string + example: Administrator + state: + type: string + example: active + avatar_url: + type: string + example: https://gravatar.com/avatar/1 + avatar_path: + type: string + example: /user/avatar/28/The-Big-Lebowski-400-400.png + custom_attributes: + type: array + items: + $ref: '#/components/schemas/API_Entities_CustomAttribute' + web_url: + type: string + example: https://gitlab.example.com/root + email: + type: string + requested_at: + type: string + description: API_Entities_AccessRequester model + API_Entities_CustomAttribute: + type: object + properties: + key: + type: string + example: foo + value: + type: string + example: bar + API_Entities_Branch: + type: object + properties: + name: + type: string + example: master + commit: + $ref: '#/components/schemas/API_Entities_Commit' + merged: + type: boolean + example: true + protected: + type: boolean + example: true + developers_can_push: + type: boolean + example: true + developers_can_merge: + type: boolean + example: true + can_push: + type: boolean + example: true + default: + type: boolean + example: true + web_url: + type: string + example: https://gitlab.example.com/Commit921/the-dude/-/tree/master + description: API_Entities_Branch model + API_Entities_Commit: + type: object + properties: + id: + type: string + example: 2695effb5807a22ff3d138d593fd856244e155e7 + short_id: + type: string + example: 2695effb + created_at: + type: string + format: date-time + example: 2017-07-26T11:08:53+02:00 + parent_ids: + type: array + items: + type: string + example: 2a4b78934375d7f53875269ffd4f45fd83a84ebe + title: + type: string + example: Initial commit + message: + type: string + example: Initial commit + author_name: + type: string + example: John Smith + author_email: + type: string + example: john@example.com + authored_date: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + committer_name: + type: string + example: Jack Smith + committer_email: + type: string + example: jack@example.com + committed_date: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + trailers: + type: object + properties: {} + example: '{ "Merged-By": "Jane Doe janedoe@gitlab.com" }' + web_url: + type: string + example: https://gitlab.example.com/janedoe/gitlab-foss/-/commit/ed899a2f4b50b4370feeea94676502b42383c746 + API_Entities_MetricImage: + type: object + properties: + id: + type: integer + format: int32 + example: 23 + created_at: + type: string + format: date-time + example: 2020-11-13T00:06:18.084Z + filename: + type: string + example: file.png + file_path: + type: string + example: /uploads/-/system/alert_metric_image/file/23/file.png + url: + type: string + example: https://example.com/metric + url_text: + type: string + example: An example metric + description: API_Entities_MetricImage model + API_Entities_BatchedBackgroundMigration: + type: object + properties: + id: + type: string + example: "1234" + job_class_name: + type: string + example: CopyColumnUsingBackgroundMigrationJob + table_name: + type: string + example: events + status: + type: string + example: active + progress: + type: number + format: float + example: 50.0 + created_at: + type: string + format: date-time + example: 2022-11-28T16:26:39+02:00 + description: API_Entities_BatchedBackgroundMigration model + API_Entities_Ci_Variable: + type: object + properties: + variable_type: + type: string + example: env_var + key: + type: string + example: TEST_VARIABLE_1 + value: + type: string + example: TEST_1 + protected: + type: boolean + masked: + type: boolean + raw: + type: boolean + environment_scope: + type: string + example: '*' + description: API_Entities_Ci_Variable model + API_Entities_Dictionary_Table: + type: object + properties: + table_name: + type: string + example: users + feature_categories: + type: array + items: + type: string + example: database + description: API_Entities_Dictionary_Table model + API_Entities_Cluster: + type: object + properties: + id: + type: string + name: + type: string + created_at: + type: string + domain: + type: string + enabled: + type: string + managed: + type: string + provider_type: + type: string + platform_type: + type: string + environment_scope: + type: string + cluster_type: + type: string + namespace_per_environment: + type: string + user: + $ref: '#/components/schemas/API_Entities_UserBasic' + platform_kubernetes: + $ref: '#/components/schemas/API_Entities_Platform_Kubernetes' + provider_gcp: + $ref: '#/components/schemas/API_Entities_Provider_Gcp' + management_project: + $ref: '#/components/schemas/API_Entities_ProjectIdentity' + description: API_Entities_Cluster model + API_Entities_UserBasic: + type: object + properties: + id: + type: integer + format: int32 + example: 1 + username: + type: string + example: admin + name: + type: string + example: Administrator + state: + type: string + example: active + avatar_url: + type: string + example: https://gravatar.com/avatar/1 + avatar_path: + type: string + example: /user/avatar/28/The-Big-Lebowski-400-400.png + custom_attributes: + type: array + items: + $ref: '#/components/schemas/API_Entities_CustomAttribute' + web_url: + type: string + example: https://gitlab.example.com/root + email: + type: string + API_Entities_Platform_Kubernetes: + type: object + properties: + api_url: + type: string + namespace: + type: string + authorization_type: + type: string + ca_cert: + type: string + API_Entities_Provider_Gcp: + type: object + properties: + cluster_id: + type: string + status_name: + type: string + gcp_project_id: + type: string + zone: + type: string + machine_type: + type: string + num_nodes: + type: string + endpoint: + type: string + API_Entities_ProjectIdentity: + type: object + properties: + id: + type: integer + format: int32 + example: 1 + description: + type: string + example: desc + name: + type: string + example: project1 + name_with_namespace: + type: string + example: John Doe / project1 + path: + type: string + example: project1 + path_with_namespace: + type: string + example: namespace1/project1 + created_at: + type: string + format: date-time + example: 2020-05-07T04:27:17.016Z + API_Entities_Application: + type: object + properties: + id: + type: string + application_id: + type: string + example: 5832fc6e14300a0d962240a8144466eef4ee93ef0d218477e55f11cf12fc3737 + application_name: + type: string + example: MyApplication + callback_url: + type: string + example: https://redirect.uri + confidential: + type: boolean + example: true + description: API_Entities_Application model + API_Entities_ApplicationWithSecret: + type: object + properties: + id: + type: string + application_id: + type: string + example: 5832fc6e14300a0d962240a8144466eef4ee93ef0d218477e55f11cf12fc3737 + application_name: + type: string + example: MyApplication + callback_url: + type: string + example: https://redirect.uri + confidential: + type: boolean + example: true + secret: + type: string + example: ee1dd64b6adc89cf7e2c23099301ccc2c61b441064e9324d963c46902a85ec34 + description: API_Entities_ApplicationWithSecret model + API_Entities_Avatar: + type: object + properties: + avatar_url: + type: string + description: API_Entities_Avatar model + API_Entities_BroadcastMessage: + type: object + properties: + id: + type: string + message: + type: string + starts_at: + type: string + ends_at: + type: string + color: + type: string + font: + type: string + target_access_levels: + type: string + target_path: + type: string + broadcast_type: + type: string + dismissable: + type: string + active: + type: string + description: API_Entities_BroadcastMessage model + API_Entities_BulkImports: + type: object + properties: + id: + type: integer + format: int32 + example: 1 + bulk_import_id: + type: integer + format: int32 + example: 1 + status: + type: string + example: created + enum: + - created + - started + - finished + - timeout + - failed + entity_type: + type: string + enum: + - group + - project + source_full_path: + type: string + example: source_group + destination_full_path: + type: string + example: some_group/source_project + destination_name: + type: string + example: destination_slug + destination_slug: + type: string + example: destination_slug + destination_namespace: + type: string + example: destination_path + parent_id: + type: integer + format: int32 + example: 1 + namespace_id: + type: integer + format: int32 + example: 1 + project_id: + type: integer + format: int32 + example: 1 + created_at: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + updated_at: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + failures: + type: array + items: + $ref: '#/components/schemas/API_Entities_BulkImports_EntityFailure' + migrate_projects: + type: boolean + example: true + description: API_Entities_BulkImports model + API_Entities_BulkImports_EntityFailure: + type: object + properties: + relation: + type: string + example: group + step: + type: string + example: extractor + exception_message: + type: string + example: error message + exception_class: + type: string + example: Exception + correlation_id_value: + type: string + example: dfcf583058ed4508e4c7c617bd7f0edd + created_at: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + pipeline_class: + type: string + example: BulkImports::Groups::Pipelines::GroupPipeline + pipeline_step: + type: string + example: extractor + API_Entities_BulkImport: + type: object + properties: + id: + type: integer + format: int32 + example: 1 + status: + type: string + example: finished + enum: + - created + - started + - finished + - timeout + - failed + source_type: + type: string + example: gitlab + created_at: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + updated_at: + type: string + format: date-time + example: 2012-05-28T04:42:42-07:00 + description: API_Entities_BulkImport model + API_Entities_Appearance: + type: object + properties: + title: + type: string + description: + type: string + pwa_name: + type: string + pwa_short_name: + type: string + pwa_description: + type: string + logo: + type: string + pwa_icon: + type: string + header_logo: + type: string + favicon: + type: string + new_project_guidelines: + type: string + profile_image_guidelines: + type: string + header_message: + type: string + footer_message: + type: string + message_background_color: + type: string + message_font_color: + type: string + email_header_and_footer_enabled: + type: string + description: API_Entities_Appearance model + API_Entities_PlanLimit: + type: object + properties: + ci_pipeline_size: + type: integer + format: int32 + example: 0 + ci_active_jobs: + type: integer + format: int32 + example: 0 + ci_project_subscriptions: + type: integer + format: int32 + example: 2 + ci_pipeline_schedules: + type: integer + format: int32 + example: 10 + ci_needs_size_limit: + type: integer + format: int32 + example: 50 + ci_registered_group_runners: + type: integer + format: int32 + example: 1000 + ci_registered_project_runners: + type: integer + format: int32 + example: 1000 + conan_max_file_size: + type: integer + format: int32 + example: 3221225472 + enforcement_limit: + type: integer + format: int32 + example: 15000 + generic_packages_max_file_size: + type: integer + format: int32 + example: 5368709120 + helm_max_file_size: + type: integer + format: int32 + example: 5242880 + limits_history: + type: object + properties: {} + example: |- + {"enforcement_limit"=>[{"timestamp"=>1686909124, "user_id"=>1, "username"=>"x", "value"=>5}], + "notification_limit"=>[{"timestamp"=>1686909124, "user_id"=>2, "username"=>"y", "value"=>7}]} + maven_max_file_size: + type: integer + format: int32 + example: 3221225472 + notification_limit: + type: integer + format: int32 + example: 15000 + npm_max_file_size: + type: integer + format: int32 + example: 524288000 + nuget_max_file_size: + type: integer + format: int32 + example: 524288000 + pipeline_hierarchy_size: + type: integer + format: int32 + example: 1000 + pypi_max_file_size: + type: integer + format: int32 + example: 3221225472 + terraform_module_max_file_size: + type: integer + format: int32 + example: 1073741824 + storage_size_limit: + type: integer + format: int32 + example: 15000 + description: API_Entities_PlanLimit model + API_Entities_Metadata: + type: object + properties: + version: + type: string + example: 15.2-pre + revision: + type: string + example: c401a659d0c + kas: + type: object + properties: + enabled: + type: boolean + externalUrl: + type: string + example: grpc://gitlab.example.com:8150 + version: + type: string + example: 15.0.0 + enterprise: + type: boolean + description: API_Entities_Metadata model + securitySchemes: + ApiKeyAuth: + type: apiKey + in: header + name: Private-Token diff --git a/doc/api/packages/npm.md b/doc/api/packages/npm.md index 6f4d8446dbf..3aa23ad1044 100644 --- a/doc/api/packages/npm.md +++ b/doc/api/packages/npm.md @@ -124,7 +124,7 @@ different scopes: - Use the instance-level prefix to make requests in the scope of the entire instance. - Use the project-level prefix to make requests in a single project's scope. -- Use the group-level prefix to make requests in a group’s scope. +- Use the group-level prefix to make requests in a group's scope. The examples in this document all use the project-level prefix. diff --git a/doc/api/packages/pypi.md b/doc/api/packages/pypi.md index 4e7c59adf3a..1b155feb8b5 100644 --- a/doc/api/packages/pypi.md +++ b/doc/api/packages/pypi.md @@ -28,8 +28,8 @@ is recommended when [FIPS mode](../../development/fips_compliance.md) is enabled > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225545) in GitLab 13.12. -Download a PyPI package file. The [simple API](#group-level-simple-api-entry-point) -normally supplies this URL. +Download a PyPI package file. The [simple API](#group-level-simple-api-entry-point) +usually supplies this URL. ```plaintext GET groups/:id/-/packages/pypi/files/:sha256/:file_identifier @@ -142,7 +142,7 @@ This writes the downloaded file to `simple.html` in the current directory. > Introduced in GitLab 12.10. Download a PyPI package file. The [simple API](#project-level-simple-api-entry-point) -normally supplies this URL. +usually supplies this URL. ```plaintext GET projects/:id/packages/pypi/files/:sha256/:file_identifier diff --git a/doc/api/pipeline_schedules.md b/doc/api/pipeline_schedules.md index 5dc0dead683..115e5b279b8 100644 --- a/doc/api/pipeline_schedules.md +++ b/doc/api/pipeline_schedules.md @@ -16,10 +16,10 @@ Get a list of the pipeline schedules of a project. GET /projects/:id/pipeline_schedules ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `scope` | string | no | The scope of pipeline schedules, one of: `active`, `inactive` | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `scope` | string | No | The scope of pipeline schedules, must be one of: `active`, `inactive` | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules" @@ -57,10 +57,10 @@ Get the pipeline schedule of a project. GET /projects/:id/pipeline_schedules/:pipeline_schedule_id ``` -| Attribute | Type | required | Description | -|--------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13" @@ -115,8 +115,8 @@ Supported attributes: | Attribute | Type | Required | Description | |------------------------|----------------|----------|-------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID. | +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID. | Example request: @@ -163,14 +163,14 @@ Create a new pipeline schedule of a project. POST /projects/:id/pipeline_schedules ``` -| Attribute | Type | required | Description | -|-----------------|----------------|----------|-------------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `description` | string | yes | The description of the pipeline schedule. | -| `ref` | string | yes | The branch or tag name that is triggered. | -| `cron` | string | yes | The [cron](https://en.wikipedia.org/wiki/Cron) schedule, for example: `0 1 * * *`. | -| `cron_timezone` | string | no | The time zone supported by `ActiveSupport::TimeZone`, for example: `Pacific Time (US & Canada)` (default: `UTC`). | -| `active` | boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule is initially deactivated (default: `true`). | +| Attribute | Type | Required | Description | +|-----------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `description` | string | Yes | The description of the pipeline schedule. | +| `ref` | string | Yes | The branch or tag name that is triggered. | +| `cron` | string | Yes | The [cron](https://en.wikipedia.org/wiki/Cron) schedule, for example: `0 1 * * *`. | +| `cron_timezone` | string | No | The time zone supported by `ActiveSupport::TimeZone`, for example: `Pacific Time (US & Canada)` (default: `UTC`). | +| `active` | boolean | No | The activation of pipeline schedule. If false is set, the pipeline schedule is initially deactivated (default: `true`). | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -203,21 +203,21 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ ## Edit a pipeline schedule -Updates the pipeline schedule of a project. Once the update is done, it is rescheduled automatically. +Updates the pipeline schedule of a project. After the update is done, it is rescheduled automatically. ```plaintext PUT /projects/:id/pipeline_schedules/:pipeline_schedule_id ``` -| Attribute | Type | required | Description | -|------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID. | -| `description` | string | no | The description of the pipeline schedule. | -| `ref` | string | no | The branch or tag name that is triggered. | -| `cron` | string | no | The [cron](https://en.wikipedia.org/wiki/Cron) schedule, for example: `0 1 * * *`. | -| `cron_timezone` | string | no | The time zone supported by `ActiveSupport::TimeZone` (for example `Pacific Time (US & Canada)`), or `TZInfo::Timezone` (for example `America/Los_Angeles`). | -| `active` | boolean | no | The activation of pipeline schedule. If false is set, the pipeline schedule is initially deactivated. | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID. | +| `description` | string | No | The description of the pipeline schedule. | +| `ref` | string | No | The branch or tag name that is triggered. | +| `cron` | string | No | The [cron](https://en.wikipedia.org/wiki/Cron) schedule, for example: `0 1 * * *`. | +| `cron_timezone` | string | No | The time zone supported by `ActiveSupport::TimeZone` (for example `Pacific Time (US & Canada)`), or `TZInfo::Timezone` (for example `America/Los_Angeles`). | +| `active` | boolean | No | The activation of pipeline schedule. If false is set, the pipeline schedule is initially deactivated. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -260,10 +260,10 @@ Update the owner of the pipeline schedule of a project. POST /projects/:id/pipeline_schedules/:pipeline_schedule_id/take_ownership ``` -| Attribute | Type | required | Description | -|---------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/take_ownership" @@ -305,10 +305,10 @@ Delete the pipeline schedule of a project. DELETE /projects/:id/pipeline_schedules/:pipeline_schedule_id ``` -| Attribute | Type | required | Description | -|----------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13" @@ -344,8 +344,6 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git ## Run a scheduled pipeline immediately -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201786) in GitLab 12.8. - Trigger a new scheduled pipeline, which runs immediately. The next scheduled run of this pipeline is not affected. @@ -353,10 +351,10 @@ of this pipeline is not affected. POST /projects/:id/pipeline_schedules/:pipeline_schedule_id/play ``` -| Attribute | Type | required | Description | -| ---------------- | --------- | ---------- | -------------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | Example request: @@ -382,13 +380,13 @@ Create a new variable of a pipeline schedule. POST /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables ``` -| Attribute | Type | required | Description | -|------------------------|----------------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | -| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | +| `key` | string | Yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --form "key=NEW_VARIABLE" \ @@ -411,13 +409,13 @@ Updates the variable of a pipeline schedule. PUT /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables/:key ``` -| Attribute | Type | required | Description | -|------------------------|----------------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | -| `key` | string | yes | The `key` of a variable | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | +| `key` | string | Yes | The `key` of a variable | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -441,11 +439,11 @@ Delete the variable of a pipeline schedule. DELETE /projects/:id/pipeline_schedules/:pipeline_schedule_id/variables/:key ``` -| Attribute | Type | required | Description | -|------------------------|----------------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_schedule_id` | integer | yes | The pipeline schedule ID | -| `key` | string | yes | The `key` of a variable | +| Attribute | Type | Required | Description | +|------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_schedule_id` | integer | Yes | The pipeline schedule ID | +| `key` | string | Yes | The `key` of a variable | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/29/pipeline_schedules/13/variables/NEW_VARIABLE" diff --git a/doc/api/pipeline_triggers.md b/doc/api/pipeline_triggers.md index 50acac6bc2a..c62e622e31e 100644 --- a/doc/api/pipeline_triggers.md +++ b/doc/api/pipeline_triggers.md @@ -16,9 +16,9 @@ Get a list of a project's pipeline trigger tokens. GET /projects/:id/triggers ``` -| Attribute | Type | required | Description | -|-----------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/triggers" @@ -49,10 +49,10 @@ Get details of a project's pipeline trigger. GET /projects/:id/triggers/:trigger_id ``` -| Attribute | Type | required | Description | -|--------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `trigger_id` | integer | yes | The trigger ID | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `trigger_id` | integer | Yes | The trigger ID | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/triggers/5" @@ -78,10 +78,10 @@ Create a pipeline trigger for a project. POST /projects/:id/triggers ``` -| Attribute | Type | required | Description | -|---------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `description` | string | yes | The trigger name | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `description` | string | Yes | The trigger name | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -108,11 +108,11 @@ Update a pipeline trigger token for a project. PUT /projects/:id/triggers/:trigger_id ``` -| Attribute | Type | required | Description | -|---------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `trigger_id` | integer | yes | The trigger ID | -| `description` | string | no | The trigger name | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `trigger_id` | integer | Yes | The trigger ID | +| `description` | string | No | The trigger name | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -139,10 +139,10 @@ Remove a project's pipeline trigger token. DELETE /projects/:id/triggers/:trigger_id ``` -| Attribute | Type | required | Description | -|----------------|---------|----------|--------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `trigger_id` | integer | yes | The trigger ID | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `trigger_id` | integer | Yes | The trigger ID | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/triggers/5" @@ -165,12 +165,12 @@ POST /projects/:id/trigger/pipeline Supported attributes: -| Attribute | Type | Required | Description | -|:------------|:---------------|:-----------------------|:---------------------| -| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `ref` | string | **{check-circle}** Yes | The branch or tag to run the pipeline on. | -| `token` | string | **{check-circle}** Yes | The trigger token or CI/CD job token. | -| `variables` | hash | **{dotted-circle}** No | A map of key-valued strings containing the pipeline variables. For example: `{ VAR1: "value1", VAR2: "value2" }`. | +| Attribute | Type | Required | Description | +|-------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `ref` | string | Yes | The branch or tag to run the pipeline on. | +| `token` | string | Yes | The trigger token or CI/CD job token. | +| `variables` | hash | No | A map of key-valued strings containing the pipeline variables. For example: `{ VAR1: "value1", VAR2: "value2" }`. | Example request: diff --git a/doc/api/pipelines.md b/doc/api/pipelines.md index fed0e553a9e..4fca878fcec 100644 --- a/doc/api/pipelines.md +++ b/doc/api/pipelines.md @@ -20,7 +20,7 @@ Read more on [pagination](rest/index.md#pagination). FLAG: On self-managed GitLab, by default the `name` field is not available. -To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `pipeline_name_in_api`. This feature is not ready for production use. On GitLab.com, this feature is not available. @@ -31,21 +31,21 @@ but you can [get child pipeline](pipelines.md#get-a-single-pipeline) individuall GET /projects/:id/pipelines ``` -| Attribute | Type | Required | Description | -|-----------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `scope` | string | no | The scope of pipelines, one of: `running`, `pending`, `finished`, `branches`, `tags` | -| `status` | string | no | The status of pipelines, one of: `created`, `waiting_for_resource`, `preparing`, `pending`, `running`, `success`, `failed`, `canceled`, `skipped`, `manual`, `scheduled` | -| `source` | string | no | In [GitLab 14.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/325439), how the pipeline was triggered, one of: `push`, `web`, `trigger`, `schedule`, `api`, `external`, `pipeline`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, `ondemand_dast_scan`, or `ondemand_dast_validation`. | -| `ref` | string | no | The ref of pipelines | -| `sha` | string | no | The SHA of pipelines | -| `yaml_errors`| boolean | no | Returns pipelines with invalid configurations | -| `username`| string | no | The username of the user who triggered pipelines | -| `updated_after` | datetime | no | Return pipelines updated after the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `updated_before` | datetime | no | Return pipelines updated before the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | -| `name` | string | no | Return pipelines with the specified name. Introduced in GitLab 15.11, not available by default. | -| `order_by`| string | no | Order pipelines by `id`, `status`, `ref`, `updated_at` or `user_id` (default: `id`) | -| `sort` | string | no | Sort pipelines in `asc` or `desc` order (default: `desc`) | +| Attribute | Type | Required | Description | +|------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `scope` | string | No | The scope of pipelines, one of: `running`, `pending`, `finished`, `branches`, `tags` | +| `status` | string | No | The status of pipelines, one of: `created`, `waiting_for_resource`, `preparing`, `pending`, `running`, `success`, `failed`, `canceled`, `skipped`, `manual`, `scheduled` | +| `source` | string | No | In [GitLab 14.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/325439), how the pipeline was triggered, one of: `push`, `web`, `trigger`, `schedule`, `api`, `external`, `pipeline`, `chat`, `webide`, `merge_request_event`, `external_pull_request_event`, `parent_pipeline`, `ondemand_dast_scan`, or `ondemand_dast_validation`. | +| `ref` | string | No | The ref of pipelines | +| `sha` | string | No | The SHA of pipelines | +| `yaml_errors` | boolean | No | Returns pipelines with invalid configurations | +| `username` | string | No | The username of the user who triggered pipelines | +| `updated_after` | datetime | No | Return pipelines updated after the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `updated_before` | datetime | No | Return pipelines updated before the specified date. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`). | +| `name` | string | No | Return pipelines with the specified name. Introduced in GitLab 15.11, not available by default. | +| `order_by` | string | No | Order pipelines by `id`, `status`, `ref`, `updated_at` or `user_id` (default: `id`) | +| `sort` | string | No | Sort pipelines in `asc` or `desc` order (default: `desc`) | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines" @@ -91,23 +91,22 @@ Example of response FLAG: On self-managed GitLab, by default the `name` field is not available. -To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `pipeline_name_in_api`. This feature is not ready for production use. On GitLab.com, this feature is not available. Get one pipeline from a project. You can also get a single [child pipeline](../ci/pipelines/downstream_pipelines.md#parent-child-pipelines). -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36494) in GitLab 13.3. ```plaintext GET /projects/:id/pipelines/:pipeline_id ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/46" @@ -153,10 +152,10 @@ Example of response GET /projects/:id/pipelines/:pipeline_id/variables ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/46/variables" @@ -180,8 +179,6 @@ Example of response ### Get a pipeline's test report -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202525) in GitLab 13.0. - NOTE: This API route is part of the [Unit test report](../ci/testing/unit_test_reports.md) feature. @@ -189,10 +186,10 @@ This API route is part of the [Unit test report](../ci/testing/unit_test_reports GET /projects/:id/pipelines/:pipeline_id/test_report ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | Sample request: @@ -245,10 +242,10 @@ This API route is part of the [Unit test report](../ci/testing/unit_test_reports GET /projects/:id/pipelines/:pipeline_id/test_report_summary ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | Sample request: @@ -293,7 +290,7 @@ Sample response: FLAG: On self-managed GitLab, by default the `name` field is not available. -To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) +To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `pipeline_name_in_api`. This feature is not ready for production use. On GitLab.com, this feature is not available. @@ -303,12 +300,12 @@ Get the latest pipeline for a specific ref in a project. GET /projects/:id/pipelines/latest ``` -| Attribute | Type | Required | Description | -|-------------|---------|----------|---------------------| -| `ref` | string | no | The branch or tag to check for the latest pipeline. Defaults to the default branch when not specified. | +| Attribute | Type | Required | Description | +|-----------|--------|----------|-------------| +| `ref` | string | No | The branch or tag to check for the latest pipeline. Defaults to the default branch when not specified. | ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/latest" +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/latest" ``` Example of response @@ -365,11 +362,11 @@ Example of response POST /projects/:id/pipeline ``` -| Attribute | Type | Required | Description | -|-------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `ref` | string | yes | The branch or tag to run the pipeline on. | -| `variables` | array | no | An [array of hashes](rest/index.md#array-of-hashes) containing the variables available in the pipeline, matching the structure `[{ 'key': 'UPLOAD_TO_S3', 'variable_type': 'file', 'value': 'true' }, {'key': 'TEST', 'value': 'test variable'}]`. If `variable_type` is excluded, it defaults to `env_var`. | +| Attribute | Type | Required | Description | +|-------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `ref` | string | Yes | The branch or tag to run the pipeline on. | +| `variables` | array | No | An [array of hashes](rest/index.md#array-of-hashes) containing the variables available in the pipeline, matching the structure `[{ 'key': 'UPLOAD_TO_S3', 'variable_type': 'file', 'value': 'true' }, {'key': 'TEST', 'value': 'test variable'}]`. If `variable_type` is excluded, it defaults to `env_var`. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipeline?ref=main" @@ -416,10 +413,10 @@ Example of response POST /projects/:id/pipelines/:pipeline_id/retry ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/46/retry" @@ -464,10 +461,10 @@ Response: POST /projects/:id/pipelines/:pipeline_id/cancel ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/pipelines/46/cancel" @@ -508,8 +505,6 @@ Response: ## Delete a pipeline -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22988) in GitLab 11.6. - Deleting a pipeline expires all pipeline caches, and deletes all immediately related objects, such as builds, logs, artifacts, and triggers. **This action cannot be undone.** @@ -523,10 +518,10 @@ for details. DELETE /projects/:id/pipelines/:pipeline_id ``` -| Attribute | Type | Required | Description | -|------------|---------|----------|---------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `pipeline_id` | integer | yes | The ID of a pipeline | +| Attribute | Type | Required | Description | +|---------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `pipeline_id` | integer | Yes | The ID of a pipeline | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" --request "DELETE" "https://gitlab.example.com/api/v4/projects/1/pipelines/46" diff --git a/doc/api/product_analytics.md b/doc/api/product_analytics.md index c37fe223778..70f0d40f7b5 100644 --- a/doc/api/product_analytics.md +++ b/doc/api/product_analytics.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `cube_api_proxy`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../administration/feature_flags.md) named `cube_api_proxy`. On GitLab.com, this feature is not available. This feature is not ready for production use. diff --git a/doc/api/project_access_tokens.md b/doc/api/project_access_tokens.md index 437bdaa70f4..36129bf6576 100644 --- a/doc/api/project_access_tokens.md +++ b/doc/api/project_access_tokens.md @@ -84,6 +84,7 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55408) in GitLab 13.10. > - The `token` attribute was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55408) in GitLab 13.10. +> - The `expires_at` attribute default was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120213) in GitLab 16.0. Create a [project access token](../user/project/settings/project_access_tokens.md). @@ -105,7 +106,7 @@ POST projects/:id/access_tokens | `name` | String | yes | Name of the project access token | | `scopes` | `Array[String]` | yes | [List of scopes](../user/project/settings/project_access_tokens.md#scopes-for-a-project-access-token) | | `access_level` | Integer | no | Access level. Valid values are `10` (Guest), `20` (Reporter), `30` (Developer), `40` (Maintainer), and `50` (Owner). Defaults to `40`. | -| `expires_at` | Date | no | Token expires at midnight UTC on that date | +| `expires_at` | Date | yes | Expiration date of the access token in ISO format (`YYYY-MM-DD`). The date cannot be set later than the [maximum allowable lifetime of an access token](../user/profile/personal_access_tokens.md#when-personal-access-tokens-expire). | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index fbbefe95cd8..bae300efaf4 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -32,7 +32,7 @@ project to a web server or to any S3-compatible platform. For exports, GitLab: time and is available throughout the export process. - Administrators can modify the maximum export file size. By default, the maximum is unlimited (`0`). To change this, edit `max_export_size` using either: - - [GitLab UI](../user/admin_area/settings/account_and_limit_settings.md). + - [GitLab UI](../administration/settings/account_and_limit_settings.md). - [Application settings API](settings.md#change-application-settings) - Has a fixed limit for the maximum import file size on GitLab.com. For more information, see [Account and limit settings](../user/gitlab_com/index.md#account-and-limit-settings). @@ -203,14 +203,14 @@ requests.post(url, headers=headers, data=data, files=files) NOTE: The maximum import file size can be set by the Administrator. It defaults to `0` (unlimited). -As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](settings.md#change-application-settings) or the [Admin Area](../user/admin_area/settings/account_and_limit_settings.md). Default [modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to 0 in GitLab 13.8. +As an administrator, you can modify the maximum import file size. To do so, use the `max_import_size` option in the [Application settings API](settings.md#change-application-settings) or the [Admin Area](../administration/settings/account_and_limit_settings.md). Default [modified](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to 0 in GitLab 13.8. ## Import a file from a remote object storage (Beta) > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282503) in GitLab 13.12 in [Beta](../policy/experiment-beta-support.md#beta) [with a flag](../administration/feature_flags.md) named `import_project_from_remote_file`. Enabled by default. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `import_project_from_remote_file`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../administration/feature_flags.md) named `import_project_from_remote_file`. On GitLab.com, this feature is available. ```plaintext diff --git a/doc/api/project_job_token_scopes.md b/doc/api/project_job_token_scopes.md index c73e6ea203e..4d1e013ee14 100644 --- a/doc/api/project_job_token_scopes.md +++ b/doc/api/project_job_token_scopes.md @@ -22,16 +22,16 @@ GET /projects/:id/job_token_scope Supported attributes: -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | If successful, returns [`200`](rest/index.md#status-codes) and the following response attributes: -| Attribute | Type | Description | -|:-------------------|:--------|:----------------------| +| Attribute | Type | Description | +|--------------------|---------|-------------| | `inbound_enabled` | boolean | Indicates if the CI/CD job token generated in other projects has access to this project. | -| `outbound_enabled` | boolean | Indicates if the CI/CD job token generated in this project has access to other projects. [Deprecated and planned for removal in GitLab 17.0 .](../update/removals.md#limit-ci_job_token-scope-is-disabled) | +| `outbound_enabled` | boolean | Indicates if the CI/CD job token generated in this project has access to other projects. [Deprecated and planned for removal in GitLab 17.0](../update/deprecations.md#default-cicd-job-token-ci_job_token-scope-changed). | Example request: @@ -58,10 +58,10 @@ PATCH /projects/:id/job_token_scope Supported attributes: -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `enabled` | boolean | **{check-circle}** Yes | Indicates CI/CD job tokens generated in other projects have restricted access to this project. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `enabled` | boolean | Yes | Indicates CI/CD job tokens generated in other projects have restricted access to this project. | If successful, returns [`204`](rest/index.md#status-codes) and no response body. @@ -85,9 +85,9 @@ GET /projects/:id/job_token_scope/allowlist Supported attributes: -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | This endpoint supports [offset-based pagination](rest/index.md#offset-based-pagination). @@ -152,10 +152,10 @@ POST /projects/:id/job_token_scope/allowlist Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `target_project_id` | integer | **{check-circle}** Yes | The ID of the project added to the CI/CD job token inbound allowlist. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `target_project_id` | integer | Yes | The ID of the project added to the CI/CD job token inbound allowlist. | If successful, returns [`201`](rest/index.md#status-codes) and the following response attributes: @@ -193,10 +193,10 @@ DELETE /projects/:id/job_token_scope/allowlist/:target_project_id Supported attributes: -| Attribute | Type | Required | Description | -|---------------------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `target_project_id` | integer | **{check-circle}** Yes | The ID of the project that is removed from the CI/CD job token inbound allowlist. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `target_project_id` | integer | Yes | The ID of the project that is removed from the CI/CD job token inbound allowlist. | If successful, returns [`204`](rest/index.md#status-codes) and no response body. diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md index fa699c34a8a..46e491453f9 100644 --- a/doc/api/project_level_variables.md +++ b/doc/api/project_level_variables.md @@ -15,9 +15,9 @@ Get list of a project's variables. GET /projects/:id/variables ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID of a project or [URL-encoded NAMESPACE/PROJECT_NAME of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables" @@ -32,7 +32,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": true, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null }, { "variable_type": "env_var", @@ -41,7 +42,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ] ``` @@ -55,11 +57,11 @@ use `filter` to select the correct `environment_scope`. GET /projects/:id/variables/:key ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID of a project or [URL-encoded NAMESPACE/PROJECT_NAME of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | -| `filter` | hash | no | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | +| `filter` | hash | No | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/TEST_VARIABLE_1" @@ -73,7 +75,8 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a "protected": false, "masked": true, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ``` @@ -87,16 +90,17 @@ must have a different `environment_scope`. Otherwise, GitLab returns a message s POST /projects/:id/variables ``` -| Attribute | Type | Required | Description | -| ------------------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID of a project or [URL-encoded NAMESPACE/PROJECT_NAME of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected. Default: `false` | -| `masked` | boolean | no | Whether the variable is masked. Default: `false` | -| `raw` | boolean | no | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | -| `environment_scope` | string | no | The `environment_scope` of the variable. Default: `*` | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | No | Whether the variable is protected. Default: `false` | +| `masked` | boolean | No | Whether the variable is masked. Default: `false` | +| `raw` | boolean | No | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | +| `environment_scope` | string | No | The `environment_scope` of the variable. Default: `*` | +| `description` | string | No | The description of the variable. Default: `null`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409641) in GitLab 16.2. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -111,7 +115,8 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "protected": false, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": null } ``` @@ -124,17 +129,18 @@ use `filter` to select the correct `environment_scope`. PUT /projects/:id/variables/:key ``` -| Attribute | Type | Required | Description | -| ------------------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID of a project or [URL-encoded NAMESPACE/PROJECT_NAME of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected | -| `masked` | boolean | no | Whether the variable is masked | -| `raw` | boolean | no | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | -| `environment_scope` | string | no | The `environment_scope` of the variable | -| `filter` | hash | no | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | +| `value` | string | Yes | The `value` of a variable | +| `variable_type` | string | No | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | No | Whether the variable is protected | +| `masked` | boolean | No | Whether the variable is masked | +| `raw` | boolean | No | Whether the variable is treated as a raw string. Default: `false`. When `true`, variables in the value are not [expanded](../ci/variables/index.md#prevent-cicd-variable-expansion). | +| `environment_scope` | string | No | The `environment_scope` of the variable | +| `filter` | hash | No | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | +| `description` | string | No | The description of the variable. Default: `null`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/409641) in GitLab 16.2. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -149,7 +155,8 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ "protected": true, "masked": false, "raw": false, - "environment_scope": "*" + "environment_scope": "*", + "description": "null" } ``` @@ -162,11 +169,11 @@ use `filter` to select the correct `environment_scope`. DELETE /projects/:id/variables/:key ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| `id` | integer/string | yes | The ID of a project or [URL-encoded NAMESPACE/PROJECT_NAME of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | -| `filter` | hash | no | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|-------------| +| `id` | integer/string | Yes | ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `key` | string | Yes | The `key` of a variable | +| `filter` | hash | No | Available filters: `[environment_scope]`. See the [`filter` parameter details](#the-filter-parameter). | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/VARIABLE_1" diff --git a/doc/api/project_relations_export.md b/doc/api/project_relations_export.md index 1fef2722bb8..e209259c6cc 100644 --- a/doc/api/project_relations_export.md +++ b/doc/api/project_relations_export.md @@ -6,14 +6,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project relations export API **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70330) in GitLab 14.4 behind the `bulk_import` [feature flag](../administration/feature_flags.md), disabled by default. -> - New application setting `bulk_import_enabled` introduced in GitLab 15.8. `bulk_import` feature flag removed. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70330) in GitLab 14.4 behind + the `bulk_import` [feature flag](../administration/feature_flags.md), disabled by default. +> - New application setting `bulk_import_enabled` introduced in GitLab 15.8. `bulk_import` feature + flag removed. -The project relations export API partially exports a project's structure as separate files for each top-level +The project relations export API partially exports a project's structure as separate files for each +top-level relation (for example, milestones, issues, and labels). The project relations export API is primarily used in -[group migration](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) can't be used with the +[group migration](../user/group/import/index.md#migrate-groups-by-direct-transfer-recommended) can't +be used with the [project import and export API](project_import_export.md). ## Schedule new export @@ -24,9 +28,10 @@ Start a new project relations export: POST /projects/:id/export_relations ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | +| Attribute | Type | Required | Description | +|-----------|----------------|----------|----------------------------------------------------| | `id` | integer/string | yes | ID of the project owned by the authenticated user. | +| `batched` | boolean | no | Whether to export in batches. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/export_relations" @@ -46,9 +51,10 @@ View the status of the relations export: GET /projects/:id/export_relations/status ``` -| Attribute | Type | Required | Description | -| --------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | ID of the project owned by the authenticated user. | +| Attribute | Type | Required | Description | +|------------|----------------|----------|----------------------------------------------------| +| `id` | integer/string | yes | ID of the project owned by the authenticated user. | +| `relation` | string | no | Name of the project top-level relation to view. | ```shell curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" \ @@ -71,13 +77,24 @@ The status can be one of the following: "relation": "project_badges", "status": 1, "error": null, - "updated_at": "2021-05-04T11:25:20.423Z" + "updated_at": "2021-05-04T11:25:20.423Z", + "batched": true, + "batches": [ + { + "status": 1, + "batch_number": 1, + "objects_count": 1, + "error": null, + "updated_at": "2021-05-04T11:25:20.423Z" + } + ] }, { "relation": "boards", "status": 1, "error": null, - "updated_at": "2021-05-04T11:25:20.085Z" + "updated_at": "2021-05-04T11:25:20.085Z", + "batched": false } ] ``` @@ -90,10 +107,12 @@ Download the finished relations export: GET /projects/:id/export_relations/download ``` -| Attribute | Type | Required | Description | -| --------------- | -------------- | -------- | ---------------------------------------- | -| `id` | integer/string | yes | ID of the project owned by the authenticated user. | -| `relation` | string | yes | Name of the project top-level relation to download. | +| Attribute | Type | Required | Description | +|----------------|----------------|----------|-----------------------------------------------------| +| `id` | integer/string | yes | ID of the project owned by the authenticated user. | +| `relation` | string | yes | Name of the project top-level relation to download. | +| `batched` | boolean | no | Whether the export is batched. | +| `batch_number` | integer | no | Number of export batch to download. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" --remote-header-name \ diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index e39836f2781..b82e46a03cc 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -14,7 +14,7 @@ You can set it with the `visibility` field in the snippet. Constants for snippet visibility levels are: - **Private**: The snippet is visible only to project members. -- **Internal**: The snippet is visible for any authenticated user except [external users](../user/admin_area/external_users.md). +- **Internal**: The snippet is visible for any authenticated user except [external users](../administration/external_users.md). - **Public**: The snippet can be accessed without any authentication. NOTE: diff --git a/doc/api/projects.md b/doc/api/projects.md index 5547546e6cc..6afed915135 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -304,6 +304,10 @@ Keyset pagination supports only `order_by=id`. Other sorting options aren't avai Get a list of visible projects owned by the given user. When accessed without authentication, only public projects are returned. +Prerequisite: + +- To view [certain attributes](https://gitlab.com/gitlab-org/gitlab/-/blob/520776fa8e5a11b8275b7c597d75246fcfc74c89/lib/api/entities/project.rb#L109-130), you must be an administrator or have the Owner role for the project. + NOTE: Only the projects in the user's (specified in `user_id`) namespace are returned. Projects owned by the user in any group or subgroups are not returned. An empty list is returned if a profile is set to private. @@ -578,6 +582,248 @@ GET /users/:user_id/projects ] ``` +## List projects a user has contributed to + +Get a list of visible projects a given user has contributed to. + +```plaintext +GET /users/:user_id/contributed_projects +``` + +| Attribute | Type | Required | Description | +|-------------------------------|---------|------------------------|-------------| +| `user_id` | string | **{check-circle}** Yes | The ID or username of the user. | +| `order_by` | string | **{dotted-circle}** No | Return projects ordered by `id`, `name`, `path`, `created_at`, `updated_at`, or `last_activity_at` fields. Default is `created_at`. | +| `simple` | boolean | **{dotted-circle}** No | Return only limited fields for each project. Without authentication, this operation is a no-op; only simple fields are returned. | +| `sort` | string | **{dotted-circle}** No | Return projects sorted in `asc` or `desc` order. Default is `desc`. | + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/users/5/contributed_projects" +``` + +Example response: + +```json +[ + { + "id": 4, + "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", + "description_html": "<p data-sourcepos=\"1:1-1:56\" dir=\"auto\">Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p>", + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git", + "web_url": "http://example.com/diaspora/diaspora-client", + "readme_url": "http://example.com/diaspora/diaspora-client/blob/master/README.md", + "tag_list": [ //deprecated, use `topics` instead + "example", + "disapora client" + ], + "topics": [ + "example", + "disapora client" + ], + "owner": { + "id": 3, + "name": "Diaspora", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Diaspora Client", + "name_with_namespace": "Diaspora / Diaspora Client", + "path": "diaspora-client", + "path_with_namespace": "diaspora/diaspora-client", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "can_create_merge_request_in": true, + "resolve_outdated_diff_discussions": false, + "container_registry_enabled": false, // deprecated, use container_registry_access_level instead + "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", + "created_at": "2013-09-30T13:46:02Z", + "updated_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 3, + "name": "Diaspora", + "path": "diaspora", + "kind": "group", + "full_path": "diaspora" + }, + "import_status": "none", + "archived": false, + "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png", + "shared_runners_enabled": true, + "group_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "allow_merge_on_skipped_pipeline": false, + "restrict_user_defined_variables": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "remove_source_branch_after_merge": false, + "request_access_enabled": false, + "merge_method": "merge", + "squash_option": "default_on", + "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, + "suggestion_commit_message": null, + "merge_commit_template": null, + "squash_commit_template": null, + "issue_branch_template": "gitlab/%{id}-%{title}", + "statistics": { + "commit_count": 37, + "storage_size": 1038090, + "repository_size": 1038090, + "lfs_objects_size": 0, + "job_artifacts_size": 0, + "pipeline_artifacts_size": 0, + "packages_size": 0, + "snippets_size": 0, + "uploads_size": 0 + }, + "container_registry_image_prefix": "registry.example.com/diaspora/diaspora-client", + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members", + "cluster_agents": "http://example.com/api/v4/projects/1/cluster_agents" + } + }, + { + "id": 6, + "description": "Lorem ipsum dolor sit amet, consectetur adipiscing elit.", + "description_html": "<p data-sourcepos=\"1:1-1:56\" dir=\"auto\">Lorem ipsum dolor sit amet, consectetur adipiscing elit.</p>", + "default_branch": "master", + "visibility": "private", + "ssh_url_to_repo": "git@example.com:brightbox/puppet.git", + "http_url_to_repo": "http://example.com/brightbox/puppet.git", + "web_url": "http://example.com/brightbox/puppet", + "readme_url": "http://example.com/brightbox/puppet/blob/master/README.md", + "tag_list": [ //deprecated, use `topics` instead + "example", + "puppet" + ], + "topics": [ + "example", + "puppet" + ], + "owner": { + "id": 4, + "name": "Brightbox", + "created_at": "2013-09-30T13:46:02Z" + }, + "name": "Puppet", + "name_with_namespace": "Brightbox / Puppet", + "path": "puppet", + "path_with_namespace": "brightbox/puppet", + "issues_enabled": true, + "open_issues_count": 1, + "merge_requests_enabled": true, + "jobs_enabled": true, + "wiki_enabled": true, + "snippets_enabled": false, + "can_create_merge_request_in": true, + "resolve_outdated_diff_discussions": false, + "container_registry_enabled": false, // deprecated, use container_registry_access_level instead + "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", + "created_at": "2013-09-30T13:46:02Z", + "updated_at": "2013-09-30T13:46:02Z", + "last_activity_at": "2013-09-30T13:46:02Z", + "creator_id": 3, + "namespace": { + "id": 4, + "name": "Brightbox", + "path": "brightbox", + "kind": "group", + "full_path": "brightbox" + }, + "import_status": "none", + "import_error": null, + "permissions": { + "project_access": { + "access_level": 10, + "notification_level": 3 + }, + "group_access": { + "access_level": 50, + "notification_level": 3 + } + }, + "archived": false, + "avatar_url": null, + "shared_runners_enabled": true, + "group_runners_enabled": true, + "forks_count": 0, + "star_count": 0, + "runners_token": "b8547b1dc37721d05889db52fa2f02", + "public_jobs": true, + "shared_with_groups": [], + "only_allow_merge_if_pipeline_succeeds": false, + "allow_merge_on_skipped_pipeline": false, + "restrict_user_defined_variables": false, + "only_allow_merge_if_all_discussions_are_resolved": false, + "remove_source_branch_after_merge": false, + "request_access_enabled": false, + "merge_method": "merge", + "squash_option": "default_on", + "auto_devops_enabled": true, + "auto_devops_deploy_strategy": "continuous", + "repository_storage": "default", + "approvals_before_merge": 0, // Deprecated. Use merge request approvals API instead. + "mirror": false, + "mirror_user_id": 45, + "mirror_trigger_builds": false, + "only_mirror_protected_branches": false, + "mirror_overwrites_diverged_branches": false, + "external_authorization_classification_label": null, + "packages_enabled": true, + "service_desk_enabled": false, + "service_desk_address": null, + "autoclose_referenced_issues": true, + "enforce_auth_checks_on_uploads": true, + "suggestion_commit_message": null, + "merge_commit_template": null, + "squash_commit_template": null, + "issue_branch_template": "gitlab/%{id}-%{title}", + "statistics": { + "commit_count": 12, + "storage_size": 2066080, + "repository_size": 2066080, + "lfs_objects_size": 0, + "job_artifacts_size": 0, + "pipeline_artifacts_size": 0, + "packages_size": 0, + "snippets_size": 0, + "uploads_size": 0 + }, + "container_registry_image_prefix": "registry.example.com/brightbox/puppet", + "_links": { + "self": "http://example.com/api/v4/projects", + "issues": "http://example.com/api/v4/projects/1/issues", + "merge_requests": "http://example.com/api/v4/projects/1/merge_requests", + "repo_branches": "http://example.com/api/v4/projects/1/repository_branches", + "labels": "http://example.com/api/v4/projects/1/labels", + "events": "http://example.com/api/v4/projects/1/events", + "members": "http://example.com/api/v4/projects/1/members", + "cluster_agents": "http://example.com/api/v4/projects/1/cluster_agents" + } + } +] +``` + ## List projects starred by a user > The `_links.cluster_agents` attribute in the response [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347047) in GitLab 14.10. @@ -1306,7 +1552,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your-token>" \ | `template_name` | string | **{dotted-circle}** No | When used without `use_custom_template`, name of a [built-in project template](../user/project/index.md#create-a-project-from-a-built-in-template). When used with `use_custom_template`, name of a custom project template. | | `template_project_id` **(PREMIUM)** | integer | **{dotted-circle}** No | When used with `use_custom_template`, project ID of a custom project template. Using a project ID is preferable to using `template_name` since `template_name` may be ambiguous. | | `topics` | array | **{dotted-circle}** No | The list of topics for a project; put array of topics, that should be finally assigned to a project. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328226) in GitLab 14.0.)_ | -| `use_custom_template` **(PREMIUM)** | boolean | **{dotted-circle}** No | Use either custom [instance](../user/admin_area/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template. | +| `use_custom_template` **(PREMIUM)** | boolean | **{dotted-circle}** No | Use either custom [instance](../administration/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template. | | `visibility` | string | **{dotted-circle}** No | See [project visibility level](#project-visibility-level). | | `wiki_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `wiki_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable wiki for this project. Use `wiki_access_level` instead. | @@ -1394,7 +1640,7 @@ POST /projects/user/:user_id | `tag_list` | array | **{dotted-circle}** No | _([Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/328226) in GitLab 14.0)_ The list of tags for a project; put array of tags, that should be finally assigned to a project. Use `topics` instead. | | `template_name` | string | **{dotted-circle}** No | When used without `use_custom_template`, name of a [built-in project template](../user/project/index.md#create-a-project-from-a-built-in-template). When used with `use_custom_template`, name of a custom project template. | | `topics` | array | **{dotted-circle}** No | The list of topics for the project. _([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328226) in GitLab 14.0.)_ | -| `use_custom_template` **(PREMIUM)** | boolean | **{dotted-circle}** No | Use either custom [instance](../user/admin_area/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template. | +| `use_custom_template` **(PREMIUM)** | boolean | **{dotted-circle}** No | Use either custom [instance](../administration/custom_project_templates.md) or [group](../user/group/custom_project_templates.md) (with `group_with_project_templates_id`) project template. | | `visibility` | string | **{dotted-circle}** No | See [project visibility level](#project-visibility-level). | | `wiki_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `wiki_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable wiki for this project. Use `wiki_access_level` instead. | @@ -2223,7 +2469,7 @@ This endpoint: - From [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/220382) on [Premium or Ultimate](https://about.gitlab.com/pricing/) tiers, delayed project deletion is enabled by default. The deletion happens after the number of days specified in the - [default deletion delay](../user/admin_area/settings/visibility_and_access_controls.md#deletion-protection). + [default deletion delay](../administration/settings/visibility_and_access_controls.md#deletion-protection). WARNING: The option to delete projects immediately from deletion protection settings in the Admin Area was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.9 and removed in GitLab 16.0. @@ -2916,15 +3162,9 @@ Example response: ## Configure pull mirroring for a project **(PREMIUM)** -> - Moved to GitLab Premium in GitLab 13.9. > - Field `mirror_branch_regex` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 15.8 [with a flag](../administration/feature_flags.md) named `mirror_only_branches_match_regex`. Disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default the field `mirror_branch_regex` is available. -To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) -named `mirror_only_branches_match_regex`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/410354) in GitLab 16.2. Feature flag `mirror_only_branches_match_regex` removed. Configure pull mirroring while [creating a new project](#create-project) or [updating an existing project](#edit-project) using the API diff --git a/doc/api/protected_branches.md b/doc/api/protected_branches.md index 4a82fab125d..0de32a4a25d 100644 --- a/doc/api/protected_branches.md +++ b/doc/api/protected_branches.md @@ -228,10 +228,10 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla | -------------------------------------------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | `name` | string | yes | The name of the branch or wildcard. -| `allow_force_push` | boolean | no | Allow all users with push access to force push. (default: `false`) -| `allowed_to_merge` **(PREMIUM)** | array | no | Array of access levels allowed to merge, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. -| `allowed_to_push` **(PREMIUM)** | array | no | Array of access levels allowed to push, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. -| `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of access levels allowed to unprotect, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. The access level `No access` is not available for this field. | +| `allow_force_push` | boolean | no | When enabled, members who can push to this branch can also force push. (default: `false`) +| `allowed_to_merge` **(PREMIUM)** | array | no | Array of merge access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. +| `allowed_to_push` **(PREMIUM)** | array | no | Array of push access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. +| `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of unprotect access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. The access level `No access` is not available for this field. | | `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). (defaults: false) | `merge_access_level` | integer | no | Access levels allowed to merge. (defaults: `40`, Maintainer role) | `push_access_level` | integer | no | Access levels allowed to push. (defaults: `40`, Maintainer role) @@ -458,12 +458,12 @@ curl --request PATCH --header "PRIVATE-TOKEN: <your_access_token>" "https://gitl | Attribute | Type | Required | Description | | -------------------------------------------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. -| `name` | string | yes | The name of the branch. -| `allow_force_push` | boolean | no | When enabled, members who can push to this branch can also force push. -| `allowed_to_push` **(PREMIUM)** | array | no | Array of push access levels, with each described by a hash. -| `allowed_to_merge` **(PREMIUM)** | array | no | Array of merge access levels, with each described by a hash. -| `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of unprotect access levels, with each described by a hash. The access level `No access` is not available for this field. -| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). Defaults to `false`. | +| `name` | string | yes | The name of the branch or wildcard. +| `allow_force_push` | boolean | no | When enabled, members who can push to this branch can also force push. +| `allowed_to_merge` **(PREMIUM)** | array | no | Array of merge access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. +| `allowed_to_push` **(PREMIUM)** | array | no | Array of push access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, or `{access_level: integer}`. +| `allowed_to_unprotect` **(PREMIUM)** | array | no | Array of unprotect access levels, with each described by a hash of the form `{user_id: integer}`, `{group_id: integer}`, `{access_level: integer}`, or `{id: integer, _destroy: true}` to destroy an existing access level. The access level `No access` is not available for this field. | +| `code_owner_approval_required` **(PREMIUM)** | boolean | no | Prevent pushes to this branch if it matches an item in the [`CODEOWNERS` file](../user/project/codeowners/index.md). | Elements in the `allowed_to_push`, `allowed_to_merge` and `allowed_to_unprotect` arrays should be one of `user_id`, `group_id` or `access_level`, and take the form `{user_id: integer}`, `{group_id: integer}` or @@ -510,7 +510,7 @@ Example response: ```shell curl --header 'Content-Type: application/json' --request PATCH \ - --data '{"allowed_to_push": [{"id": 12, "access_level": 0}]' \ + --data '{"allowed_to_push": [{"id": 12, "access_level": 0}]}' \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/22034114/protected_branches/master" ``` diff --git a/doc/api/releases/index.md b/doc/api/releases/index.md index 86c23283588..3ec4b77a646 100644 --- a/doc/api/releases/index.md +++ b/doc/api/releases/index.md @@ -435,7 +435,7 @@ GET /projects/:id/releases/:tag_name/downloads/:direct_asset_path Example request: ```shell -curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/24/releases/v0.1/downloads/bin/asset.exe" +curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/24/releases/v0.1/downloads/bin/asset.exe" ``` ### Get the latest release diff --git a/doc/api/remote_mirrors.md b/doc/api/remote_mirrors.md index b59619b3477..a755553b645 100644 --- a/doc/api/remote_mirrors.md +++ b/doc/api/remote_mirrors.md @@ -93,13 +93,7 @@ Learn how to [configure a pull mirror](projects.md#configure-pull-mirroring-for- > - Field `mirror_branch_regex` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 15.8 [with a flag](../administration/feature_flags.md) named `mirror_only_branches_match_regex`. Disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default the field `mirror_branch_regex` is available. -To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) -named `mirror_only_branches_match_regex`. -On GitLab.com, this feature is available. - +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/410354) in GitLab 16.2. Feature flag `mirror_only_branches_match_regex` removed. Push mirroring is disabled by default. To enable it, include the optional parameter `enabled` when you create the mirror: @@ -141,12 +135,6 @@ Example response: ## Update a remote mirror's attributes -FLAG: -On self-managed GitLab, by default the field `mirror_branch_regex` is not available. -To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) -named `mirror_only_branches_match_regex`. -On GitLab.com, this feature is not available. - Toggle a remote mirror on or off, or change which types of branches are mirrored: diff --git a/doc/api/repository_files.md b/doc/api/repository_files.md index 23243cdadc2..65ed67541d3 100644 --- a/doc/api/repository_files.md +++ b/doc/api/repository_files.md @@ -8,7 +8,7 @@ type: reference, api # Repository files API **(FREE)** You can fetch, create, update, and delete files in your repository with this API. -You can also [configure rate limits](../user/admin_area/settings/files_api_rate_limits.md) +You can also [configure rate limits](../administration/settings/files_api_rate_limits.md) for this API. ## Available scopes for personal access tokens @@ -19,6 +19,7 @@ in the following table. | Scope | Description | | ----- | ----------- | | `api` | Allows read-write access to the repository files. | +| `read_api` | Allows read access to the repository files. | | `read_repository` | Allows read-access to the repository files. | ## Get file from repository diff --git a/doc/api/rest/deprecations.md b/doc/api/rest/deprecations.md index 20fa999516e..80480e3f88f 100644 --- a/doc/api/rest/deprecations.md +++ b/doc/api/rest/deprecations.md @@ -110,3 +110,10 @@ A runner's status will only relate to runner contact status, such as: When checking if a runner is `paused`, API users are advised to check the boolean attribute `paused` to be `true` instead. When checking if a runner is `active`, check if `paused` is `false`. + +## Runner will not return `ip_address` + +Breaking change. [Related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/415159). + +In GitLab 17.0, the [Runners API](../runners.md) will return `""` in place of `ip_address` for runners. +In v5 of the REST API, the field will be removed. diff --git a/doc/api/runners.md b/doc/api/runners.md index 574bce82793..525cfaaff3e 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -776,13 +776,12 @@ Example response: } ``` -## Reset instance's runner registration token (deprecated) +## Reset instance's runner registration token -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383341) in GitLab 15.7 and is planned for removal in 17.0. This change is a breaking change. +Runner registration tokens, and support for certain configuration arguments, were [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. After GitLab 17.0, you will no longer be able to reset runner registration tokens and the `reset_registration_token` endpoint will not function. Reset the runner registration token for the GitLab instance. @@ -795,13 +794,12 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/runners/reset_registration_token" ``` -## Reset project's runner registration token (deprecated) +## Reset project's runner registration token -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383341) in GitLab 15.7 and is planned for removal in 17.0. This change is a breaking change. +Runner registration tokens, and support for certain configuration arguments, were [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. After GitLab 17.0, you will no longer be able to reset runner registration tokens and the `reset_registration_token` endpoint will not function. Reset the runner registration token for a project. @@ -814,13 +812,12 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/projects/9/runners/reset_registration_token" ``` -## Reset group's runner registration token (deprecated) +## Reset group's runner registration token -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104691) in GitLab 15.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 14.3. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/383341) in GitLab 15.7 and is planned for removal in 17.0. This change is a breaking change. +Runner registration tokens, and support for certain configuration arguments, were [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. After GitLab 17.0, you will no longer be able to reset runner registration tokens and the `reset_registration_token` endpoint will not function. Reset the runner registration token for a group. diff --git a/doc/api/saml.md b/doc/api/saml.md index b1d7692fbaf..168c6ed2b6a 100644 --- a/doc/api/saml.md +++ b/doc/api/saml.md @@ -4,7 +4,7 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SAML API **(PREMIUM SAAS)** +# SAML API **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227841) in GitLab 15.5. @@ -35,7 +35,7 @@ response attributes: Example request: ```shell -curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/saml/identities" --header "<PRIVATE-TOKEN>" +curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/saml/identities" --header "PRIVATE-TOKEN: <PRIVATE-TOKEN>" ``` Example response: @@ -67,7 +67,7 @@ Supported attributes: Example request: ```shell -curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/saml/sydney_jones" --header "<PRIVATE TOKEN>" +curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/saml/sydney_jones" --header "PRIVATE-TOKEN: <PRIVATE TOKEN>" ``` Example response: @@ -102,6 +102,6 @@ Example request: ```shell curl --location --request PATCH "https://gitlab.example.com/api/v4/groups/33/saml/sydney_jones" \ ---header "<PRIVATE TOKEN>" \ ---form "extern_uid=sydney_jones_new" \ +--header "PRIVATE-TOKEN: <PRIVATE TOKEN>" \ +--form "extern_uid=sydney_jones_new" ``` diff --git a/doc/api/scim.md b/doc/api/scim.md index df0d90756d2..95e4435f0d7 100644 --- a/doc/api/scim.md +++ b/doc/api/scim.md @@ -4,7 +4,7 @@ stage: Manage group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SCIM API **(PREMIUM SAAS)** +# SCIM API **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98354) in GitLab 15.5. @@ -76,7 +76,7 @@ Supported attributes: Example request: ```shell -curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/scim/sydney_jones" --header "<PRIVATE TOKEN>" +curl --location --request GET "https://gitlab.example.com/api/v4/groups/33/scim/sydney_jones" --header "PRIVATE-TOKEN: <PRIVATE TOKEN>" ``` Example response: diff --git a/doc/api/search.md b/doc/api/search.md index f7fa7babe71..b412d86a613 100644 --- a/doc/api/search.md +++ b/doc/api/search.md @@ -292,7 +292,8 @@ Example response: "id": null, "ref": "master", "startline": 5, - "project_id": 6 + "project_id": 6, + "group_id": null } ] ``` @@ -682,7 +683,8 @@ Example response: "id": null, "ref": "master", "startline": 5, - "project_id": 6 + "project_id": 6, + "group_id": 1 } ] ``` @@ -1094,7 +1096,8 @@ Example response: "id": null, "ref": "master", "startline": 5, - "project_id": 6 + "project_id": 6, + "group_id": 1 } ] ``` diff --git a/doc/api/secure_files.md b/doc/api/secure_files.md index 0b85bf05410..c912746f07e 100644 --- a/doc/api/secure_files.md +++ b/doc/api/secure_files.md @@ -29,9 +29,9 @@ GET /projects/:project_id/secure_files Supported attributes: -| Attribute | Type | Required | Description | -|--------------|----------------|------------------------|-------------| -| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `project_id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | Example request: @@ -90,10 +90,10 @@ GET /projects/:project_id/secure_files/:id Supported attributes: -| Attribute | Type | Required | Description | -|--------------|----------------|------------------------|-------------| -| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `project_id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `id` | integer | Yes | The ID of a secure file. | Example request: @@ -125,11 +125,11 @@ POST /projects/:project_id/secure_files Supported attributes: -| Attribute | Type | Required | Description | -|-----------------|----------------|------------------------|-------------| -| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `name` | string | **{check-circle}** Yes | The `name` of the file being uploaded. The filename must be unique within the project. | -| `file` | file | **{check-circle}** Yes | The `file` being uploaded (5 MB limit). | +| Attribute | Type | Required | Description | +|-----------------|----------------|----------|-------------| +| `project_id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `name` | string | Yes | The name of the file being uploaded. The filename must be unique in the project. | +| `file` | file | Yes | The file being uploaded (5 MB limit). | Example request: @@ -162,10 +162,10 @@ GET /projects/:project_id/secure_files/:id/download Supported attributes: -| Attribute | Type | Required | Description | -|--------------|----------------|------------------------|-------------| -| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `project_id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `id` | integer | Yes | The ID of a secure file. | Example request: @@ -183,10 +183,10 @@ DELETE /projects/:project_id/secure_files/:id Supported attributes: -| Attribute | Type | Required | Description | -|--------------|----------------|------------------------|-------------| -| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) owned by the authenticated user. | -| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | +| Attribute | Type | Required | Description | +|--------------|----------------|----------|-------------| +| `project_id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding). | +| `id` | integer | Yes | The ID of a secure file. | Example request: diff --git a/doc/api/settings.md b/doc/api/settings.md index ab78b9b7c74..ea023a25a1d 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -57,7 +57,6 @@ Example response: "default_project_visibility" : "private", "default_group_visibility" : "private", "gravatar_enabled" : true, - "sign_in_text" : null, "container_expiration_policies_enable_historic_entries": true, "container_registry_cleanup_tags_service_max_list_size": 200, "container_registry_delete_tags_service_timeout": 250, @@ -115,9 +114,20 @@ Example response: ``` Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) may also see -the `group_owners_can_manage_default_branch_protection`, `file_template_project_id`, `delayed_project_deletion`, -`delayed_group_deletion`, `default_project_deletion_protection`, `deletion_adjourned_period`, `disable_personal_access_tokens`, `geo_node_allowed_ips`, -or the `security_policy_global_group_approvers_enabled` parameters. +these parameters: + +- `group_owners_can_manage_default_branch_protection` +- `file_template_project_id` +- `geo_node_allowed_ips` +- `geo_status_timeout` +- `delayed_project_deletion` +- `delayed_group_deletion` +- `default_project_deletion_protection` +- `deletion_adjourned_period` +- `disable_personal_access_tokens` +- `security_policy_global_group_approvers_enabled` +- `delete_unconfirmed_users` +- `unconfirmed_users_delete_after_days` From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, the `delayed_project_deletion` and `delayed_group_deletion` attributes will not be exposed. These attributes will be removed in GitLab 16.0. @@ -161,7 +171,6 @@ Example response: "signup_enabled": false, "password_authentication_enabled_for_web": true, "gravatar_enabled": true, - "sign_in_text": "", "created_at": "2015-06-12T15:51:55.432Z", "updated_at": "2015-06-30T13:22:42.210Z", "home_page_url": "", @@ -257,6 +266,8 @@ these parameters: - `deletion_adjourned_period` - `disable_personal_access_tokens` - `security_policy_global_group_approvers_enabled` +- `delete_unconfirmed_users` +- `unconfirmed_users_delete_after_days` From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, the `delayed_project_deletion` and `delayed_group_deletion` attributes will not be exposed. These attributes will be removed in GitLab 16.0. @@ -270,7 +281,8 @@ Example responses: **(PREMIUM SELF)** ## List of settings that can be accessed via API calls -> Fields `housekeeping_full_repack_period`, `housekeeping_gc_period`, and `housekeeping_incremental_repack_period` [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106963) in GitLab 15.8. Use `housekeeping_optimize_repository_period` instead. +> - Fields `housekeeping_full_repack_period`, `housekeeping_gc_period`, and `housekeeping_incremental_repack_period` [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/106963) in GitLab 15.8. Use `housekeeping_optimize_repository_period` instead. +> - Parameters `sign_in_text` and `help_text` were [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124461) in GitLab 16.2. Use `description` parameter in the [Appearance API](../api/appearance.md) instead. In general, all settings are optional. Certain settings though, if enabled, require other settings to be set to function properly. These requirements are @@ -279,8 +291,8 @@ listed in the descriptions of the relevant settings. | Attribute | Type | Required | Description | |------------------------------------------|------------------|:------------------------------------:|-------------| | `admin_mode` | boolean | no | Require administrators to enable Admin Mode by re-authenticating for administrative tasks. | -| `admin_notification_email` | string | no | Deprecated: Use `abuse_notification_email` instead. If set, [abuse reports](../user/admin_area/review_abuse_reports.md) are sent to this address. Abuse reports are always available in the Admin Area. | -| `abuse_notification_email` | string | no | If set, [abuse reports](../user/admin_area/review_abuse_reports.md) are sent to this address. Abuse reports are always available in the Admin Area. | +| `admin_notification_email` | string | no | Deprecated: Use `abuse_notification_email` instead. If set, [abuse reports](../administration/review_abuse_reports.md) are sent to this address. Abuse reports are always available in the Admin Area. | +| `abuse_notification_email` | string | no | If set, [abuse reports](../administration/review_abuse_reports.md) are sent to this address. Abuse reports are always available in the Admin Area. | | `after_sign_out_path` | string | no | Where to redirect users after logout. | | `after_sign_up_text` | string | no | Text shown to the user after signing up. | | `akismet_api_key` | string | required by: `akismet_enabled` | API key for Akismet spam protection. | @@ -299,8 +311,8 @@ listed in the descriptions of the relevant settings. | `authorized_keys_enabled` | boolean | no | By default, we write to the `authorized_keys` file to support Git over SSH without additional configuration. GitLab can be optimized to authenticate SSH keys via the database file. Only disable this if you have configured your OpenSSH server to use the AuthorizedKeysCommand. | | `auto_devops_domain` | string | no | Specify a domain to use by default for every project's Auto Review Apps and Auto Deploy stages. | | `auto_devops_enabled` | boolean | no | Enable Auto DevOps for projects by default. It automatically builds, tests, and deploys applications based on a predefined CI/CD configuration. | -| `automatic_purchased_storage_allocation` | boolean | no | Enabling this permits automatic allocation of purchased storage in a namespace. | -| `bulk_import_enabled` | boolean | no | Enable migrating GitLab groups by direct transfer. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383268) in GitLab 15.8. Setting also [available](../user/admin_area/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer) in the Admin Area. | +| `automatic_purchased_storage_allocation` | boolean | no | Enabling this permits automatic allocation of purchased storage in a namespace. Relevant only to EE distributions. | +| `bulk_import_enabled` | boolean | no | Enable migrating GitLab groups by direct transfer. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383268) in GitLab 15.8. Setting also [available](../administration/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer) in the Admin Area. | | `can_create_group` | boolean | no | Indicates whether users can create top-level groups. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367754) in GitLab 15.5. Defaults to `true`. | | `check_namespace_plan` **(PREMIUM)** | boolean | no | Enabling this makes only licensed EE features available to projects if the project namespace's plan includes the feature or if the project is public. | | `ci_max_includes` | integer | no | The maximum number of [includes](../ci/yaml/includes.md) per pipeline. Default is `150`. | @@ -312,7 +324,7 @@ listed in the descriptions of the relevant settings. | `container_registry_expiration_policies_worker_capacity` | integer | no | Number of workers for [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#set-cleanup-limits-to-conserve-resources). | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes. | | `package_registry_cleanup_policies_worker_capacity` | integer | no | Number of workers assigned to the packages cleanup policies. | -| `deactivate_dormant_users` | boolean | no | Enable [automatic deactivation of dormant users](../user/admin_area/moderate_users.md#automatically-deactivate-dormant-users). | +| `deactivate_dormant_users` | boolean | no | Enable [automatic deactivation of dormant users](../administration/moderate_users.md#automatically-deactivate-dormant-users). | | `deactivate_dormant_users_period` | integer | no | Length of time (in days) after which a user is considered dormant. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336747) in GitLab 15.3. | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts. | | `default_branch_name` | string | no | [Instance-level custom initial branch name](../user/project/repository/branches/default.md#instance-level-custom-initial-branch-name). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225258) in GitLab 13.2. | @@ -328,12 +340,13 @@ listed in the descriptions of the relevant settings. | `delayed_project_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed project deletion by default in new groups. Default is `false`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), can only be enabled when `delayed_group_deletion` is true. From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, this attribute has been removed. This attribute will be completely removed in GitLab 16.0. | | `delayed_group_deletion` **(PREMIUM SELF)** | boolean | no | Enable delayed group deletion. Default is `true`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352959) in GitLab 15.0. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), disables and locks the group-level setting for delayed protect deletion when set to `false`. From [GitLab 15.11](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/113332), with the `always_perform_delayed_deletion` feature flag enabled, this attribute has been removed. This attribute will be completely removed in GitLab 16.0. | | `default_project_deletion_protection` **(PREMIUM SELF)** | boolean | no | Enable default project deletion protection so only administrators can delete projects. Default is `false`. | +| `delete_unconfirmed_users` **(PREMIUM SELF)** | boolean | no | Specifies whether users who have not confirmed their email should be deleted. Default is `false`. When set to `true`, unconfirmed users are deleted after `unconfirmed_users_delete_after_days` days. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352514) in GitLab 16.1. | | `deletion_adjourned_period` **(PREMIUM SELF)** | integer | no | The number of days to wait before deleting a project or group that is marked for deletion. Value must be between `1` and `90`. Defaults to `7`. [From GitLab 15.1](https://gitlab.com/gitlab-org/gitlab/-/issues/352960), a hook on `deletion_adjourned_period` sets the period to `1` on every update, and sets both `delayed_project_deletion` and `delayed_group_deletion` to `false` if the period is `0`. | | `diagramsnet_enabled` | boolean | no | (If enabled, requires `diagramsnet_url`) Enable [Diagrams.net integration](../administration/integration/diagrams_net.md). Default is `true`. | | `diagramsnet_url` | string | required by: `diagramsnet_enabled` | The Diagrams.net instance URL for integration. | -| `diff_max_patch_bytes` | integer | no | Maximum [diff patch size](../user/admin_area/diff_limits.md), in bytes. | -| `diff_max_files` | integer | no | Maximum [files in a diff](../user/admin_area/diff_limits.md). | -| `diff_max_lines` | integer | no | Maximum [lines in a diff](../user/admin_area/diff_limits.md). | +| `diff_max_patch_bytes` | integer | no | Maximum [diff patch size](../administration/diff_limits.md), in bytes. | +| `diff_max_files` | integer | no | Maximum [files in a diff](../administration/diff_limits.md). | +| `diff_max_lines` | integer | no | Maximum [lines in a diff](../administration/diff_limits.md). | | `disable_admin_oauth_scopes` | boolean | no | Stops administrators from connecting their GitLab accounts to non-trusted OAuth 2.0 applications that have the `api`, `read_api`, `read_repository`, `write_repository`, `read_registry`, `write_registry`, or `sudo` scopes. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375043) in GitLab 15.6. | | `disable_feed_token` | boolean | no | Disable display of RSS/Atom and calendar feed tokens. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231493) in GitLab 13.7. | | `disable_personal_access_token` **(PREMIUM SELF)** | boolean | no | Disable personal access tokens. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/384201) in GitLab 15.7. | @@ -358,8 +371,10 @@ listed in the descriptions of the relevant settings. | `elasticsearch_indexed_field_length_limit` **(PREMIUM)** | integer | no | Maximum size of text fields to index by Elasticsearch. 0 value means no limit. This does not apply to repository and wiki indexing. | | `elasticsearch_indexed_file_size_limit_kb` **(PREMIUM)** | integer | no | Maximum size of repository and wiki files that are indexed by Elasticsearch. | | `elasticsearch_indexing` **(PREMIUM)** | boolean | no | Enable Elasticsearch indexing. | +| `elasticsearch_requeue_workers` **(PREMIUM)** | boolean | no | Enable automatic requeuing of indexing workers. This improves non-code indexing throughput by enqueuing Sidekiq jobs until all documents are processed. | | `elasticsearch_limit_indexing` **(PREMIUM)** | boolean | no | Limit Elasticsearch to index certain namespaces and projects. | | `elasticsearch_max_bulk_concurrency` **(PREMIUM)** | integer | no | Maximum concurrency of Elasticsearch bulk requests per indexing operation. This only applies to repository indexing operations. | +| `elasticsearch_worker_number_of_shards` **(PREMIUM)** | integer | no | Number of indexing worker shards. This improves non-code indexing throughput by enqueuing more parallel Sidekiq jobs. Default is `2`. | | `elasticsearch_max_bulk_size_mb` **(PREMIUM)** | integer | no | Maximum size of Elasticsearch bulk indexing requests in MB. This only applies to repository indexing operations. | | `elasticsearch_namespace_ids` **(PREMIUM)** | array of integers | no | The namespaces to index via Elasticsearch if `elasticsearch_limit_indexing` is enabled. | | `elasticsearch_project_ids` **(PREMIUM)** | array of integers | no | The projects to index via Elasticsearch if `elasticsearch_limit_indexing` is enabled. | @@ -400,7 +415,7 @@ listed in the descriptions of the relevant settings. | `help_page_hide_commercial_content` | boolean | no | Hide marketing-related entries from help. | | `help_page_support_url` | string | no | Alternate support URL for help page and help dropdown list. | | `help_page_text` | string | no | Custom text displayed on the help page. | -| `help_text` **(PREMIUM)** | string | no | GitLab server administrator information. | +| `help_text` **(PREMIUM)** | string | no | Deprecated: Use `description` parameter in the [Appearance API](../api/appearance.md). Custom text in sign-in page. | | `hide_third_party_offers` | boolean | no | Do not display offers from third parties in GitLab. | | `home_page_url` | string | no | Redirect to this URL when not logged in. | | `housekeeping_bitmaps_enabled` | boolean | no | Deprecated. Git pack file bitmap creation is always enabled and cannot be changed via API and UI. Always returns `true`. | @@ -431,6 +446,8 @@ listed in the descriptions of the relevant settings. | `metrics_method_call_threshold` | integer | no | A method call is only tracked when it takes longer than the given amount of milliseconds. | | `max_number_of_repository_downloads` **(ULTIMATE SELF)** | integer | no | Maximum number of unique repositories a user can download in the specified time period before they are banned. Default: 0, Maximum: 10,000 repositories. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87980) in GitLab 15.1. | | `max_number_of_repository_downloads_within_time_period` **(ULTIMATE SELF)** | integer | no | Reporting time period (in seconds). Default: 0, Maximum: 864000 seconds (10 days). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87980) in GitLab 15.1. | +| `max_yaml_depth` | integer | no | The maximum depth of nested CI/CD configuration added with the [`include` keyword](../ci/yaml/index.md#include). Default: `100`. | +| `max_yaml_size_bytes` | integer | no | The maximum size in bytes of a single CI/CD configuration file. Default: `1048576`. | | `git_rate_limit_users_allowlist` **(ULTIMATE SELF)** | array of strings | no | List of usernames excluded from Git anti-abuse rate limits. Default: `[]`, Maximum: 100 usernames. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90815) in GitLab 15.2. | | `git_rate_limit_users_alertlist` **(ULTIMATE SELF)** | array of integers | no | List of user IDs that are emailed when the Git abuse rate limit is exceeded. Default: `[]`, Maximum: 100 user IDs. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110201) in GitLab 15.9. | | `auto_ban_user_on_excessive_projects_download` **(ULTIMATE SELF)** | boolean | no | When enabled, users will get automatically banned from the application when they download more than the maximum number of unique projects in the time period specified by `max_number_of_repository_downloads` and `max_number_of_repository_downloads_within_time_period` respectively. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94153) in GitLab 15.4 | @@ -461,7 +478,7 @@ listed in the descriptions of the relevant settings. | `projects_api_rate_limit_unauthenticated` | integer | no | [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112283) in GitLab 15.10. Max number of requests per 10 minutes per IP address for unauthenticated requests to the [list all projects API](projects.md#list-all-projects). Default: 400. To disable throttling set to 0.| | `prometheus_metrics_enabled` | boolean | no | Enable Prometheus metrics. | | `protected_ci_variables` | boolean | no | CI/CD variables are protected by default. | -| `push_event_activities_limit` | integer | no | Maximum number of changes (branches or tags) in a single push above which a [bulk push event is created](../user/admin_area/settings/push_event_activities_limit.md). Setting to `0` does not disable throttling. | +| `push_event_activities_limit` | integer | no | Maximum number of changes (branches or tags) in a single push above which a [bulk push event is created](../administration/settings/push_event_activities_limit.md). Setting to `0` does not disable throttling. | | `push_event_hooks_limit` | integer | no | Maximum number of changes (branches or tags) in a single push above which webhooks and integrations are not triggered. Setting to `0` does not disable throttling. | | `rate_limiting_response_text` | string | no | When rate limiting is enabled via the `throttle_*` settings, send this plain text response when a rate limit is exceeded. 'Retry later' is sent if this is blank. | | `raw_blob_request_limit` | integer | no | Maximum number of requests per minute for each raw path (default is `300`). Set to `0` to disable throttling.| @@ -476,27 +493,27 @@ listed in the descriptions of the relevant settings. | `repository_size_limit` **(PREMIUM)** | integer | no | Size limit per repository (MB) | | `repository_storages_weighted` | hash of strings to integers | no | (GitLab 13.1 and later) Hash of names of taken from `gitlab.yml` to [weights](../administration/repository_storage_paths.md#configure-where-new-repositories-are-stored). New projects are created in one of these stores, chosen by a weighted random selection. | | `repository_storages` | array of strings | no | (GitLab 13.0 and earlier) List of names of enabled storage paths, taken from `gitlab.yml`. New projects are created in one of these stores, chosen at random. | -| `require_admin_approval_after_user_signup` | boolean | no | When enabled, any user that signs up for an account using the registration form is placed under a **Pending approval** state and has to be explicitly [approved](../user/admin_area/moderate_users.md) by an administrator. | +| `require_admin_approval_after_user_signup` | boolean | no | When enabled, any user that signs up for an account using the registration form is placed under a **Pending approval** state and has to be explicitly [approved](../administration/moderate_users.md) by an administrator. | | `require_two_factor_authentication` | boolean | no | (**If enabled, requires:** `two_factor_grace_period`) Require all users to set up Two-factor authentication. | | `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-Administrator users for groups, projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is `null` which means there is no restriction. | | `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. | | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes. | | `security_policy_global_group_approvers_enabled` | boolean | no | Whether to look up scan result policy approval groups globally or within project hierarchies. | | `shared_runners_enabled` | boolean | no | (**If enabled, requires:** `shared_runners_text` and `shared_runners_minutes`) Enable shared runners for new projects. | -| `shared_runners_minutes` **(PREMIUM)** | integer | required by: `shared_runners_enabled` | Set the maximum number of units of compute that a group can use on shared runners per month. | +| `shared_runners_minutes` **(PREMIUM)** | integer | required by: `shared_runners_enabled` | Set the maximum number of compute minutes that a group can use on shared runners per month. | | `shared_runners_text` | string | required by: `shared_runners_enabled` | Shared runners text. | -| `sidekiq_job_limiter_mode` | string | no | `track` or `compress`. Sets the behavior for [Sidekiq job size limits](../user/admin_area/settings/sidekiq_job_limits.md). Default: 'compress'. | +| `sidekiq_job_limiter_mode` | string | no | `track` or `compress`. Sets the behavior for [Sidekiq job size limits](../administration/settings/sidekiq_job_limits.md). Default: 'compress'. | | `sidekiq_job_limiter_compression_threshold_bytes` | integer | no | The threshold in bytes at which Sidekiq jobs are compressed before being stored in Redis. Default: 100,000 bytes (100 KB). | | `sidekiq_job_limiter_limit_bytes` | integer | no | The threshold in bytes at which Sidekiq jobs are rejected. Default: 0 bytes (doesn't reject any job). | -| `sign_in_text` | string | no | Text on the login page. | +| `sign_in_text` | string | no | Deprecated: Use `description` parameter in the [Appearance API](../api/appearance.md). Custom text in sign-in page. | | `signin_enabled` | string | no | (Deprecated: Use `password_authentication_enabled_for_web` instead) Flag indicating if password authentication is enabled for the web interface. | | `signup_enabled` | boolean | no | Enable registration. Default is `true`. | | `silent_mode_enabled` | boolean | no | Enable [Silent mode](../administration/silent_mode/index.md). Default is `false`. | -| `slack_app_enabled` | boolean | no | (**If enabled, requires:** `slack_app_id`, `slack_app_secret` and `slack_app_secret`) Enable Slack app. | -| `slack_app_id` | string | required by: `slack_app_enabled` | The app ID of the Slack-app. | -| `slack_app_secret` | string | required by: `slack_app_enabled` | The app secret of the Slack-app. | -| `slack_app_signing_secret` | string | no | The signing secret of the Slack-app. | -| `slack_app_verification_token` | string | required by: `slack_app_enabled` | The verification token of the Slack-app. | +| `slack_app_enabled` | boolean | no | (**If enabled, requires:** `slack_app_id`, `slack_app_secret`, `slack_app_signing_secret`, and `slack_app_verification_token`) Enable the GitLab for Slack app. | +| `slack_app_id` | string | required by: `slack_app_enabled` | The client ID of the GitLab for Slack app. | +| `slack_app_secret` | string | required by: `slack_app_enabled` | The client secret of the GitLab for Slack app. Used for authenticating OAuth requests from the app. | +| `slack_app_signing_secret` | string | required by: `slack_app_enabled` | The signing secret of the GitLab for Slack app. Used for authenticating API requests from the app. | +| `slack_app_verification_token` | string | required by: `slack_app_enabled` | The verification token of the GitLab for Slack app. This method of authentication is deprecated by Slack and used only for authenticating slash commands from the app. | | `snippet_size_limit` | integer | no | Max snippet content size in **bytes**. Default: 52428800 Bytes (50 MB).| | `snowplow_app_id` | string | no | The Snowplow site name / application ID. (for example, `gitlab`) | | `snowplow_collector_hostname` | string | required by: `snowplow_enabled` | The Snowplow collector hostname. (for example, `snowplow.trx.gitlab.net`) | @@ -514,9 +531,9 @@ listed in the descriptions of the relevant settings. | `throttle_authenticated_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_authenticated_api_period_in_seconds` and `throttle_authenticated_api_requests_per_period`) Enable authenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). | | `throttle_authenticated_api_period_in_seconds` | integer | required by:<br>`throttle_authenticated_api_enabled` | Rate limit period (in seconds). | | `throttle_authenticated_api_requests_per_period` | integer | required by:<br>`throttle_authenticated_api_enabled` | Maximum requests per period per user. | -| `throttle_authenticated_packages_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_authenticated_packages_api_period_in_seconds` and `throttle_authenticated_packages_api_requests_per_period`) Enable authenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | -| `throttle_authenticated_packages_api_period_in_seconds` | integer | required by:<br>`throttle_authenticated_packages_api_enabled` | Rate limit period (in seconds). View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | -| `throttle_authenticated_packages_api_requests_per_period` | integer | required by:<br>`throttle_authenticated_packages_api_enabled` | Maximum requests per period per user. View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | +| `throttle_authenticated_packages_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_authenticated_packages_api_period_in_seconds` and `throttle_authenticated_packages_api_requests_per_period`) Enable authenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | +| `throttle_authenticated_packages_api_period_in_seconds` | integer | required by:<br>`throttle_authenticated_packages_api_enabled` | Rate limit period (in seconds). View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | +| `throttle_authenticated_packages_api_requests_per_period` | integer | required by:<br>`throttle_authenticated_packages_api_enabled` | Maximum requests per period per user. View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | | `throttle_authenticated_web_enabled` | boolean | no | (**If enabled, requires:** `throttle_authenticated_web_period_in_seconds` and `throttle_authenticated_web_requests_per_period`) Enable authenticated web request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). | | `throttle_authenticated_web_period_in_seconds` | integer | required by:<br>`throttle_authenticated_web_enabled` | Rate limit period (in seconds). | | `throttle_authenticated_web_requests_per_period` | integer | required by:<br>`throttle_authenticated_web_enabled` | Maximum requests per period per user. | @@ -526,17 +543,19 @@ listed in the descriptions of the relevant settings. | `throttle_unauthenticated_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_unauthenticated_api_period_in_seconds` and `throttle_unauthenticated_api_requests_per_period`) Enable unauthenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). | | `throttle_unauthenticated_api_period_in_seconds` | integer | required by:<br>`throttle_unauthenticated_api_enabled` | Rate limit period in seconds. | | `throttle_unauthenticated_api_requests_per_period` | integer | required by:<br>`throttle_unauthenticated_api_enabled` | Max requests per period per IP. | -| `throttle_unauthenticated_packages_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_unauthenticated_packages_api_period_in_seconds` and `throttle_unauthenticated_packages_api_requests_per_period`) Enable authenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | -| `throttle_unauthenticated_packages_api_period_in_seconds` | integer | required by:<br>`throttle_unauthenticated_packages_api_enabled` | Rate limit period (in seconds). View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | -| `throttle_unauthenticated_packages_api_requests_per_period` | integer | required by:<br>`throttle_unauthenticated_packages_api_enabled` | Maximum requests per period per user. View [Package Registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) for more details. | +| `throttle_unauthenticated_packages_api_enabled` | boolean | no | (**If enabled, requires:** `throttle_unauthenticated_packages_api_period_in_seconds` and `throttle_unauthenticated_packages_api_requests_per_period`) Enable authenticated API request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | +| `throttle_unauthenticated_packages_api_period_in_seconds` | integer | required by:<br>`throttle_unauthenticated_packages_api_enabled` | Rate limit period (in seconds). View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | +| `throttle_unauthenticated_packages_api_requests_per_period` | integer | required by:<br>`throttle_unauthenticated_packages_api_enabled` | Maximum requests per period per user. View [Package Registry rate limits](../administration/settings/package_registry_rate_limits.md) for more details. | | `throttle_unauthenticated_web_enabled` | boolean | no | (**If enabled, requires:** `throttle_unauthenticated_web_period_in_seconds` and `throttle_unauthenticated_web_requests_per_period`) Enable unauthenticated web request rate limit. Helps reduce request volume (for example, from crawlers or abusive bots). | | `throttle_unauthenticated_web_period_in_seconds` | integer | required by:<br>`throttle_unauthenticated_web_enabled` | Rate limit period in seconds. | | `throttle_unauthenticated_web_requests_per_period` | integer | required by:<br>`throttle_unauthenticated_web_enabled` | Max requests per period per IP. | | `time_tracking_limit_to_hours` | boolean | no | Limit display of time tracking units to hours. Default is `false`. | | `two_factor_grace_period` | integer | required by: `require_two_factor_authentication` | Amount of time (in hours) that users are allowed to skip forced configuration of two-factor authentication. | +| `unconfirmed_users_delete_after_days` **(PREMIUM SELF)** | integer | no | Specifies how many days after sign-up to delete users who have not confirmed their email. Only applicable if `delete_unconfirmed_users` is set to `true`. Must be `1` or greater. Default is `7`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352514) in GitLab 16.1. | | `unique_ips_limit_enabled` | boolean | no | (**If enabled, requires:** `unique_ips_limit_per_user` and `unique_ips_limit_time_window`) Limit sign in from multiple IPs. | | `unique_ips_limit_per_user` | integer | required by: `unique_ips_limit_enabled` | Maximum number of IPs per user. | | `unique_ips_limit_time_window` | integer | required by: `unique_ips_limit_enabled` | How many seconds an IP is counted towards the limit. | +| `update_runner_versions_enabled` | boolean | no | Fetch GitLab Runner release version data from GitLab.com. For more information, see how to [determine which runners need to be upgraded](../ci/runners/runners_scope.md#determine-which-runners-need-to-be-upgraded). | | `usage_ping_enabled` | boolean | no | Every week GitLab reports license usage back to GitLab, Inc. | | `user_deactivation_emails_enabled` | boolean | no | Send an email to users upon account deactivation. | | `user_default_external` | boolean | no | Newly registered users are external by default. | @@ -550,6 +569,7 @@ listed in the descriptions of the relevant settings. | `wiki_page_max_content_bytes` | integer | no | Maximum wiki page content size in **bytes**. Default: 52428800 Bytes (50 MB). The minimum value is 1024 bytes. | | `jira_connect_application_key` | String | no | Application ID of the OAuth application that should be used to authenticate with the GitLab for Jira Cloud app | | `jira_connect_proxy_url` | String | no | URL of the GitLab instance that should be used as a proxy for the GitLab for Jira Cloud app | +| `gitlab_shell_operation_limit` | integer | no | Maximum number of Git operations per minute a user can perform. Default: `600`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412088) in GitLab 16.2. | ### Configure inactive project deletion diff --git a/doc/api/snippets.md b/doc/api/snippets.md index 39965ae91f3..578c72a0502 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -20,7 +20,7 @@ Valid values for snippet visibility levels are: | Visibility | Description | |:-----------|:----------------------------------------------------| | `private` | Snippet is visible only to the snippet creator. | -| `internal` | Snippet is visible for any authenticated user except [external users](../user/admin_area/external_users.md). | +| `internal` | Snippet is visible for any authenticated user except [external users](../administration/external_users.md). | | `public` | Snippet can be accessed without any authentication. | ## List all snippets for a user diff --git a/doc/api/templates/dockerfiles.md b/doc/api/templates/dockerfiles.md index e9d57061510..70c104612b8 100644 --- a/doc/api/templates/dockerfiles.md +++ b/doc/api/templates/dockerfiles.md @@ -16,7 +16,7 @@ in the GitLab repository. In [GitLab Premium and Ultimate](https://about.gitlab.com/pricing/) tiers, GitLab instance administrators can override templates in the -[Admin Area](../../user/admin_area/settings/instance_template_repository.md). +[Admin Area](../../administration/settings/instance_template_repository.md). ## List Dockerfile templates diff --git a/doc/api/templates/gitlab_ci_ymls.md b/doc/api/templates/gitlab_ci_ymls.md index 29ee93c4e45..69346f8ab3d 100644 --- a/doc/api/templates/gitlab_ci_ymls.md +++ b/doc/api/templates/gitlab_ci_ymls.md @@ -120,9 +120,9 @@ Get a single GitLab CI/CD YAML template. GET /templates/gitlab_ci_ymls/:key ``` -| Attribute | Type | Required | Description | -| ---------- | ------ | -------- | ------------------------------------- | -| `key` | string | yes | The key of the GitLab CI/CD YAML template | +| Attribute | Type | Required | Description | +|-----------|--------|----------|-------------| +| `key` | string | Yes | The key of the GitLab CI/CD YAML template | Example request: @@ -135,7 +135,7 @@ Example response: ```json { "name": "Ruby", - "content": "# This file is a template, and might need editing before it works on your project.\n# To contribute improvements to CI/CD templates, please follow the Development guide at:\n# https://docs.gitlab.com/ee/development/cicd/templates.html\n# This specific template is located at:\n# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml\n\n# Official language image. Look for the different tagged releases at:\n# https://hub.docker.com/r/library/ruby/tags/\nimage: ruby:latest\n\n# Pick zero or more services to be used on all builds.\n# Only needed when using a docker container to run your tests in.\n# Check out: https://docs.gitlab.com/ee/ci/services/index.html\nservices:\n - mysql:latest\n - redis:latest\n - postgres:latest\n\nvariables:\n POSTGRES_DB: database_name\n\n# Cache gems in between builds\ncache:\n paths:\n - vendor/ruby\n\n# This is a basic example for a gem or script which doesn't use\n# services such as redis or postgres\nbefore_script:\n - ruby -v # Print out ruby version for debugging\n # Uncomment next line if your rails app needs a JS runtime:\n # - apt-get update -q \u0026\u0026 apt-get install nodejs -yqq\n - bundle config set --local deployment true # Install dependencies into ./vendor/ruby\n - bundle install -j $(nproc)\n\n# Optional - Delete if not using `rubocop`\nrubocop:\n script:\n - rubocop\n\nrspec:\n script:\n - rspec spec\n\nrails:\n variables:\n DATABASE_URL: \"postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB\"\n script:\n - rails db:migrate\n - rails db:seed\n - rails test\n\n# This deploy job uses a simple deploy flow to Heroku, other providers, e.g. AWS Elastic Beanstalk\n# are supported too: https://github.com/travis-ci/dpl\ndeploy:\n stage: deploy\n environment: production\n script:\n - gem install dpl\n - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_PRODUCTION_KEY\n" + "content": "# This file is a template, and might need editing before it works on your project.\n# To contribute improvements to CI/CD templates, please follow the Development guide at:\n# https://docs.gitlab.com/ee/development/cicd/templates.html\n# This specific template is located at:\n# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml\n\n# Official language image. Look for the different tagged releases at:\n# https://hub.docker.com/r/library/ruby/tags/\nimage: ruby:latest\n\n# Pick zero or more services to be used on all builds.\n# Only needed when using a docker container to run your tests in.\n# Check out: https://docs.gitlab.com/ee/ci/services/index.html\nservices:\n - mysql:latest\n - redis:latest\n - postgres:latest\n\nvariables:\n POSTGRES_DB: database_name\n\n# Cache gems in between builds\ncache:\n paths:\n - vendor/ruby\n\n# This is a basic example for a gem or script which doesn't use\n# services such as redis or postgres\nbefore_script:\n - ruby -v # Print out ruby version for debugging\n # Uncomment next line if your rails app needs a JS runtime:\n # - apt-get update -q \u0026\u0026 apt-get install nodejs -yqq\n - bundle config set --local deployment true # Install dependencies into ./vendor/ruby\n - bundle install -j $(nproc)\n\n# Optional - Delete if not using `rubocop`\nrubocop:\n script:\n - rubocop\n\nrspec:\n script:\n - rspec spec\n\nrails:\n variables:\n DATABASE_URL: \"postgresql://postgres:postgres@postgres:5432/$POSTGRES_DB\"\n script:\n - rails db:migrate\n - rails db:seed\n - rails test\n\n# This deploy job uses a simple deploy flow to Heroku, other providers, for example, AWS Elastic Beanstalk\n# are supported too: https://github.com/travis-ci/dpl\ndeploy:\n stage: deploy\n environment: production\n script:\n - gem install dpl\n - dpl --provider=heroku --app=$HEROKU_APP_NAME --api-key=$HEROKU_PRODUCTION_KEY\n" } ``` diff --git a/doc/api/users.md b/doc/api/users.md index 7b418e7a08b..3fb9d655ff9 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -22,6 +22,21 @@ This function takes pagination parameters `page` and `per_page` to restrict the GET /users ``` +| Attribute | Type | Required | Description | +| ------------------ | ------- | -------- | ---------------------------------------------------------------------------------------------------------------------- | +| `username` | string | no | Get a single user with a specific username. | +| `extern_uid` | string | no | Get a single user with a specific external authentication provider UID. | +| `provider` | string | no | The external provider. | +| `search` | string | no | Search for a username. | +| `active` | boolean | no | Filters only active users. Default is `false`. | +| `external` | boolean | no | Filters only external users. Default is `false`. | +| `exclude_external` | boolean | no | Filters only non external users. Default is `false`. | +| `blocked` | boolean | no | Filters only blocked users. Default is `false`. | +| `created_after` | DateTime| no | Returns users created after specified time. | +| `created_before` | DateTime| no | Returns users created before specified time. | +| `exclude_internal` | boolean | no | Filters only non internal users. Default is `false`. | +| `without_project_bots`| boolean | no | Filters user without project bots. Default is `false`. | + ```json [ { @@ -121,6 +136,8 @@ GET /users?without_project_bots=true GET /users ``` +You can use all [parameters available for everyone](#for-non-administrator-users) plus these additional parameters available only for administrators. + | Attribute | Type | Required | Description | | ------------------ | ------- | -------- | --------------------------------------------------------------------------------------------------------------------- | | `order_by` | string | no | Return users ordered by `id`, `name`, `username`, `created_at`, or `updated_at` fields. Default is `id` | @@ -268,6 +285,12 @@ For example: GET /users?extern_uid=1234567&provider=github ``` +Users on [GitLab Premium or Ultimate](https://about.gitlab.com/pricing/) have the `scim` provider available: + +```plaintext +GET /users?extern_uid=1234567&provider=scim +``` + You can search users by creation date time range with: ```plaintext @@ -493,7 +516,7 @@ over `password`. In addition, `reset_password` and NOTE: From [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/29888/), `private_profile` defaults to `false`. -From [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/231301), `private_profile` defaults to the value determined by [this](../user/admin_area/settings/account_and_limit_settings.md#set-profiles-of-new-users-to-private-by-default) setting. +From [GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/issues/231301), `private_profile` defaults to the value determined by [this](../administration/settings/account_and_limit_settings.md#set-profiles-of-new-users-to-private-by-default) setting. NOTE: From [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35604), `bio` defaults to `""` instead of `null`. @@ -510,12 +533,12 @@ Parameters: | `auditor` **(PREMIUM)** | No | User is an auditor. Valid values are `true` or `false`. Defaults to false. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/366404) in GitLab 15.3. | | `avatar` | No | Image file for user's avatar | | `bio` | No | User's biography | -| `can_create_group` | No | User can create groups - true or false | +| `can_create_group` | No | User can create top-level groups - true or false | | `color_scheme_id` | No | User's color scheme for the file viewer (for more information, see the [user preference documentation](../user/profile/preferences.md#syntax-highlighting-theme)) | | `email` | Yes | Email | | `extern_uid` | No | External UID | | `external` | No | Flags the user as external - true or false (default) | -| `extra_shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Additional units of compute for this user. | +| `extra_shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Additional compute minutes for this user. | | `force_random_password` | No | Set user password to a random value - true or false (default) | | `group_id_for_saml` | No | ID of group where SAML has been configured | | `linkedin` | No | LinkedIn | @@ -524,11 +547,11 @@ Parameters: | `note` | No | Administrator notes for this user | | `organization` | No | Organization name | | `password` | No | Password | -| `private_profile` | No | User's profile is private - true or false. The default value is determined by [this](../user/admin_area/settings/account_and_limit_settings.md#set-profiles-of-new-users-to-private-by-default) setting. | +| `private_profile` | No | User's profile is private - true or false. The default value is determined by [this](../administration/settings/account_and_limit_settings.md#set-profiles-of-new-users-to-private-by-default) setting. | | `projects_limit` | No | Number of projects user can create | | `provider` | No | External provider name | | `reset_password` | No | Send user password reset link - true or false(default) | -| `shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Maximum number of monthly units of compute for this user. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | +| `shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Maximum number of monthly compute minutes for this user. Can be `nil` (default; inherit system default), `0` (unlimited), or `> 0`. | | `skip_confirmation` | No | Skip confirmation - true or false (default) | | `skype` | No | Skype ID | | `theme_id` | No | GitLab theme for the user (for more information, see the [user preference documentation](../user/profile/preferences.md#navigation-theme) for more information) | @@ -565,7 +588,7 @@ Parameters: | `email` | No | Email | | `extern_uid` | No | External UID | | `external` | No | Flags the user as external - true or false (default) | -| `extra_shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Additional units of compute for this user. | +| `extra_shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Additional compute minutes for this user. | | `group_id_for_saml` | No | ID of group where SAML has been configured | | `id` | Yes | ID of the user | | `linkedin` | No | LinkedIn | @@ -579,7 +602,7 @@ Parameters: | `pronouns` | No | Pronouns | | `provider` | No | External provider name | | `public_email` | No | Public email of the user (must be already verified) | -| `shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Maximum number of monthly units of compute for this user. Can be `nil` (default; inherit system default), `0` (unlimited) or `> 0`. | +| `shared_runners_minutes_limit` **(PREMIUM)** | No | Can be set by administrators only. Maximum number of monthly compute minutes for this user. Can be `nil` (default; inherit system default), `0` (unlimited) or `> 0`. | | `skip_reconfirmation` | No | Skip reconfirmation - true or false (default) | | `skype` | No | Skype ID | | `theme_id` | No | GitLab theme for the user (for more information, see the [user preference documentation](../user/profile/preferences.md#navigation-theme) for more information) | @@ -1008,7 +1031,7 @@ Example response: } ``` -## Create Service Account User **(PREMIUM)** +## Create Service Account User **(PREMIUM SELF)** > Ability to create a service account user was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/406782) in GitLab 16.1 @@ -1062,6 +1085,8 @@ Example response: Get a list of the authenticated user's SSH keys. +This function takes pagination parameters `page` and `per_page` to restrict the list of keys. + ```plaintext GET /user/keys ``` @@ -1738,7 +1763,7 @@ Returns: - `404 User Not Found` if user cannot be found. - `403 Forbidden` when trying to deactivate a user that is: - Blocked by administrator or by LDAP synchronization. - - Not [dormant](../user/admin_area/moderate_users.md#automatically-deactivate-dormant-users). + - Not [dormant](../administration/moderate_users.md#automatically-deactivate-dormant-users). - Internal. ## Activate user **(FREE SELF)** @@ -1907,7 +1932,7 @@ Example Responses: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339925) in GitLab 14.3. -Rejects specified user that is [pending approval](../user/admin_area/moderate_users.md#users-pending-approval). Available only for administrators. +Rejects specified user that is [pending approval](../administration/moderate_users.md#users-pending-approval). Available only for administrators. ```plaintext POST /users/:id/reject @@ -2051,6 +2076,7 @@ curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://git > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17176) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/267553) in GitLab 13.8. +> - The `expires_at` attribute default was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120213) in GitLab 16.0. Use this API to create a new personal access token. Token values are returned once so, make sure you save it as you can't access it again. This API can only be used by @@ -2064,7 +2090,7 @@ POST /users/:user_id/personal_access_tokens | ------------ | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------ | | `user_id` | integer | yes | ID of the user | | `name` | string | yes | Name of the personal access token | -| `expires_at` | date | no | Expiration date of the personal access token in ISO format (`YYYY-MM-DD`) | +| `expires_at` | date | no | Expiration date of the access token in ISO format (`YYYY-MM-DD`). If no date is set, the expiration is set to the [maximum allowable lifetime of an access token](../user/profile/personal_access_tokens.md#when-personal-access-tokens-expire). | | `scopes` | array | yes | Array of scopes of the personal access token. See [personal access token scopes](../user/profile/personal_access_tokens.md#personal-access-token-scopes) for possible values. | ```shell @@ -2243,6 +2269,7 @@ Prerequisites: - You must be an administrator or have the Owner role of the target namespace or project. - For `instance_type`, you must be an administrator of the GitLab instance. +- An access token with the `create_runner` scope. Be sure to copy or save the `token` in the response, the value cannot be retrieved again. diff --git a/doc/api/visual_review_discussions.md b/doc/api/visual_review_discussions.md index 496c732b337..f6d6636280a 100644 --- a/doc/api/visual_review_discussions.md +++ b/doc/api/visual_review_discussions.md @@ -29,23 +29,23 @@ POST /projects/:id/merge_requests/:merge_request_iid/visual_review_discussions Parameters: | Attribute | Type | Required | Description | -| ------------------------- | -------------- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of a merge request | -| `body` | string | yes | The content of the thread | -| `position` | hash | no | Position when creating a diff note | -| `position[base_sha]` | string | yes | Base commit SHA in the source branch | -| `position[start_sha]` | string | yes | SHA referencing commit in target branch | -| `position[head_sha]` | string | yes | SHA referencing HEAD of this merge request | -| `position[position_type]` | string | yes | Type of the position reference. Either `text` or `image`. | -| `position[new_path]` | string | no | File path after change | -| `position[new_line]` | integer | no | Line number after change (Only stored for `text` diff notes) | -| `position[old_path]` | string | no | File path before change | -| `position[old_line]` | integer | no | Line number before change (Only stored for `text` diff notes) | -| `position[width]` | integer | no | Width of the image (Only stored for `image` diff notes) | -| `position[height]` | integer | no | Height of the image (Only stored for `image` diff notes) | -| `position[x]` | integer | no | X coordinate (Only stored for `image` diff notes) | -| `position[y]` | integer | no | Y coordinate (Only stored for `image` diff notes) | +|---------------------------|----------------|----------|-------------| +| `id` | integer/string | Yes | The ID or [URL-encoded path of the project](rest/index.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | Yes | The IID of a merge request | +| `body` | string | Yes | The content of the thread | +| `position` | hash | No | Position when creating a diff note | +| `position[base_sha]` | string | Yes | Base commit SHA in the source branch | +| `position[start_sha]` | string | Yes | SHA referencing commit in target branch | +| `position[head_sha]` | string | Yes | SHA referencing HEAD of this merge request | +| `position[position_type]` | string | Yes | Type of the position reference. Either `text` or `image`. | +| `position[new_path]` | string | No | File path after change | +| `position[new_line]` | integer | No | Line number after change (Only stored for `text` diff notes) | +| `position[old_path]` | string | No | File path before change | +| `position[old_line]` | integer | No | Line number before change (Only stored for `text` diff notes) | +| `position[width]` | integer | No | Width of the image (Only stored for `image` diff notes) | +| `position[height]` | integer | No | Height of the image (Only stored for `image` diff notes) | +| `position[x]` | integer | No | X coordinate (Only stored for `image` diff notes) | +| `position[y]` | integer | No | Y coordinate (Only stored for `image` diff notes) | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/11/visual_review_discussions?body=comment" diff --git a/doc/architecture/blueprints/ai_gateway/img/architecture.png b/doc/architecture/blueprints/ai_gateway/img/architecture.png Binary files differnew file mode 100644 index 00000000000..dea8b5ddb45 --- /dev/null +++ b/doc/architecture/blueprints/ai_gateway/img/architecture.png diff --git a/doc/architecture/blueprints/ai_gateway/index.md b/doc/architecture/blueprints/ai_gateway/index.md new file mode 100644 index 00000000000..c9947723739 --- /dev/null +++ b/doc/architecture/blueprints/ai_gateway/index.md @@ -0,0 +1,463 @@ +--- +status: ongoing +creation-date: "2023-07-14" +authors: [ "@reprazent" ] +coach: [ "@andrewn", "@stanhu" ] +approvers: [ "@m_gill", "@mksionek", "@marin" ] +owning-stage: "~devops::modelops" +participating-stages: [] +--- + +<!-- vale gitlab.FutureTense = NO --> + +# AI-gateway + +## Summary + +The AI-gateway is a standalone-service that will give access to AI +features to all users of GitLab, no matter which instance they are +using: self-managed, dedicated or GitLab.com. + +Initially, all AI-gateway deployments will be managed by GitLab (the +organization), and GitLab.com and all GitLab self-managed instances +will use the same gateway. However, in the future we could also deploy +regional gateways, or even customer-specific gateways if the need +arises. + +The AI-Gateway is an API-Gateway that takes traffic from clients, in +this case GitLab installations, and directing it to different +services, in this case AI-providers and their models. This North/South +traffic pattern allows us to control what requests go where and to +translate the content of the redirected request where needed. + + + +[src of the architecture diagram](https://docs.google.com/drawings/d/1PYl5Q5oWHnQAuxM-Jcw0C3eYoGw8a9w8atFpoLhhEas/edit) + +By using a hosted service under the control of GitLab we can ensure +that we provide all GitLab instances with AI features in a scalable +way. It is easier to scale this small stateless service, than scaling +GitLab-rails with it's dependencies (database, Redis). + +It allows users of self-managed installations to have access to +features using AI without them having to host their own models or +connect to 3rd party providers. + +## Language: Python + +The AI-Gateway was originally started as the "model-gateway" that +handled requests from IDEs to provide code suggestions. It was written +in Python. + +Python is an object oriented language that is familiar enough for +Rubyists to pick up through in the younger codebase that is the +AI-gateway. It also makes it easy for data- and ML-engineers that +already have Python experience to contribute. + +## API + +### Basic stable API for the AI-gateway + +Because the API of the AI-gateway will be consumed by a wide variety +of GitLab instances, it is important that we design a stable, yet +flexible API. + +To do this, we can implement an API-endpoint per use-case we +build. This means that the interface between GitLab and the AI-gateway +is one that we build and own. This ensures future scalability, +composability and security. + +The API is not versioned, but is backward compatible. See [cross version compatibility](#cross-version-compatibility) +for details. The AI-gateway will support the last 2 major +versions. For example when working on GitLab 17.2, we would support +both GitLab 17 and GitLab 16. + +We can add common functionality like rate-limiting, circuit-breakers and +secret redaction at this level of the stack as well as in GitLab-rails. + +#### Protocol + +We're choosing to use a simple JSON API for the AI-gateway +service. This allows us to re-use a lot of what is already in place in +the current model-gateway. It also allows us to make the endpoints +version agnostic. We could have an API that expects only a rudimentary +envelope that can contain dynamic information. We should make sure +that we make these APIs compatible with multiple versions of GitLab, +or other clients that use the gateway through GitLab. **This means +that all client versions talk to the same API endpoint, the AI-gateway +needs to support this, but we don't need to support different +endpoints per version**. + +We also considered gRPC as a the protocol for communication between +GitLab instances, they differ on these items: + +| gRPC | REST + JSON | +|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------| +| + Strict protocol definition that is easier to evolve versionless | - No strict schema, so the implementation needs to take good care of supporting multiple versions | +| + A new Ruby-gRPC server for vscode: likely faster because we can limit dependencies to load ([modular monolith](https://gitlab.com/gitlab-org/gitlab/-/issues/365293)) | - Existing Grape API for vscode: meaning slow boot time and unneeded resources loaded | +| + Bi-directional streaming | - Straight forward way to stream requests and responses (could still be added) | +| - A new Python-gRPC server: we don't have experience running gRPC-Python servers | + Existing Python fastapi server, already running for code suggestions to extend | +| - Hard to pass on unknown messages from vscode through GitLab to ai-gateway | + Easier support for newer vscode + newer ai-gatway, through old GitLab instance | +| - Unknown support for gRPC in other clients (vscode, jetbrains, other editors) | + Support in all external clients | +| - Possible protocol mismatch (VSCode --REST--> Rails --gRPC--> AI gateway) | + Same protocol across the stack | + +**Discussion:** Because we chose REST+JSON in this iteration to port +features that already partially exist does not mean we need to exclude +new features using gRPC or Websockets. For example: Chat features +might be better served by streaming requests and responses. Since we +are suggesting an endpoint per use-case, different features could also +opt for different protocols, as long as we keep cross-version +compatibility in mind. + +#### Single purpose endpoints + +For features using AI, we prefer building a single purpose endpoint +with a stable API over the [provider API we expose](#exposing-ai-providers) +as a direct proxy. + +Some features will have specific endpoints, while others can share +endpoints. For example, code suggestions or chat could have their own +endpoint, while several features that summarize issues or merge +requests could use the same endpoint but make the distinction on what +information is provided in the payload. + +The end goal is to build an API that exposes AI for building +features without having to touch the AI-gateway. This is analogous to +how we built Gitaly, adding features to Gitaly where it was needed, +and reusing existing endpoints when that was possible. We had some +cost to pay up-front in the case where we needed to implement a new +endpoint (RPC), but pays off in the long run when most of the required +functionality is implemented. + +**This does not mean that prompts need to be built inside the +AI-gateway.** But if prompts are part of the payload to a single +purpose endpoint, the payload needs to specify which model they were +built for along with other metadata about the prompts. By doing this, +we can gracefully degrade or otherwise try to support the request if +one of the prompt payloads is no longer supported by the AI +gateway. It allows us to potentially avoid breaking features in older +GitLab installations as the AI landscape changes. + +#### Cross version compatibility + +When building single purpose endpoints, we should be mindful that +these endpoints will be used by different GitLab instances indirectly +by external clients. To achieve this, we have a very simple envelope +to provide information. It has to have a series of `prompt_components` +that contain information the AI-gateway can use to build prompts and +query the model of it selects. + +Each prompt component contains 3 elements: + +- `type`: This is the kind of information that is being presented in + `payload`. The AI-gateway should ignore any types it does not know + about. +- `payload`: The actual information that can be used by the AI-gateway + to build the payload that is going to go out to AI providers. The + payload will be different depending on the type, and the version of + the client providing the payload. This means that the AI-gateway + needs to consider all fields optional. +- `metadata`: Information about the client that built this part of the + prompt. This may, or may not be used by GitLab for + telemetry. Nothing inside this field should be required. + +The only component in there that is likely to change often is the +`payload` one. There we need to make sure that all fields are +optional, and avoid renaming, removing or repurposing fields. + +When this is needed, we need to build support for the old versions of +a field in the gateway, and keep them around for at least 2 major +versions of GitLab. For example, we could consider adding 2 versions +of a prompt to the `prompt_components` payload: + +```json +{ + prompt_components: [ + { + "type": "prompt", + "metadata": { + "source": "GitLab EE", + "version": "16.3", + }, + "payload": { + "content": "You can fetch information about a resource called an issue...", + "params": { + "temperature": 0.2, + "maxOutputTokens": 1024 + }, + "model": "text-bison", + "provider": "vertex-ai" + } + }, + { + "type": "prompt", + "metadata": { + "source": "GitLab EE", + "version": "16.3", + }, + "payload": { + "content": "System: You can fetch information about a resource called an issue...\n\nHuman:", + "params": { + "temperature": 0.2, + }, + "model": "claude-2", + "provider": "anthropic" + } + } + + ] +} +``` + +Allowing the API to direct the prompt to either provider, based on +what is in the payload. + +To document and validate the content of `payload` we can specify their +format using [JSON-schema](https://json-schema.org/). + +#### Example feature: code suggestions + +For example, a rough code suggestions service could look like this: + +```plaintext +POST /internal/code-suggestions/completions +``` + +```json +{ + "prompt_components": [ + { + "type": "prompt", + "metadata": { + "source": "GitLab EE", + "version": "16.3", + }, + "payload": { + "content": "...", + "params": { + "temperature": 0.2, + "maxOutputTokens": 1024 + }, + "model": "code-gecko", + "provider": "vertex-ai" + } + }, + { + "type": "editor_content", + "metadata": { + "source": "vscode", + "version": "1.1.1" + }, + "payload": { + "filename": "application.rb", + "before_cursor": "require 'active_record/railtie'", + "after_cursor": "\nrequire 'action_controller/railtie'", + "open_files": [ + { + "filename": "app/controllers/application_controller.rb", + "content": "class ApplicationController < ActionController::Base..." + } + ] + } + } + ] +} +``` + +A response could look like this: + +```json +{ + "response": "require 'something/else'", + "metadata": { + "identifier": "deadbeef", + "model": "code-gecko", + "timestamp": 1688118443 + } +} +``` + +The `metadata` field contains information that could be used in a +telemetry endpoint on the AI-gateway where we could count +suggestion-acceptance rates among other things. + +The way we will receive telemetry for code suggestions is being +discussed in [#415745](https://gitlab.com/gitlab-org/gitlab/-/issues/415745). +We will try to come up with an architecture for all AI-related features. + +#### Exposing AI providers + +A lot of AI functionality has already been built into GitLab-Rails +that currently builds prompts and submits this directly to different +AI providers. At the time of writing, GitLab has API-clients for the +following providers: + +- [Anthropic](https://gitlab.com/gitlab-org/gitlab/blob/4344729240496a5018e19a82030d6d4b227e9c79/ee/lib/gitlab/llm/anthropic/client.rb#L6) +- [Vertex](https://gitlab.com/gitlab-org/gitlab/blob/4344729240496a5018e19a82030d6d4b227e9c79/ee/lib/gitlab/llm/vertex_ai/client.rb#L6) +- [OpenAI](https://gitlab.com/gitlab-org/gitlab/blob/4344729240496a5018e19a82030d6d4b227e9c79/ee/lib/gitlab/llm/open_ai/client.rb#L8) + +To make these features available to self-managed instances, we should +provide endpoints for each of these that GitLab.com, self-managed or +dedicated installations can use to give these customers to these +features. + +In a first iteration we could build endpoints that proxy the request +to the AI provider. This should make it easier to migrate to routing +these requests through the AI-Gateway. As an example, the endpoint for +Anthropic could look like this: + +```plaintext +POST /internal/proxy/anthropic/(*endpoint) +``` + +The `*endpoint` means that the client specifies what is going to be +called, for example `/v1/complete`. The request body is entirely +forwarded to the AI provider. The AI-gateway makes sure the request is +correctly authenticated. + +Having the proxy in between GitLab and the AI provider means that we +still have control over what goes through to the AI provider and if +the need arises, we can manipulate or reroute the request to a +different provider. Doing this means that we could keep supporting +the features of older GitLab installations even if the provider's API +changes or we decide not to work with a certain provider anymore. + +I think there is value in moving features that use API providers +directly to a feature-specific purpose built API. Doing this means +that we can improve these features as AI providers evolve by changing +the AI-gateway that is under our control. Customers using self-managed +or dedicated installations could then start getting better +AI-supported features without having to upgrade their GitLab instance. + +Features that are currently +[experimental](../../../policy/experiment-beta-support.md#experiment) +can use these generic APIs, but we should aim to convert to a single +purpose API endpoint before we make the feature [generally available](../../../policy/experiment-beta-support.md#generally-available-ga) +for self-managed installations. This makes it easier for us to support +features long-term even if the landscape of AI providers change. + +The [Experimental REST API](../../../development/ai_features.md#experimental-rest-api) +available to GitLab team members should also use this proxy in the +short term. In the longer term, we should provide developers access to +a separate proxy that allows them to use GitLab owned authentication +to several AI providers for experimentation. This will separate the +traffic from developers trying out new things from the fleet that is +serving paying customers. + +### API in GitLab instances + +This is the API that external clients can consume on their local +GitLab instance. For example VSCode that talks to a self-managed +instance. + +These versions could also widely defer: it could be that the VSCode +extension is kept up-to-date by developers. But the GitLab instance +they use for work is kept a minor version behind. So the same +requirements in terms of stability and flexibility apply for the +clients as for the AI gateway. + +In a first iteration we could consider keeping the current REST +payloads that the VSCode extension and the Web-IDE send, but direct it +to the appropriate GitLab installation. GitLab-rails can wrap the +payload in an envelope for the AI-gateway without having to interpret +it. + +When we do this then the GitLab-instance that receives the request +from the extension doesn't need to understand it to enrich it and pass +it on to the AI-Gateway. GitLab can add information to the +`prompt_components` and pass everything that was already there +straight through to the AI-gateway. + +If a request is initiated from another client (for example VSCode), +GitLab-rails needs to forward the entire payload in addition to any +other enhancements and prompts. This is required so we can potentially +support changes from a newer version of the client, traveling through +an outdated GitLab installation to a recent AI-gateway. + +**Discussion:** This first iteration is also using a REST+JSON +approach. This is how the VSCode extension is currently communicating +with the model gateway. This means that it's a smaller iteration to go +from that, to wrapping that existing payload into an envelope. With +the added advantage of cross version compatibility. But it does not +mean that future iterations also need to use REST+JSON. As each +feature would have it's own endpoint, the protocol could also be +different. + +## Authentication & Authorization + +GitLab will provide the first layer of authorization: It authenticate +the user and check if the license allows using the feature the user is +trying to use. This can be done using the authentication and license +checks that are already built into GitLab. + +Authenticating the GitLab-instance on the AI-gateway will be discussed +in[#177](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/177). +Because the AI-gateway exposes proxied endpoints to AI providers, it +is important that the authentication tokens have limited validity. + +## Embeddings + +Embeddings can be requested for all features in a single endpoint, for +example through a request like this: + +```plaintext +POST /internal/embeddings +``` + +```json +{ + "content": "The lazy fox and the jumping dog", + "content_type": "issue_title", + "metadata": { + "source": "GitLab EE", + "version": "16.3" + } +} +``` + +The `content_type` and properties `content` could in the future be +used to create embeddings from different models based on what is +appropriate. + +The response will include the embedding vector besides the used +provider and model. For example: + +```json +{ + "response": [0.2, -1, ...], + "metadata": { + "identifier": "8badf00d", + "model": "text-embedding-ada-002", + "provider": "open_ai", + } +} +``` + +When storing the embedding, we should make sure we include the model +and provider data. When embeddings are used to generate a prompt, we +could include that metadata in the payload so we can judge the quality +of the embedding. + +## Deployment + +Currently, the model-gateway that will become the AI-gateway is being +deployed using from the project repository in +[`gitlab-org/modelops/applied-ml/code-suggestions/ai-assist`](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist). + +It is deployed to a Kubernetes cluster in it's own project. There is a +staging environment that is currently used directly by engineers for +testing. + +In the future, this will be deloyed using +[Runway](https://gitlab.com/gitlab-com/gl-infra/platform/runway/). At +that time, there will be a production and staging deployment. The +staging deployment can be used for automated QA-runs that will have +the potential to stop a deployment from reaching production. + +Further testing strategy is being discussed in +[&10563](https://gitlab.com/groups/gitlab-org/-/epics/10563). + +## Alternative solutions + +Alternative solutions were discussed in +[applied-ml/code-suggestions/ai-assist#161](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/161#what-are-the-alternatives). diff --git a/doc/architecture/blueprints/cells/cells-feature-backups.md b/doc/architecture/blueprints/cells/cells-feature-backups.md index d596bdd2078..b5d5d7afdcf 100644 --- a/doc/architecture/blueprints/cells/cells-feature-backups.md +++ b/doc/architecture/blueprints/cells/cells-feature-backups.md @@ -25,10 +25,10 @@ and also Git repository data. ## 2. Data flow -Each cell has a number of application databases to back up (e.g. `main`, and `ci`). +Each cell has a number of application databases to back up (for example, `main`, and `ci`). -Additionally, there may be cluster-wide metadata tables (e.g. `users` table) -which is directly accesible via PostgreSQL. +Additionally, there may be cluster-wide metadata tables (for example, `users` table) +which is directly accessible via PostgreSQL. ## 3. Proposal diff --git a/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md b/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md index 3e498c24144..8a67383c5e4 100644 --- a/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md +++ b/doc/architecture/blueprints/cells/cells-feature-contributions-forks.md @@ -30,13 +30,13 @@ with various usage patterns: Forks allow users not having write access to parent project to make changes. The forking workflow is especially important for the Open Source community which is able to contribute back to public projects. However, it is equally important in some companies which prefer the strong split -of responsibilites and tighter access control. The access to project is restricted +of responsibilities and tighter access control. The access to project is restricted to designated list of developers. Forks enable: -- tigther control of who can modify the upstream project -- split of the responsibilites: parent project might use CI configuration connecting to production systems +- tighter control of who can modify the upstream project +- split of the responsibilities: parent project might use CI configuration connecting to production systems - run CI pipelines in context of fork in much more restrictive environment - consider all forks to be unveted which reduces risks of leaking secrets, or any other information tied with the project diff --git a/doc/architecture/blueprints/cells/cells-feature-secrets.md b/doc/architecture/blueprints/cells/cells-feature-secrets.md index 20260c89ccd..50ccf926b4d 100644 --- a/doc/architecture/blueprints/cells/cells-feature-secrets.md +++ b/doc/architecture/blueprints/cells/cells-feature-secrets.md @@ -25,10 +25,10 @@ GitLab has a lot of [secrets](https://docs.gitlab.com/charts/installation/secrets.html) that needs to be configured. -Some secrets are for inter-component communication, e.g. `GitLab Shell secret`, +Some secrets are for inter-component communication, for example, `GitLab Shell secret`, and used only within a cell. -Some secrets are used for features, e.g. `ci_jwt_signing_key`. +Some secrets are used for features, for example, `ci_jwt_signing_key`. ## 2. Data flow diff --git a/doc/architecture/blueprints/cells/index.md b/doc/architecture/blueprints/cells/index.md index 6da99e0aa6a..dcd28707890 100644 --- a/doc/architecture/blueprints/cells/index.md +++ b/doc/architecture/blueprints/cells/index.md @@ -190,7 +190,7 @@ information. For example: by one of the Cells, and the results of that can be cached. We also need to implement a mechanism for negative cache and cache eviction. -1. **GraphQL and other ambigious endpoints.** +1. **GraphQL and other ambiguous endpoints.** Most endpoints have a unique sharding key: the organization, which directly or indirectly (via a group or project) can be used to classify endpoints. @@ -281,24 +281,24 @@ changes to prepare the codebase for data split. One iteration describes one quarter's worth of work. -1. Iteration 1 - FY24Q1 +1. [Iteration 1](https://gitlab.com/groups/gitlab-org/-/epics/9667) - FY24Q1 - Data access layer: Initial Admin Area settings are shared across cluster. - Essential workflows: Allow to share cluster-wide data with database-level data access layer -1. Iteration 2 - FY24Q2 +1. [Iteration 2](https://gitlab.com/groups/gitlab-org/-/epics/9813) - FY24Q2 - Essential workflows: User accounts are shared across cluster. - Essential workflows: User can create group. -1. Iteration 3 - FY24Q3 +1. [Iteration 3](https://gitlab.com/groups/gitlab-org/-/epics/10997) - FY24Q3 - Essential workflows: User can create project. - Essential workflows: User can push to Git repository. - Cell deployment: Extend GitLab Dedicated to support GCP - Routing: Technology. -1. Iteration 4 - FY24Q4 +1. [Iteration 4](https://gitlab.com/groups/gitlab-org/-/epics/10998) - FY24Q4 - Essential workflows: User can run CI pipeline. - Essential workflows: User can create issue, merge request, and merge it after it is green. diff --git a/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md b/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md index f352fea84b1..c1ca0c60dcd 100644 --- a/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md +++ b/doc/architecture/blueprints/cells/proposal-stateless-router-with-buffering-requests.md @@ -429,7 +429,7 @@ sequenceDiagram ``` In this case the user is not on their "default organization" so their TODO -counter will not include their normal todos. We may choose to highlight this in +counter will not include their typical todos. We may choose to highlight this in the UI somewhere. A future iteration may be able to fetch that for them from their default organization. diff --git a/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md b/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md index aadc08016e3..3b3d481914f 100644 --- a/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md +++ b/doc/architecture/blueprints/cells/proposal-stateless-router-with-routes-learning.md @@ -452,7 +452,7 @@ sequenceDiagram ``` In this case the user is not on their "default organization" so their TODO -counter will not include their normal todos. We may choose to highlight this in +counter will not include their typical todos. We may choose to highlight this in the UI somewhere. A future iteration may be able to fetch that for them from their default organization. diff --git a/doc/architecture/blueprints/ci_data_decay/index.md b/doc/architecture/blueprints/ci_data_decay/index.md index 2eac27def18..25ce71a3a6f 100644 --- a/doc/architecture/blueprints/ci_data_decay/index.md +++ b/doc/architecture/blueprints/ci_data_decay/index.md @@ -55,7 +55,7 @@ of the primary database, to a different storage, that is more performant and cost effective. It is already possible to prevent processing builds -[that have been archived](../../../user/admin_area/settings/continuous_integration.md#archive-jobs). +[that have been archived](../../../administration/settings/continuous_integration.md#archive-jobs). When a build gets archived it will not be possible to retry it, but we still do keep all the processing metadata in the database, and it consumes resources that are scarce in the primary database. diff --git a/doc/architecture/blueprints/ci_pipeline_components/index.md b/doc/architecture/blueprints/ci_pipeline_components/index.md index 667c8d225db..9a8084f290b 100644 --- a/doc/architecture/blueprints/ci_pipeline_components/index.md +++ b/doc/architecture/blueprints/ci_pipeline_components/index.md @@ -12,6 +12,9 @@ participating-stages: [] # CI/CD Catalog +NOTE: +This document covers the future plans for the CI/CD Catalog feature. For information on the features already available for use in GitLab, see the [CI/CD Components documentation](../../../ci/components/index.md). + ## Summary ## Goals @@ -227,8 +230,7 @@ The version of the component can be (in order of highest priority first): 1. A commit SHA - For example: `gitlab.com/gitlab-org/dast@e3262fdd0914fa823210cdb79a8c421e2cef79d8` 1. A tag - For example: `gitlab.com/gitlab-org/dast@1.0` -1. A special moving target version that points to the most recent released tag. The target project must be -explicitly marked as a [catalog resource](#catalog-resource) - For example: `gitlab.com/gitlab-org/dast@~latest` +1. A special moving target version that points to the most recent released tag - For example: `gitlab.com/gitlab-org/dast@~latest` 1. A branch name - For example: `gitlab.com/gitlab-org/dast@master` If a tag and branch exist with the same name, the tag takes precedence over the branch. diff --git a/doc/architecture/blueprints/ci_scale/index.md b/doc/architecture/blueprints/ci_scale/index.md index 3a6ed4ae9b1..340e6557a72 100644 --- a/doc/architecture/blueprints/ci_scale/index.md +++ b/doc/architecture/blueprints/ci_scale/index.md @@ -137,7 +137,7 @@ stores more than 600 gigabytes of data, and `ci_builds.yaml_variables` more than 300 gigabytes (as of February 2021). It is a lot of data that needs to be reliably moved to a different place. -Unfortunately, right now, our [background migrations](../../../development/database/background_migrations.md) +Unfortunately, right now, our background migrations are not reliable enough to migrate this amount of data at scale. We need to build mechanisms that will give us confidence in moving this data between columns, tables, partitions or database shards. diff --git a/doc/architecture/blueprints/clickhouse_ingestion_pipeline/index.md b/doc/architecture/blueprints/clickhouse_ingestion_pipeline/index.md index 6645f390fd1..66089085d0d 100644 --- a/doc/architecture/blueprints/clickhouse_ingestion_pipeline/index.md +++ b/doc/architecture/blueprints/clickhouse_ingestion_pipeline/index.md @@ -129,7 +129,7 @@ We're also not addressing any client-side specific details into the design at th ## General Considerations -Having addressed the details of the two aformentioned problem-domains, we can model a proposed solution with the following logical structure: +Having addressed the details of the two aforementioned problem-domains, we can model a proposed solution with the following logical structure: - Ingestion - APIs/SDKs @@ -208,7 +208,7 @@ Gitlab::Database::Writer.config do |config| # then backend-specific configurations hereafter # config.url = 'tcp://user:pwd@localhost:9000/database' - # e.g. a serializer helps define how data travels over the wire + # for example, a serializer helps define how data travels over the wire config.json_serializer = ClickHouse::Serializer::JsonSerializer # ... end diff --git a/doc/architecture/blueprints/clickhouse_usage/index.md b/doc/architecture/blueprints/clickhouse_usage/index.md index 8a5530313e5..3febb09f0bf 100644 --- a/doc/architecture/blueprints/clickhouse_usage/index.md +++ b/doc/architecture/blueprints/clickhouse_usage/index.md @@ -20,6 +20,12 @@ participating-stages: ["~section::ops", "~section::dev"] In FY23-Q2, the Monitor:Observability team developed and shipped a [ClickHouse data platform](https://gitlab.com/groups/gitlab-org/-/epics/7772) to store and query data for Error Tracking and other observability features. Other teams have also begun to incorporate ClickHouse into their current or planned architectures. Given the growing interest in ClickHouse across product development teams, it is important to have a cohesive strategy for developing features using ClickHouse. This will allow teams to more efficiently leverage ClickHouse and ensure that we can maintain and support this functionality effectively for SaaS and self-managed customers. +### Use Cases + +Many product teams at GitLab are considering ClickHouse when developing new features and to improve performance of existing features. + +During the start of the ClickHouse working group, we [documented existing and potential use cases](https://gitlab.com/groups/gitlab-com/-/epics/2075#use-cases) and found that there was interest in ClickHouse from teams across all DevSecOps stage groups. + ### Goals As ClickHouse has already been selected for use at GitLab, our main goal now is to ensure successful adoption of ClickHouse across GitLab. It is helpful to break down this goal according to the different phases of the product development workflow. @@ -29,29 +35,36 @@ As ClickHouse has already been selected for use at GitLab, our main goal now is 1. Launch: Support ClickHouse-backed features for SaaS and self-managed. 1. Improve: Successfully scale our usage of ClickHouse. -### Non-Goals +### Non-goals + +ClickHouse will not be packaged by default with self-managed GitLab, due to uncertain need, complexity, and lack of operational experience. We will still work to find the best possible way to enable users to use ClickHouse themselves if they desire, but it will not be on by default. [ClickHouse maintenance and cost](self_managed_costs_and_requirements/index.md) investigations revealed an uncertain cost impact to smaller instances, and at this time unknown nuance to managing ClickHouse. This means features that depend only on ClickHouse will not be available out of the box for self-managed users (as of end of 2022, the majority of revenue comes from self-managed), so new features researching the use of ClickHouse should be aware of the potential impacts to user adoption in the near-term, until a solution is viable. ## Proposals The following are links to proposals in the form of blueprints that address technical challenges to using ClickHouse across a wide variety of features. -1. Scalable data ingestion pipeline. +1. [Scalable data ingestion pipeline](../clickhouse_ingestion_pipeline/index.md). - How do we ingest large volumes of data from GitLab into ClickHouse either directly or by replicating existing data? -1. Supporting ClickHouse for self-managed installations. - - For which use-cases and scales does it make sense to run ClickHouse for self-managed and what are the associated costs? - - How can we best support self-managed installation of ClickHouse for different types/sizes of environments? - - Consider using the [Opstrace ClickHouse operator](https://gitlab.com/gitlab-org/opstrace/opstrace/-/tree/main/clickhouse-operator) as the basis for a canonical distribution. - - Consider exposing Clickhouse backend as [GitLab Plus](https://gitlab.com/groups/gitlab-org/-/epics/308) to combine benefits of using self-managed instance and GitLab-managed database. - - Should we develop abstractions for querying and data ingestion to avoid requiring ClickHouse for small-scale installations? -1. Abstraction layer for features to leverage both ClickHouse or PostreSQL. +1. [Abstraction layer](../clickhouse_read_abstraction_layer/index.md) for features to leverage both ClickHouse and PostgreSQL. - What are the benefits and tradeoffs? For example, how would this impact our automated migration and query testing? -1. Security recommendations and secure defaults for ClickHouse usage. -Note that we are still formulating proposals and will update the blueprint accordingly. +### Product roadmap + +#### Near-term + +In the next 3 months (FY24 Q2) ClickHouse will be implemented by default only for SaaS on GitLab.com or manual enablement for self-managed instances. This is due to the uncertain costs and management requirements for self-managed instances. This near-term implementation will be used to develop best practices and strategy to direct self-managed users. This will also constantly shape our recommendations for self-managed instances that want to onboard ClickHouse early. + +#### Mid-term + +After we have formulated best practices of managing ClickHouse ourselves for GitLab.com, the plan for 3-9 months (FY24 2H) will be to offer supported recommendations for self-managed instances that want to run ClickHouse themselves or potentially to a ClickHouse cluster/VM we would manage for users. One proposal for self-managed users is to [create a proxy or abstraction layer](https://gitlab.com/groups/gitlab-org/-/epics/308) that would allow users to connect their self-managed instance to SaaS without additional effort. Another option would be to allow users to "Bring your own ClickHouse" similar to our [approach for Elasticsearch](../../../integration/advanced_search/elasticsearch.md#install-elasticsearch). For the features that require ClickHouse for optimal usage (Value Streams Dashboard, [Product Analytics](https://gitlab.com/groups/gitlab-org/-/epics/8921) and Observability), this will be the initial go-to-market action. + +#### Long-term + +We will work towards a packaged reference version of ClickHouse capable of being easily managed with minimal cost increases for self-managed users. We should be able to reliably instruct users on the management of ClickHouse and provide accurate costs for usage. This will mean any feature could depend on ClickHouse without decreasing end-user exposure. ## Best Practices -Best practices and guidelines for developing performant and scalable features using ClickHouse are located in the [ClickHouse developer documentation](../../../development/database/clickhouse/index.md). +Best practices and guidelines for developing performant, secure, and scalable features using ClickHouse are located in the [ClickHouse developer documentation](../../../development/database/clickhouse/index.md). ## Cost and maintenance analysis diff --git a/doc/architecture/blueprints/code_search_with_zoekt/index.md b/doc/architecture/blueprints/code_search_with_zoekt/index.md index 681782609ba..273d8da482c 100644 --- a/doc/architecture/blueprints/code_search_with_zoekt/index.md +++ b/doc/architecture/blueprints/code_search_with_zoekt/index.md @@ -33,7 +33,7 @@ GitLab code search functionality today is backed by Elasticsearch. Elasticsearch has proven useful for other types of search (issues, merge requests, comments and so-on) but is by design not a good choice for code search where users expect matches to be precise (ie. no false positives) and -flexible (e.g. support +flexible (for example, support [substring matching](https://gitlab.com/gitlab-org/gitlab/-/issues/325234) and [regexes](https://gitlab.com/gitlab-org/gitlab/-/issues/4175)). We have diff --git a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md index f5bd53627cb..2e0b4d40e13 100644 --- a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md +++ b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md @@ -198,7 +198,7 @@ In this phase we are migrating basic, high-priority project functionality from ` - [Unify members/members actions](https://gitlab.com/groups/gitlab-org/-/epics/8010) - on UI and API level. - Starring: Right now only projects can be starred. We want to bring this to the group level. - Common actions: Destroying, transferring, restoring. This can be unified on the controller level and then propagated lower. -- Archiving currently only works on the project level. This can be brought to the group level, similar to the mechanism for “pending deletion”. +- Archiving currently only works on the project level. This can be brought to the group level, similar to the mechanism for "pending deletion". - Avatar's serving and actions. ### Phase 4 diff --git a/doc/architecture/blueprints/container_registry_metadata_database/index.md b/doc/architecture/blueprints/container_registry_metadata_database/index.md index b77aaf598e6..a538910f553 100644 --- a/doc/architecture/blueprints/container_registry_metadata_database/index.md +++ b/doc/architecture/blueprints/container_registry_metadata_database/index.md @@ -266,7 +266,7 @@ The expected registry behavior will be covered with integration tests by manipul ##### Latency -Excessive latency on established connections is hard to detect and debug, as these might not raise an application error or network timeout in normal circumstances but usually precede them. +Excessive latency on established connections is hard to detect and debug, as these might not raise an application error or network timeout in typical circumstances but usually precede them. For this reason, the duration of database queries used to serve HTTP API requests should be instrumented using metrics, allowing the detection of unusual variations and trigger alarms accordingly before an excessive latency becomes a timeout or service unavailability. diff --git a/doc/architecture/blueprints/container_registry_metadata_database_self_managed_rollout/index.md b/doc/architecture/blueprints/container_registry_metadata_database_self_managed_rollout/index.md new file mode 100644 index 00000000000..a73f6335218 --- /dev/null +++ b/doc/architecture/blueprints/container_registry_metadata_database_self_managed_rollout/index.md @@ -0,0 +1,238 @@ +--- +status: proposed +creation-date: "2023-06-09" +authors: [ "@hswimelar" ] +coach: "@grzesiek" +approvers: [ "@trizzi ", "@sgoldstein" ] +owning-stage: "~devops::package" +participating-stages: [] +--- + +<!-- Blueprints often contain forward-looking statements --> +<!-- vale gitlab.FutureTense = NO --> + +# Container Registry Self-Managed Database Rollout + +## Summary + +The latest iteration of the [Container Registry](https://gitlab.com/gitlab-org/container-registry) +has been rearchitected to use a PostgreSQL database and deployed on GitLab.com. +Now we must bring the advantages provided by the database to self-managed users. +While the container registry retains the capacity to run without the new database, +many new and highly desired features cannot be implemented without it. +Additionally, unifying the registry used for GitLab.com and for self-managed +allows us to provide a cohesive user experience and reduces the burden +associated with maintaining the old registry implementation. To accomplish this, +we plan to eventually require all self-managed to migrate to the new registry +database, so that we may deprecate and remove support for the old object storage +metadata subsystem. + +This document seeks to describe how we may use the proven core migration +functionality, which was used to migrate millions of container images on GitLab.com, +to enable self-managed users to enjoy the benefits of the metadata database. + +## Motivation + +Enabling self-managed users to migrate to the new metadata database allows these +users to take advantage of the new features that require the database. Additionally, +the greater adoption of the database allows the container registry team to focus +our knowledge and capacity, and will eventually allow us to fully remove the old +registry metadata subsystem, greatly improving the maintainability and stability +of the container registry for both GitLab.com and for self-managed users. + +### Goals + +- Progressively rollout the new dependency of a PostgreSQL database instance for the registry for charts and omnibus deployments. +- Progressively rollout automation for the registry PostgreSQL database instance for charts and omnibus deployments. +- Develop processes and tools that self-managed admins can use to migrate existing registry deployments to the metadata database. +- Develop processes and tools that self-managed admins can use spin up fresh installs of the Container Registry which use the metadata database. +- Create a plan which will eventually allow us to fully drop support for original object storage metadata subsystem. + +### Non-Goals + +- Developing new Container Registry features outside the scope of enabling admins to migrate to the metadata database. +- Determining lifecycle support decisions, such as when to default to the database, and when to end support for non-database registries. + +## Proposal + +There are two main components that must be further developed in order for +self-managed admins to move to the registry database: the deployment environment and +the registry migration tooling. + +For the deployment environments need to document what the user needs to do to set up their +deployment such that the registry has access to a suitable database given the +expected registry workload. As well as develop tooling and automation to ease +the setup and maintenance of the registry database for new and existing deploys. + +For the registry, we need to develop and validate import tooling which +coordinates with the core import functionality which was used to migrate all +container images on GitLab.com. Additionally, we must validate that each supported +storage driver works as expected with the import process and provide estimated +import times for admins. + +We can structure our work to meet the standards outlined in support for +Experiment, Beta, and Alpha features. Doing so will help to prioritize core +functionality and to allow users who wish to be early adopters to begin using +the database and providing us with invaluable feedback. + +These levels of support could be advertised to self-managed users via a simple +chart, allowing them to tell at a glance the status of this project as it relates +to their situation. + +| Installation | GCS | AWS | Filesystem | Azure | OSS | Swift| +| ------ | ------ |------ | ------ | ------ |------ | ------ | +| Omnibus | GA | GA | Beta | Experimental | Experimental | Experimental | +| Charts | GA | GA |Beta | Experimental | Experimental | Experimental | + +### Justification of Structuring Support by Driver + +It's possible that we could simplify the proposed support matrix by structuring +it only by deployment environment and not differentiate by storage driver. The +following two sections briefly summarize several points for and against. + +#### Arguments Opposed to Structuring Support by Driver + +Each storage driver is well abstracted in the code, specifically the import process +makes use of the following Methods: + +- Walk +- List +- GetContent +- Stat +- Reader + +Each of the methods is a read method we do not need to create or delete data via +the object storage methods. Additionally, all of these methods are standard API +methods. + +Given that we're not mutating data via object storage as part of the import +process, we should not need to double-check these drivers or try to predict +potential errors. Relying on user feedback during the beta to direct any efforts +we should be making here could prevent us from scheduling unnecessary work. + +#### Arguments in Favor of Structuring Support by Driver + +Our experience with enhancing and supporting offline garbage collection has +shown that while the storage driver implementation should not matter, it does. +The drivers have proven to have important differences in performance and +reliability. Many of the planned possible driver-related improvements are +related to testing and stability, rather than outright new work for each driver. + +In particular, retries and error reporting across storage drivers are not as +standardized as one would hope for, and therefore there is a potential that a +long-running import process could be interrupted by an error that could have +been retried. + +Creating import estimates based on combinations of the registry size and storage +driver, would also be of use to self-managed admins, looking to schedule their +migration. There will be a difference here between local filesystem storage and +object storage and there could be a difference between the object storage +providers as well. + +Also, we could work with the importer to smooth out the differences in the +storage drivers. Even without unified retryable error reporting from the storage +drivers, we could have the importer retry more time and for more errors. There's +a risk we would retry several times on non-retryable errors, but since no writes +are being made to object storage, this should not ultimately be harmful. + +Additionally, implementing [Validate Self-Managed Imports](https://gitlab.com/gitlab-org/container-registry/-/issues/938) +would perform a consistency check against a sample of images before and after +import which would lead to greater consistency across all storage driver implementations. + +## Design and Implementation Details + +### The Import Tool + +The [import tool](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs-gitlab/database-import-tool.md) +is a well-validated component of the Container Registry project that we have used +from the beginning as a way to perform local testing. This tool is a thin wrapper +over the core import functionality — the code which handles the import logic has +been extensively validated. + +While the core import functionality is solid, we must ensure that this tool and +the surrounding process will enable non-expert users to import their registries +with both minimal risk and with minimal support from GitLab team members. +Therefore, the most important work remaining is crafting the UX of this tooling +such that those goals are met. This +[epic](https://gitlab.com/groups/gitlab-org/-/epics/8602) captures many of the +proposed improvements. + +#### Design + +The tool is designed such that a single execution flow can support both users +with large registries with strict uptime requirements who can take advantage of +a more involved process to reduce read-only time to the absolute minimum as well +as users with small registries who benefit from a streamlined workflow. This is +achieved via the same pre import, then full import cycle that was used on +GitLab.com, along with an additional step to catalog all unreferenced blobs held +in common storage. + +##### One-Shot Import + +In most cases, a user can simply choose to run the import tool while the registry +is offline or read-only in mode. This will be similar to what admins must +already do in order to run offline garbage collection. Each step completes in +sequence, moving directly to the next. The command exits when the import process +is complete and the registry is ready to make full use of the metadata database. + +##### Minimal Downtime Import + +For users with large registries and who are interested in the minimum possible +downtime, each step can be ran independently when the tool is passed the appropriate +flag. The user will first run the pre-import step while the registry is +performing its usual workload. Once that has completed, and the user is ready +to stop writes to the registry, the tag import step can be ran. As with the GitLab.com +migration, importing tags requires that the registry be offline or in +read-only mode. This step does the minimum possible work to achieve fast and +efficient tag imports and will always be the fastest of the three steps, reducing +the downtime component to a fraction of the total import time. The user can then +bring up the registry configured to use the metadata database. After that, the +user is free to run the third step during standard registry operations. This step +makes any dangling blobs in common storage visible to the database and therefore +the online garbage collection process. + +### Distribution Paths + +Tooling, process, and documentation will need to be developed in order to +support users who wish to use the metadata database, especially in regards to +providing a foundation for the new database instance required for the migration. + +For new deployments, we should wait until we've moved to general support, have +automation in place for the registry database and migration, and have a major +GitLab version bump before enabling the database by default for self-managed. + +#### Omnibus + +#### Charts + +## Alternative Solutions + +### Do Nothing + +#### Pros + +- The database and associated features are generally most useful for large-scale, high-availability deployments. +- Eliminate the need to support an additional logical or physical database for self-managed deployments. + +#### Cons + +- The registry on GitLab.com and the registry used by self-managed will greatly diverge in supported features over time. +- The maintenance burden of supporting two registry implementations will reduce the velocity at which new registry features can be released. +- The registry on GitLab.com stops being an effective way to validate changes before they are released to self-managed. +- Large self-managed users continue to not be able to scale the registry to suit their needs. + +### Gradual Migration + +This approach would be to exactly replicate the GitLab.com migration on +self-managed. + +#### Pros + +- Replicate an already successful process. +- Scope downtime by repository, rather than instance. + +#### Cons + +- Dramatically increased complexity in all aspects of the migration process. +- Greatly increased possibility of data consistency issues. +- Less clear demarcation of registry migration progress. diff --git a/doc/architecture/blueprints/database/automated_query_analysis/index.md b/doc/architecture/blueprints/database/automated_query_analysis/index.md index c08784dab48..40f6b2af412 100644 --- a/doc/architecture/blueprints/database/automated_query_analysis/index.md +++ b/doc/architecture/blueprints/database/automated_query_analysis/index.md @@ -12,7 +12,7 @@ participating-stages: [] ## Problem Summary -Our overarching goal is to improve the reliability and throughput of GitLab’s +Our overarching goal is to improve the reliability and throughput of the GitLab database review process. The current process requires merge request authors to manually provide query plans and raw SQL when introducing new queries or updating existing queries. This is both time consuming and error prone. diff --git a/doc/architecture/blueprints/gitlab_agent_deployments/index.md b/doc/architecture/blueprints/gitlab_agent_deployments/index.md index d8d26389d7d..798c8a3045d 100644 --- a/doc/architecture/blueprints/gitlab_agent_deployments/index.md +++ b/doc/architecture/blueprints/gitlab_agent_deployments/index.md @@ -1,11 +1,11 @@ --- -status: proposed +status: implemented creation-date: "2022-11-23" authors: [ "@shinya.maeda" ] coach: "@DylanGriffith" approvers: [ "@nagyv-gitlab", "@cbalane", "@hustewart", "@hfyngvason" ] -owning-stage: "~devops::release" -participating-stages: [Configure, Release] +owning-stage: "~devops::deploy" +participating-stages: [Environments] --- <!-- vale gitlab.FutureTense = NO --> @@ -28,9 +28,7 @@ This blueprint describes how the association is established and how these domain - The proposed architecture can be used in [Organization-level Environment dashboard](https://gitlab.com/gitlab-org/gitlab/-/issues/241506). - The cluster resources and events can be visualized per [GitLab Environment](../../../ci/environments/index.md). An environment-specific view scoped to the resources managed either directly or indirectly by a deployment commit. -- Support both [GitOps mode](../../../user/clusters/agent/gitops.md#gitops-configuration-reference) and [CI Access mode](../../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent). - - NOTE: At the moment, we focus on the solution for CI Access mode. GitOps mode will have significant architectural changes _outside of_ this blueprint, - such as [Flux switching](https://gitlab.com/gitlab-org/gitlab/-/issues/357947) and [Manifest projects outside of the Agent configuration project](https://gitlab.com/groups/gitlab-org/-/epics/7704). In order to derisk potential rework, we'll revisit the GitOps mode after these upstream changes have been settled. +- Support both [GitOps mode](../../../user/clusters/agent/gitops/agent.md#gitops-configuration-reference) and [CI Access mode](../../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent). ### Non-Goals @@ -41,22 +39,22 @@ This blueprint describes how the association is established and how these domain ### Overview -- GitLab Environment and Agent-managed Resource Group have 1-to-1 relationship. -- Agent-managed Resource Group tracks all resources produced by the connected [agent](../../../user/clusters/agent/index.md). This includes not only resources written in manifest files but also subsequently generated resources (e.g. `Pod`s created by `Deployment` manifest file). -- Agent-managed Resource Group renders dependency graph, such as `Deployment` => `ReplicaSet` => `Pod`. This is for providing ArgoCD-style resource view. -- Agent-managed Resource Group has the Resource Health status that represents a summary of resource statuses, such as `Healthy`, `Progressing` or `Degraded`. +- GitLab Environment and GitLab Agent For Kubernetes have 1-to-1 relationship. +- GitLab Environment tracks all resources produced by the connected [agent](../../../user/clusters/agent/index.md). This includes not only resources written in manifest files but also subsequently generated resources (for example, `Pod`s created by `Deployment` manifest file). +- GitLab Environment renders dependency graph, such as `Deployment` => `ReplicaSet` => `Pod`. This is for providing ArgoCD-style resource view. +- GitLab Environment has the Resource Health status that represents a summary of resource statuses, such as `Healthy`, `Progressing` or `Degraded`. ```mermaid flowchart LR subgraph Kubernetes["Kubernetes"] - subgraph ResourceGroupProduction["ResourceGroup"] + subgraph ResourceGroupProduction["Production"] direction LR ResourceGroupProductionService(["Service"]) ResourceGroupProductionDeployment(["Deployment"]) ResourceGroupProductionPod1(["Pod1"]) ResourceGroupProductionPod2(["Pod2"]) end - subgraph ResourceGroupStaging["ResourceGroup"] + subgraph ResourceGroupStaging["Staging"] direction LR ResourceGroupStagingService(["Service"]) ResourceGroupStagingDeployment(["Deployment"]) @@ -88,28 +86,20 @@ flowchart LR - [GitLab Project](../../../user/project/working_with_projects.md) and GitLab Environment have 1-to-many relationship. - GitLab Project and Agent have 1-to-many _direct_ relationship. Only one project can own a specific agent. -- [GitOps mode](../../../user/clusters/agent/gitops.md#gitops-configuration-reference) +- [GitOps mode](../../../user/clusters/agent/gitops/agent.md#gitops-configuration-reference) - GitLab Project and Agent do _NOT_ have many-to-many _indirect_ relationship yet. This will be supported in [Manifest projects outside of the Agent configuration project](https://gitlab.com/groups/gitlab-org/-/epics/7704). - - Agent and Agent-managed Resource Group have 1-to-1 relationship. Inventory IDs are used to group Kubernetes resources. This might be changed in [Flux switching](https://gitlab.com/gitlab-org/gitlab/-/issues/357947). - [CI Access mode](../../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent) - GitLab Project and Agent have many-to-many _indirect_ relationship. The project owning the agent can [share the access with the other proejcts](../../../user/clusters/agent/ci_cd_workflow.md#authorize-the-agent-to-access-projects-in-your-groups). (NOTE: Technically, only running jobs inside the project are allowed to access the cluster due to job-token authentication.) - - Agent and Agent-managed Resource Group do _NOT_ have relationships yet. ### Issues -- Agent-managed Resource Group should have environment ID as the foreign key, which must be unique across resource groups. -- Agent-managed Resource Group should have parameters how to group resources in the associated cluster, for example, `namespace`, `lable` and `inventory-id` (GitOps mode only) can passed as parameters. -- Agent-managed Resource Group should be able to fetch all relevant resources, including both default resource kinds and other [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). -- Agent-managed Resource Group should be aware of dependency graph. -- Agent-managed Resource Group should be able to compute Resource Health status from the associated resources. +- GitLab Environment should have ID of GitLab Agent For Kubernetes as the foreign key. +- GitLab Environment should have parameters how to group resources in the associated cluster, for example, `namespace`, `lable` and `inventory-id` (GitOps mode only) can passed as parameters. +- GitLab Environment should be able to fetch all relevant resources, including both default resource kinds and other [Custom Resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). +- GitLab Environment should be aware of dependency graph. +- GitLab Environment should be able to compute Resource Health status from the associated resources. -### Example: Pull-based deployment (GitOps mode) - -NOTE: -At the moment, we focus on the solution for CI Access mode. GitOps mode will have significant architectural changes _outside of_ this blueprint, -such as [Flux switching](https://gitlab.com/gitlab-org/gitlab/-/issues/357947) and [Manifest projects outside of the Agent configuration project](https://gitlab.com/groups/gitlab-org/-/epics/7704). In order to derisk potential rework, we'll revisit the GitOps mode after these upstream changes have been settled. - -### Example: Push-based deployment (CI access mode) +### Example This is an example of how the architecture works in push-based deployment. The feature is documented [here](../../../user/clusters/agent/ci_cd_workflow.md) as CI access mode. @@ -117,21 +107,21 @@ The feature is documented [here](../../../user/clusters/agent/ci_cd_workflow.md) ```mermaid flowchart LR subgraph ProductionKubernetes["Production Kubernetes"] - subgraph ResourceGroupProductionFrontend["ResourceGroup"] + subgraph ResourceGroupProductionFrontend["Production"] direction LR ResourceGroupProductionFrontendService(["Service"]) ResourceGroupProductionFrontendDeployment(["Deployment"]) ResourceGroupProductionFrontendPod1(["Pod1"]) ResourceGroupProductionFrontendPod2(["Pod2"]) end - subgraph ResourceGroupProductionBackend["ResourceGroup"] + subgraph ResourceGroupProductionBackend["Staging"] direction LR ResourceGroupProductionBackendService(["Service"]) ResourceGroupProductionBackendDeployment(["Deployment"]) ResourceGroupProductionBackendPod1(["Pod1"]) ResourceGroupProductionBackendPod2(["Pod2"]) end - subgraph ResourceGroupProductionPrometheus["ResourceGroup"] + subgraph ResourceGroupProductionPrometheus["Monitoring"] direction LR ResourceGroupProductionPrometheusService(["Service"]) ResourceGroupProductionPrometheusDeployment(["Deployment"]) @@ -202,21 +192,21 @@ The microservice project setup can be improved by [Multi-Project Deployment Pipe ```mermaid flowchart LR subgraph ProductionKubernetes["Production Kubernetes"] - subgraph ResourceGroupProductionFrontend["ResourceGroup"] + subgraph ResourceGroupProductionFrontend["Frontend"] direction LR ResourceGroupProductionFrontendService(["Service"]) ResourceGroupProductionFrontendDeployment(["Deployment"]) ResourceGroupProductionFrontendPod1(["Pod1"]) ResourceGroupProductionFrontendPod2(["Pod2"]) end - subgraph ResourceGroupProductionBackend["ResourceGroup"] + subgraph ResourceGroupProductionBackend["Backend"] direction LR ResourceGroupProductionBackendService(["Service"]) ResourceGroupProductionBackendDeployment(["Deployment"]) ResourceGroupProductionBackendPod1(["Pod1"]) ResourceGroupProductionBackendPod2(["Pod2"]) end - subgraph ResourceGroupProductionPrometheus["ResourceGroup"] + subgraph ResourceGroupProductionPrometheus["Monitoring"] direction LR ResourceGroupProductionPrometheusService(["Service"]) ResourceGroupProductionPrometheusDeployment(["Deployment"]) @@ -266,104 +256,18 @@ flowchart LR DeploymentPipelines -- "Deploy" --> ResourceGroupProductionBackend ``` -#### View all Agent-managed Resource Groups on production environment - -At the group-level, we can accumulate all environments match a specific tier, for example, -listing all environments with `production` tier from subsequent projects. -This is useful to see the entire Agent-managed Resource Groups on production environment. -The following diagram examplifies the relationship between GitLab group and Kubernetes resources: - -```mermaid -flowchart LR - subgraph Kubernetes["Kubernetes"] - subgraph ResourceGroupProduction["ResourceGroup"] - direction LR - ResourceGroupProductionService(["Service"]) - ResourceGroupProductionDeployment(["Deployment"]) - ResourceGroupProductionPod1(["Pod1"]) - ResourceGroupProductionPod2(["Pod2"]) - end - subgraph ResourceGroupStaging["ResourceGroup"] - direction LR - ResourceGroupStagingService(["Service"]) - ResourceGroupStagingDeployment(["Deployment"]) - ResourceGroupStagingPod1(["Pod1"]) - ResourceGroupStagingPod2(["Pod2"]) - end - end - - subgraph GitLab - subgraph Organization - OrganizationProduction["All resources on production"] - subgraph Frontend project - FrontendEnvironmentProduction["production environment"] - end - subgraph Backend project - BackendEnvironmentProduction["production environment"] - end - end - end - - FrontendEnvironmentProduction --- ResourceGroupProduction - BackendEnvironmentProduction --- ResourceGroupStaging - ResourceGroupProductionService -.- ResourceGroupProductionDeployment - ResourceGroupProductionDeployment -.- ResourceGroupProductionPod1 - ResourceGroupProductionDeployment -.- ResourceGroupProductionPod2 - ResourceGroupStagingService -.- ResourceGroupStagingDeployment - ResourceGroupStagingDeployment -.- ResourceGroupStagingPod1 - ResourceGroupStagingDeployment -.- ResourceGroupStagingPod2 - OrganizationProduction --- FrontendEnvironmentProduction - OrganizationProduction --- BackendEnvironmentProduction -``` - -A few notes: - -- In the future, we'd have more granular filters for resource search. - For example, there are two environments `production/us-region` and `production/eu-region` in each project - and show only resources in US region at the group-level. - This could be achivable by query filtering in PostgreSQL or label/namespace filtering in Kubernetes. -- Please see [Add dynamically populated organization-level environments page](https://gitlab.com/gitlab-org/gitlab/-/issues/241506) for more information. - ## Design and implementation details -NOTE: -The following solution might be only applicable for CI Access mode. GitOps mode will have significant architectural changes _outside of_ this blueprint, -such as [Flux switching](https://gitlab.com/gitlab-org/gitlab/-/issues/357947) and [Manifest projects outside of the Agent configuration project](https://gitlab.com/groups/gitlab-org/-/epics/7704). In order to derisk potential rework, we'll revisit the GitOps mode after these upstream changes have been settled. - ### Associate Environment with Agent -As a preliminary step, we allow users to explicitly define "which deployment job" uses "which agent" and deploy to "which namespace". The following keywords are supported in `.gitlab-ci.yml`. - -- `environment:kubernetes:agent` ... Define which agent the deployment job uses. It can select the appropriate context from the `KUBE_CONFIG`. -- `environment:kubernetes:namespace` ... Define which namespace the deployment job deploys to. It injects `KUBE_NAMESPACE` predefined variable into the job. This keyword already [exists](../../../ci/yaml/index.md#environmentkubernetes). - -Here is an example of `.gitlab-ci.yml`. - -```yaml -deploy-production: - environment: - name: production - kubernetes: - agent: path/to/agent/repository:agent-name - namespace: default - script: - - helm --context="$KUBE_CONTEXT" --namespace="$KUBE_NAMESPACE" upgrade --install -``` - -When a deployment job is created, GitLab persists the relationship of specified agent, namespace and deployment job. If the CI job is NOT authorized to access the agent (Please refer [`Clusters::Agents::FilterAuthorizationsService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/clusters/agents/filter_authorizations_service.rb) for more details), this relationship aren't recorded. This process happens in [`Deployments::CreateForBuildService`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/deployments/create_for_build_service.rb). The database table scheme is: - -```plaintext -agent_deployments: - - deployment_id (bigint/FK/NOT NULL/Unique) - - agent_id (bigint/FK/NOT NULL) - - kubernetes_namespace (character varying(255)/NOT NULL) -``` +Users can explicitly set a GitLab Agent For Kubernetes to a GitLab Environment in setting UI. +Frontend will use this associated agent for authenticating/authorizing the user access, which is described in a latter section. -To idenfity an associated agent for a specific environment, `environment.last_deployment.agent` can be used in Rails. +We need to adjust the `read_cluster_agent` permission in DeclarivePolicy for supporting agents shared by an external project (also known as the Agent management project). ### Fetch resources through `user_access` -When user visits an environment page, GitLab frontend fetches an environment via GraphQL. Frontend additionally fetches the associated agent-ID and namespace through deployment relationship, which being tracked by the `agent_deployments` table. +When user visits an environment page, GitLab frontend fetches an environment via GraphQL. Frontend additionally fetches the associated agent-ID and namespace. Here is an example of GraphQL query: @@ -373,12 +277,12 @@ Here is an example of GraphQL query: id environment(name: "<environment-name>") { slug - lastDeployment(status: SUCCESS) { - agent { - id + kubernetesNamespace + clusterAgent { + id + name + project { name - project - kubernetesNamespace } } } @@ -388,20 +292,17 @@ Here is an example of GraphQL query: GitLab frontend authenticate/authorize the user access with [browser cookie](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kubernetes_user_access.md#browser-cookie-on-gitlab-frontend). If the access is forbidden, frontend shows an error message that `You don't have access to an agent that deployed to this environment. Please contact agent administrator if you are allowed in "user_access" in agent config file. See <troubleshooting-doc-link>`. -After the user gained access to the agent, GitLab frontend fetches available API Resource list in the Kubernetes and fetches the resources with the following parameters: +After the user gained access to the agent, GitLab frontend fetches specific Resource kinds (for example, `Deployment`, `Pod`) in the Kubernetes with the following parameters: -- `namespace` ... `#{environment.lastDeployment.agent.kubernetesNamespace}` -- `labels` - - `app.gitlab.com/project_id=#{project.id}` _AND_ - - `app.gitlab.com/environment_slug: #{environment.slug}` +- `namespace` ... `#{environment.kubernetesNamespace}` If no resources are found, this is likely that the users have not embedded these lables into their resources. In this case, frontend shows an warning message `There are no resources found for the environment. Do resources have GitLab preserved labels? See <troubleshooting-doc-link>`. ### Dependency graph - GitLab frontend uses [Owner References](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/) to idenfity the dependencies between resources. These are embedded in resources as `metadata.ownerReferences` field. -- For the resoruces that don't have owner references, we can use [Well-Known Labels, Annotations and Taints](https://kubernetes.io/docs/reference/labels-annotations-taints/) as complement. e.g. `EndpointSlice` doesn't have `metadata.ownerReferences`, but has `kubernetes.io/service-name` as a reference to the parent `Service` resource. +- For the resoruces that don't have owner references, we can use [Well-Known Labels, Annotations and Taints](https://kubernetes.io/docs/reference/labels-annotations-taints/) as complement. for example, `EndpointSlice` doesn't have `metadata.ownerReferences`, but has `kubernetes.io/service-name` as a reference to the parent `Service` resource. ### Health status of resources -- GitLab frontend computes the status summary from the fetched resources. Something similar to ArgoCD's [Resource Health](https://argo-cd.readthedocs.io/en/stable/operator-manual/health/) e.g. `Healthy`, `Progressing`, `Degraded` and `Suspended`. The formula is TBD. +- GitLab frontend computes the status summary from the fetched resources. Something similar to ArgoCD's [Resource Health](https://argo-cd.readthedocs.io/en/stable/operator-manual/health/) for example, `Healthy`, `Progressing`, `Degraded` and `Suspended`. The formula is TBD. diff --git a/doc/architecture/blueprints/gitlab_ci_events/index.md b/doc/architecture/blueprints/gitlab_ci_events/index.md index 7ce8fea9410..fb78c0f5d9d 100644 --- a/doc/architecture/blueprints/gitlab_ci_events/index.md +++ b/doc/architecture/blueprints/gitlab_ci_events/index.md @@ -2,6 +2,7 @@ status: proposed creation-date: "2023-03-15" authors: [ "@furkanayhan" ] +owners: [ "@furkanayhan" ] coach: "@grzesiek" approvers: [ "@jreporter", "@cheryl.li" ] owning-stage: "~devops::verify" @@ -45,7 +46,7 @@ Events" blueprint is about making it possible to: ## Proposals -For now, we have technical 4 proposals; +For now, we have technical 5 proposals; 1. [Proposal 1: Using the `.gitlab-ci.yml` file](proposal-1-using-the-gitlab-ci-file.md) Based on; @@ -55,9 +56,7 @@ For now, we have technical 4 proposals; Highly inefficient way. 1. [Proposal 3: Using the `.gitlab/ci/events` folder](proposal-3-using-the-gitlab-ci-events-folder.md) Involves file reading for every event. -1. [Proposal 4: Creating events via CI files](proposal-4-creating-events-via-ci-files.md) - Combination of some proposals. - -Each of them has its pros and cons. There could be many more proposals and we -would like to discuss them all. We can combine the best part of those proposals -and create a new one. +1. [Proposal 4: Creating events via a CI config file](proposal-4-creating-events-via-ci-files.md) + Separate configuration files for defininig events. +1. [Proposal 5: Combined proposal](proposal-5-combined-proposal.md) + Combination of all of the proposals listed above. diff --git a/doc/architecture/blueprints/gitlab_ci_events/proposal-1-using-the-gitlab-ci-file.md b/doc/architecture/blueprints/gitlab_ci_events/proposal-1-using-the-gitlab-ci-file.md index 7dfc3873ada..f4cde963224 100644 --- a/doc/architecture/blueprints/gitlab_ci_events/proposal-1-using-the-gitlab-ci-file.md +++ b/doc/architecture/blueprints/gitlab_ci_events/proposal-1-using-the-gitlab-ci-file.md @@ -12,7 +12,7 @@ Currently, we have two proof-of-concept (POC) implementations: They both have similar ideas; -1. Find a new CI Config syntax to define the pipeline events. +1. Find a new CI Config syntax to define pipeline events. Example 1: @@ -42,19 +42,13 @@ They both have similar ideas; script: echo "Hello World" ``` -1. Upsert an event to the database when creating a pipeline. -1. Create [EventStore subscriptions](../../../development/event_store.md) to handle the events. +1. Upsert a workflow definition to the database when new configuration gets + pushed. +1. Match subscriptions and publishers whenever something happens at GitLab. -## Problems & Questions +## Discussion -1. The CI config of a project can be anything; - - `.gitlab-ci.yml` by default - - another file in the project - - another file in another project - - completely a remote/external file - - How do we handle these cases? -1. Since we have these problems above, should we keep the events in its own file? (`.gitlab-ci-events.yml`) -1. Do we only accept the changes in the main branch? -1. We try to create event subscriptions every time a pipeline is created. -1. Can we move the existing workflows into the new CI events, for example, `merge_request_event`? +1. How to efficiently detect changes to the subscriptions? +1. How do we handle differences between workflows / events / subscriptions on + different branches? +1. Do we need to upsert subscriptions on every push? diff --git a/doc/architecture/blueprints/gitlab_ci_events/proposal-2-using-the-rules-keyword.md b/doc/architecture/blueprints/gitlab_ci_events/proposal-2-using-the-rules-keyword.md index 6f69a0f11f0..1f59a8ccf20 100644 --- a/doc/architecture/blueprints/gitlab_ci_events/proposal-2-using-the-rules-keyword.md +++ b/doc/architecture/blueprints/gitlab_ci_events/proposal-2-using-the-rules-keyword.md @@ -23,16 +23,13 @@ test_package_removed: - events: ["package/removed"] ``` -1. We don't upsert anything to the database. -1. We'll have a single worker which subcribes to events -like `store.subscribe ::Ci::CreatePipelineFromEventWorker, to: ::Issues::CreatedEvent`. -1. The worker just runs `Ci::CreatePipelineService` with the correct parameters, the rest -will be handled by the `rules` system. Of course, we'll need modifications to the `rules` system to support `events`. - -## Problems & Questions - -1. For every defined event run, we need to enqueue a new `Ci::CreatePipelineFromEventWorker` job. -1. The worker will need to run `Ci::CreatePipelineService` for every event run. -This may be costly because we go through every cycle of `Ci::CreatePipelineService`. -1. This would be highly inefficient. -1. Can we move the existing workflows into the new CI events, for example, `merge_request_event`? +1. We don't upsert subscriptions to the database. +1. We'll have a single worker which runs when something happens in GitLab. +1. The worker just tries to create a pipeline with the correct parameters. +1. Pipeline runs when `rules` subsystem finds a job to run. + +## Challenges + +1. For every defined event run, we need to enqueue a new pipeline creation worker. +1. Creating pipelines and selecting builds to run is a relatively expensive operation +1. This will not work on GitLab.com scale. diff --git a/doc/architecture/blueprints/gitlab_ci_events/proposal-3-using-the-gitlab-ci-events-folder.md b/doc/architecture/blueprints/gitlab_ci_events/proposal-3-using-the-gitlab-ci-events-folder.md index ad76b7f8dd4..8a8efe2be08 100644 --- a/doc/architecture/blueprints/gitlab_ci_events/proposal-3-using-the-gitlab-ci-events-folder.md +++ b/doc/architecture/blueprints/gitlab_ci_events/proposal-3-using-the-gitlab-ci-events-folder.md @@ -5,11 +5,8 @@ description: 'GitLab CI Events Proposal 3: Using the .gitlab/ci/events folder' # GitLab CI Events Proposal 3: Using the `.gitlab/ci/events` folder -We can also approach this problem by creating separate files for events. - -Let's say we'll have the `.gitlab/ci/events` folder (or `.gitlab/workflows/ci`). - -We can define events in the following format: +In this proposal we want to create separate files for each group of events. We +can define events in the following format: ```yaml # .gitlab/ci/events/package-published.yml @@ -17,9 +14,7 @@ We can define events in the following format: spec: events: - name: package/published - --- - include: - local: .gitlab-ci.yml with: @@ -35,9 +30,7 @@ spec: inputs: event: default: push - --- - job1: script: echo "Hello World" @@ -61,4 +54,4 @@ When an event happens; 1. For every defined event run, we need to enqueue a new job. 1. Every event-job will need to search for files. 1. This would be only for the project-scope events. -1. This can be inefficient because of searching for files for the project for every event. +1. This will not work for GitLab.com scale. diff --git a/doc/architecture/blueprints/gitlab_ci_events/proposal-4-creating-events-via-ci-files.md b/doc/architecture/blueprints/gitlab_ci_events/proposal-4-creating-events-via-ci-files.md index 5f10ba1fbb2..debca82d148 100644 --- a/doc/architecture/blueprints/gitlab_ci_events/proposal-4-creating-events-via-ci-files.md +++ b/doc/architecture/blueprints/gitlab_ci_events/proposal-4-creating-events-via-ci-files.md @@ -1,12 +1,13 @@ --- owning-stage: "~devops::verify" -description: 'GitLab CI Events Proposal 4: Creating events via CI files' +description: 'GitLab CI Events Proposal 4: Defining subscriptions in a dedicated configuration file' --- -# GitLab CI Events Proposal 4: Creating events via CI files +# GitLab CI Events Proposal 4: Defining subscriptions in a dedicated configuration file -Each project can have its own event configuration file. Let's call it `.gitlab-ci-event.yml` for now. -In this file, we can define events in the following format: +Each project can have its own configuration file for defining subscriptions to +events. For example, `.gitlab-ci-event.yml`. In this file, we can define events +in the following format: ```yaml events: @@ -14,12 +15,13 @@ events: - issue/created ``` -When this file is changed in the project repository, it is parsed and the events are created, updated, or deleted. -This is highly similar to [Proposal 1](proposal-1-using-the-gitlab-ci-file.md) except that we don't need to -track pipeline creations every time. +When this file is changed in the project repository, it is parsed and the +events are created, updated, or deleted. This is highly similar to +[Proposal 1](proposal-1-using-the-gitlab-ci-file.md) except that we don't need +to track pipeline creations every time. -1. Upsert events to the database when `.gitlab-ci-event.yml` is updated. -1. Create [EventStore subscriptions](../../../development/event_store.md) to handle the events. +1. Upsert events to the database when `.gitlab-ci-event.yml` gets updated. +1. Create inline reactions to events in code to trigger pipelines. ## Filtering jobs @@ -51,7 +53,7 @@ test_package_removed: - if: $CI_EVENT == "package/removed" ``` -or an input like in the [Proposal 3](proposal-3-using-the-gitlab-ci-events-folder.md); +or an input like in the [Proposal 3](proposal-3-using-the-gitlab-ci-events-folder.md): ```yaml spec: @@ -71,3 +73,7 @@ test_package_removed: rules: - if: $[[ inputs.event ]] == "package/removed" ``` + +## Challenges + +1. This will not work on GitLab.com scale. diff --git a/doc/architecture/blueprints/gitlab_ci_events/proposal-5-combined-proposal.md b/doc/architecture/blueprints/gitlab_ci_events/proposal-5-combined-proposal.md new file mode 100644 index 00000000000..3a596b21526 --- /dev/null +++ b/doc/architecture/blueprints/gitlab_ci_events/proposal-5-combined-proposal.md @@ -0,0 +1,99 @@ +--- +owning-stage: "~devops::verify" +description: 'GitLab CI Events Proposal 5: Combined proposal' +--- + +# GitLab CI Events Proposal 5: Combined proposal + +In this proposal we have separate files for cohesive groups of events. The +files are being included into the main `.gitlab-ci.yml` configuration file. + +```yaml +# my/events/packages.yaml + +spec: + events: + - events/package/published + - events/audit/package/* + inputs: + env: +--- +do_something: + script: ./run_for $[[ event.name ]] --env $[[ inputs.env ]] + rules: + - if: $[[ event.payload.package.name ]] == "my_package" +``` + +In the `.gitlab-ci.yml` file, we can enable the subscription: + +```yaml +# .gitlab-ci.yml + +include: + - local: my/events/packages.yaml + inputs: + env: test + +``` + +GitLab will detect changes in the included files, and parse their specs. All +the information required to define a subscription will be encapsulated in the +spec, hence we will not need to read a whole file. We can easily read `spec` +header and calculate its checksum what can become a workflow identifier. + +Once we see a new identifier, we can redefine subscriptions for a particular +project and then to upsert them into the database. + +We will use an efficient GIN index matching technique to match publishers with +the subscribers to run pipelines. + +The syntax is also compatible with CI Components, and make it easier to define +components that will only be designed to run for events happening inside +GitLab. + +## No entrypoint file variant + +Another variant of this proposal is to move away from the single GitLab CI YAML +configuration file. In such case we would define another search **directory**, +like `.gitlab/workflows/` where we would store all YAML files. + +We wouldn't need to `include` workflow / events files anywhere, because these +would be found by GitLab automatically. In order to implement this feature this +way we would need to extend features like "custom location for `.gitlab-ci.yml` +file". + +Example, without using a main configuration file (the GitLab CI YAML file would +be still supported): + +```yaml +# .gitlab/workflows/push.yml + +spec: + events: + - events/repository/push +--- +rspec-on-push: + script: bundle exec rspec +``` + +```yaml +# .gitlab/workflows/merge_requests.yml + +spec: + events: + - events/merge_request/push +--- +rspec-on-mr-push: + script: bundle exec rspec +``` + +```yaml +# .gitlab/workflows/schedules.yml + +spec: + events: + - events/pipeline/schedule/run +--- +smoke-test: + script: bundle exec rspec --smoke +``` diff --git a/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md b/doc/architecture/blueprints/gitlab_observability_backend/index.md index 3edb01d9140..5b99235e18c 100644 --- a/doc/architecture/blueprints/gitlab_observability_backend/metrics/index.md +++ b/doc/architecture/blueprints/gitlab_observability_backend/index.md @@ -37,7 +37,7 @@ With the development of the proposed system, we have the following goals: - Support for long-term storage for Prometheus/OpenTelemetry formatted metrics, ingested via Prometheus remote_write API and queried via Prometheus remote_read API, PromQL or SQL with support for metadata and exemplars. -The aformentioned goals can further be broken down into the following four sub-goals: +The aforementioned goals can further be broken down into the following four sub-goals: #### Ingesting data @@ -52,7 +52,7 @@ The aformentioned goals can further be broken down into the following four sub-g NOTE: Although remote_write_sender does not test the correctness of a remote write receiver itself as is our case, it does bring some inspiration to implement/develop one within the scope of this project. -- We aim to also ensure compatibility for special Prometheus data types, e.g. Prometheus histogram(s), summary(s). +- We aim to also ensure compatibility for special Prometheus data types, for example, Prometheus histogram(s), summary(s). #### Reading data @@ -78,22 +78,22 @@ With the goals established above, we also want to establish what specific things - We do not aim to support ingesting Prometheus exemplars in our first iteration, though we do aim to account for them in our design from the beginning. NOTE: -Worth noting that we intend to model exemplars the same way we’re modeling metric-labels, so building on top of the same data structure should help implementt support for metadata/exemplars rather easily. +Worth noting that we intend to model exemplars the same way we're modeling metric-labels, so building on top of the same data structure should help implementt support for metadata/exemplars rather easily. ## Proposal -We intend to use GitLab Observability Backend as a framework for the Metrics implementation so that its lifecycle is also managed via already existing Kubernetes controllers e.g. scheduler, tenant-operator. +We intend to use GitLab Observability Backend as a framework for the Metrics implementation so that its lifecycle is also managed via already existing Kubernetes controllers for example, scheduler, tenant-operator.  -From a development perspective, what’s been marked as our “Application Server” above needs to be developed as a part of this proposal while the remaining peripheral components either already exist or can be provisioned via existing code in `scheduler`/`tenant-operator`. +From a development perspective, what's been marked as our "Application Server" above needs to be developed as a part of this proposal while the remaining peripheral components either already exist or can be provisioned via existing code in `scheduler`/`tenant-operator`. -**On the write path**, we expect to receive incoming data via `HTTP`/`gRPC` `Ingress` similar to what we do for our existing services, e.g. errortracking, tracing. +**On the write path**, we expect to receive incoming data via `HTTP`/`gRPC` `Ingress` similar to what we do for our existing services, for example, errortracking, tracing. NOTE: Additionally, since we intend to ingest data via Prometheus `remote_write` API, the received data will be Protobuf-encoded, Snappy-compressed. All received data therefore needs to be decompressed & decoded to turn it into a set of `prompb.TimeSeries` objects, which the rest of our components interact with. -We also need to make sure to avoid writing a lot of small writes into Clickhouse, therefore it’d be prudent to batch data before writing it into Clickhouse. +We also need to make sure to avoid writing a lot of small writes into Clickhouse, therefore it'd be prudent to batch data before writing it into Clickhouse. We must also make sure ingestion remains decoupled with `Storage` so as to reduce undue dependence on a given storage implementation. While we do intend to use Clickhouse as our backing storage for any foreseeable future, this ensures we do not tie ourselves in into Clickhouse too much should future business requirements warrant the usage of a different backend/technology. A good way to implement this in Go would be our implementations adhering to a standard interface, the following for example: @@ -177,7 +177,7 @@ Keeping inline with our current operational structure, we intend to deploy the m ```sql CREATE TABLE IF NOT EXISTS samples ON CLUSTER '{cluster}' ( series_id UUID, - timestamp DateTime64(3, ‘UTC’) CODEC(Delta(4), ZSTD), + timestamp DateTime64(3, 'UTC') CODEC(Delta(4), ZSTD), value Float64 CODEC(Gorilla, ZSTD) ) ENGINE = ReplicatedMergeTree() PARTITION BY toYYYYMMDD(timestamp) @@ -189,7 +189,7 @@ ORDER BY (series_id, timestamp) ```sql CREATE TABLE IF NOT EXISTS samples_metadata ON CLUSTER '{cluster}' ( series_id UUID, - timestamp DateTime64(3, ‘UTC’) CODEC(Delta(4), ZSTD), + timestamp DateTime64(3, 'UTC') CODEC(Delta(4), ZSTD), metadata Map(String, String) CODEC(ZSTD), ) ENGINE = ReplicatedMergeTree() PARTITION BY toYYYYMMDD(timestamp) @@ -207,7 +207,7 @@ PRIMARY KEY (labels, series_id) ``` ```sql -CREATE TABLE IF NOT EXISTS group_to_series ON CLUSTER ‘{cluster}’ ( +CREATE TABLE IF NOT EXISTS group_to_series ON CLUSTER '{cluster}'' ( group_id Uint64, series_id UUID, ) ORDER BY (group_id, series_id) @@ -217,7 +217,7 @@ CREATE TABLE IF NOT EXISTS group_to_series ON CLUSTER ‘{cluster}’ ( - sharding considerations for a given tenant when ingesting/persisting data if we intend to co-locate data specific to multiple tenants within the same database tables. To simplify things, segregating tenant-specific data to their own dedicated set of tables would make a lot of sense. -- structural considerations for “timestamps” when ingesting data across tenants. +- structural considerations for "timestamps" when ingesting data across tenants. - creation_time vs ingestion_time @@ -228,7 +228,7 @@ Slightly non-trivial but we can potentially investigate the possibility of using ### Pros - multiple tables -- Normalised data structuring allows for efficient storage of data, removing any redundancy across multiple samples for a given timeseries. Evidently, for the “samples” schema, we expect to store 32 bytes of data per metric point. +- Normalised data structuring allows for efficient storage of data, removing any redundancy across multiple samples for a given timeseries. Evidently, for the "samples" schema, we expect to store 32 bytes of data per metric point. - Better search complexity when filtering timeseries by labels/metadata, via the use of better indexed columns. @@ -245,17 +245,18 @@ Slightly non-trivial but we can potentially investigate the possibility of using ### Storage - multiple tables A major portion of our writes are made into the `samples` schema which contains a tuple containing three data points per metric point written: -|column|data type|byte size| -|---|---|---| -|series_id|UUID|16 bytes| -|timestamp|DateTime64|8 bytes| -|value|Float64|8 bytes| + +| Column | Data type | Byte size | +|:------------|:-----------|:----------| +| `series_id` | UUID | 16 bytes | +| `timestamp` | DateTime64 | 8 bytes | +| `value` | Float64 | 8 bytes | Therefore, we estimate to use 32 bytes per sample ingested. ### Compression - multiple tables -Inspecting the amount of compression we’re able to get with the given design on our major schemas, we see it as a good starting point. Following measurements for both primary tables: +Inspecting the amount of compression we're able to get with the given design on our major schemas, we see it as a good starting point. Following measurements for both primary tables: **Schema**: `labels_to_series` containing close to 12k unique `series_id`, each mapping to a set of 10-12 label string pairs @@ -308,7 +309,7 @@ Query id: 04219cea-06ea-4c5f-9287-23cb23c023d2 ### Performance - multiple tables -From profiling our reference implementation, it can also be noted that most of our time right now is spent in the application writing data to Clickhouse and/or its related operations. A “top” pprof profile sampled from the implementation looked like: +From profiling our reference implementation, it can also be noted that most of our time right now is spent in the application writing data to Clickhouse and/or its related operations. A "top" pprof profile sampled from the implementation looked like: ```shell (pprof) top @@ -329,26 +330,26 @@ Showing top 10 nodes out of 58 As is evident above from our preliminary analysis, writing data into Clickhouse can be a potential bottleneck. Therefore, on the write path, it'd be prudent to batch our writes into Clickhouse so as to reduce the amount of work the application server ends up doing making the ingestion path more efficient. -On the read path, it’s also possible to parallelize reads for the samples table either by series_id(s) OR by blocks of time between the queried start and end timestamps. +On the read path, it's also possible to parallelize reads for the samples table either by `series_id` OR by blocks of time between the queried start and end timestamps. ### Caveats -- When dropping labels from already existing metrics, we treat their new counterparts as completely new series and hence attribute them to a new series_id. This avoids having to merge series data and/or values. The old series, if not actively written into, should eventually fall off their retention and get deleted. +- When dropping labels from already existing metrics, we treat their new counterparts as completely new series and hence attribute them to a new `series_id`. This avoids having to merge series data and/or values. The old series, if not actively written into, should eventually fall off their retention and get deleted. -- We have not yet accounted for any data aggregation. Our assumption is that the backing store (in Clickhouse) should allow us to keep a “sufficient” amount of data in its raw form and that we should be able to query against it within our query latency SLOs. +- We have not yet accounted for any data aggregation. Our assumption is that the backing store (in Clickhouse) should allow us to keep a "sufficient" amount of data in its raw form and that we should be able to query against it within our query latency SLOs. ### **Rejected alternative**: Single, centralized table ### single, centralized data table ```sql -CREATE TABLE IF NOT EXISTS metrics ON CLUSTER ‘{cluster}’ ( +CREATE TABLE IF NOT EXISTS metrics ON CLUSTER '{cluster}' ( group_id UInt64, name LowCardinality(String) CODEC(ZSTD), labels Map(String, String) CODEC(ZSTD), metadata Map(String, String) CODEC(ZSTD), value Float64 CODEC (Gorilla, ZSTD), - timestamp DateTime64(3, ‘UTC’) CODEC(Delta(4),ZSTD) + timestamp DateTime64(3, 'UTC') CODEC(Delta(4),ZSTD) ) ENGINE = ReplicatedMergeTree() PARTITION BY toYYYYMMDD(timestamp) ORDER BY (group_id, name, timestamp); @@ -364,7 +365,7 @@ ORDER BY (group_id, name, timestamp); - Huge redundancy built into the data structure since attributes such as name, labels, metadata are stored repeatedly for each sample collected. -- Non-trivial complexity to search timeseries with values for labels/metadata given how they’re stored when backed by Maps/Arrays. +- Non-trivial complexity to search timeseries with values for labels/metadata given how they're stored when backed by Maps/Arrays. - High query latencies by virtue of having to scan large amounts of data per query made. @@ -372,14 +373,14 @@ ORDER BY (group_id, name, timestamp); ### Storage - single table -|column|data type|byte size| -|---|---|---| -|group_id|UUID|16 bytes| -|name|String|-| -|labels|Map(String, String)|-| -|metadata|Map(String, String)|-| -|value|Float64|8 bytes| -|timestamp|DateTime64|8 bytes| +| Column | Data type | Byte size | +|:------------|:--------------------|:----------| +| `group_id` | UUID | 16 bytes | +| `name` | String | - | +| `labels` | Map(String, String) | - | +| `metadata` | Map(String, String) | - | +| `value` | Float64 | 8 bytes | +| `timestamp` | DateTime64 | 8 bytes | NOTE: Strings are of an arbitrary length, the length is not limited. Their value can contain an arbitrary set of bytes, including null bytes. We will need to regulate what we write into these columns application side. @@ -486,7 +487,9 @@ We should only store data for a predetermined period of time, post which we eith ### Data access via SQL -While our corpus of data is PromQL-queryable, it would be prudent to make sure we make the SQL interface “generally available” as well. This capability opens up multiple possibilities to query resident data and allows our users to slice and dice their datasets whichever way they prefer to and/or need to. +While our corpus of data is PromQL-queryable, it would be prudent to make sure we make the SQL interface +"generally available" as well. This capability opens up multiple possibilities to query resident data and +allows our users to slice and dice their datasets whichever way they prefer to and/or need to. #### Challenges @@ -545,7 +548,7 @@ value: 0 On the read path, we first query all timeseries identifiers by searching for the labels under consideration. Once we have all the `series_id`(s), we then look up all corresponding samples between the query start timestamp and end timestamp. -For e.g. +For example: ```plaintext kernel{service_environment=~"prod.*", measurement="boot_time"} @@ -572,7 +575,7 @@ To account for newer writes when maintaining this cache: - Have TTLs on the keys, jittered per key so as to rebuild them frequently enough to account for new writes. -Once we know which timeseries we’re querying for, from there, we can easily look up all samples via the following query: +Once we know which timeseries we're querying for, from there, we can easily look up all samples via the following query: ```sql SELECT * @@ -592,13 +595,13 @@ yielding all timeseries samples we were interested in. We then render these into an array of `prometheus.QueryResult` object(s) and return back to the caller as a `prometheus.ReadResponse` object. NOTE: -The queries have been broken down into multiple queries only during our early experimentation/iteration, it’d be prudent to use subqueries within the same roundtrip to the database going forward into production/benchmarking. +The queries have been broken down into multiple queries only during our early experimentation/iteration, it'd be prudent to use subqueries within the same roundtrip to the database going forward into production/benchmarking. ## Production Readiness ### Batching -Considering we’ll need to batch data before ingesting large volumes of small writes into Clickhouse, the design must account for app-local persistence to allow it to locally batch incoming data before landing it into Clickhouse in batches of a predetermined size in order to increase performance and allow the table engine to continue to persist data successfully. +Considering we'll need to batch data before ingesting large volumes of small writes into Clickhouse, the design must account for app-local persistence to allow it to locally batch incoming data before landing it into Clickhouse in batches of a predetermined size in order to increase performance and allow the table engine to continue to persist data successfully. We have considered the following alternatives to implement app-local batching: @@ -623,7 +626,7 @@ We propose the following three dimensions be tested while benchmarking the propo - On-disk storage requirements (accounting for replication if applicable) - Mean query response times -For understanding performance, we’ll need to first compile a list of such queries given the data we ingest for our tests. Clickhouse query logging is super helpful while doing this. +For understanding performance, we'll need to first compile a list of such queries given the data we ingest for our tests. Clickhouse query logging is super helpful while doing this. NOTE: Ideally, we aim to benchmark the system to be able to ingest >1M metric points/sec while consistently serving most queries under <1 sec. diff --git a/doc/architecture/blueprints/gitlab_observability_backend/metrics/supported-deployments.png b/doc/architecture/blueprints/gitlab_observability_backend/supported-deployments.png Binary files differindex 9dccc515129..9dccc515129 100644 --- a/doc/architecture/blueprints/gitlab_observability_backend/metrics/supported-deployments.png +++ b/doc/architecture/blueprints/gitlab_observability_backend/supported-deployments.png diff --git a/doc/architecture/blueprints/modular_monolith/bounded_contexts.md b/doc/architecture/blueprints/modular_monolith/bounded_contexts.md new file mode 100644 index 00000000000..0f71e24864e --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/bounded_contexts.md @@ -0,0 +1,119 @@ +--- +status: proposed +creation-date: "2023-06-21" +authors: [ "@fabiopitino" ] +coach: [ ] +approvers: [ ] +owning-stage: "" +--- + +# Defining bounded contexts + +## Status quo + +Today the GitLab codebase doesn't have a clear domain structure. +We have [forced the creation of some modules](https://gitlab.com/gitlab-org/gitlab/-/issues/212156) +as a first step but we don't have a well defined strategy for doing it consistently. + +The majority of the code is not properly namespaced and organized: + +- Ruby namespaces used don't always represent the SSoT. We have overlapping concepts spread across multiple + namespaces. For example: `Abuse::` and `Spam::` or `Security::Orchestration::` and `Security::SecurityOrchestration`. +- Domain code related to the same bounded context is scattered across multiple directories. +- Domain code is present in `lib/` directory under namespaces that differ from the same domain under `app/`. +- Some namespaces are very shallow, containing a few classes while other namespaces are very deep and large. +- A lot of the old code is not namespaced, making it difficult to understand the context where it's used. + +## Goal + +1. Define a list of characteristics that bounded contexts should have. For example: must relate to at least 1 product category. +1. Have a list of top-level bounded contexts where all domain code is broken down into. +1. Engineers can clearly see the list of available bounded contexts and can make an easy decision where to add + new classes and modules. +1. Define a process for adding a new bounded context to the application. This should occur quite infrequently + and new bounded contexts need to adhere to the characteristics defined previously. +1. Enforce the list of bounded contexts so that no new top-level namespaces can be used aside from the authorized ones. + +## Iterations + +### 0. Extract libraries out of the codebase + +In June 2023 we've started extracing gems out of the main codebase, into +[`gems/` directory inside the monorepo](https://gitlab.com/gitlab-org/gitlab/-/blob/4c6e120069abe751d3128c05ade45ea749a033df/doc/development/gems.md). + +This is our first step towards modularization: externalize code that can be +extracted to prevent coupling from being introduced into modules that have been +designed as separate components. + +These gems as still part of the monorepo. + +### 1. What makes a bounded context? + +From the research in [Proposal: split GitLab monolith into components](https://gitlab.com/gitlab-org/gitlab/-/issues/365293) +it seems that following [product categories](https://about.gitlab.com/handbook/product/categories/#hierarchy), as a guideline, +would be much better than translating organization structure into folder structure (for example, `app/modules/verify/pipeline-execution/...`). + +However, this guideline alone is not sufficient and we need a more specific strategy: + +- Product categories can change ownership and we have seen some pretty frequent changes, even back and forth. + Moving code every time a product category changes ownership adds too much maintenance overhead. +- Teams and organization changes should just mean relabelling the ownership of specific modules. +- Bounded contexts (top level modules) should be [sufficiently deep](../../../development/software_design.md#use-namespaces-to-define-bounded-contexts) + to encapsulate implementation details and provide a smaller interface. +- Some product categories, such as Browser Performance Testing, are just too small to represent a bounded context on their own. + We should have a strategy for grouping product categories together when makes sense. +- Product categories don't necessarily translate into clean boundaries. + `Category:Pipeline Composition` and `Category:Continuous Integration` are some examples where Pipeline Authoring team + and Pipeline Execution team share a lot of code. +- Some parts of the code might not have a clear product category associated to it. + +Despite the above, product categories provide a rough view of the bounded contexts at play in the application. + +One idea could be to use product categories to sketch the initial set of bounded contexts. +Then, group related or strongly coupled categories under the same bounded context and create new bounded contexts if missing. + +### 2. Identify existing bounded contexts + +Start with listing all the Ruby files in a spreadsheet and categorize them into components following the guidelines above. +Some of them are already pretty explicit like Ci::, Packages::, etc. Components should follow our +[existing naming guide](../../../development/software_design.md#use-namespaces-to-define-bounded-contexts). + +This could be a short-lived Working Group with representative members of each DevOps stage (for example, Senior+ engineers). +The WG would help defining high-level components and will be the DRIs for driving the changes in their respective DevOps stage. + +### 3. Publish the list of bounded contexts + +The list of bounded contexts (top-level namespaces) extracted from the codebase should be defined statically so it can be +used programmatically. + +```yaml +# file: config/bounded_contexts.yml +bounded_contexts: + continuous_integration: + dir: modules/ci + namespace: 'Ci::' + packages: ... + merge_requests: ... + git: ... +``` + +With this static list we could: + +- Document the existing bounded contexts for engineers to see the big picture. +- Understand where to place new classes and modules. +- Enforce if any top-level namespaces are used that are not in the list of bounded contexts. +- Autoload non-standard Rails directories based on the given list. + +## Glossary + +- `modules` are Ruby modules and can be used to nest code hierarchically. +- `namespaces` are unique hierarchies of Ruby constants. For example, `Ci::` but also `Ci::JobArtifacts::` or `Ci::Pipeline::Chain::`. +- `packages` are Packwerk packages to group together related functionalities. These packages can be big or small depending on the design and architecture. Inside a package all constants (classes and modules) have the same namespace. For example: + - In a package `ci`, all the classes would be nested under `Ci::` namespace. There can be also nested namespaces like `Ci::PipelineProcessing::`. + - In a package `ci-pipeline_creation` all classes are nested under `Ci::PipelineCreation`, like `Ci::PipelineCreation::Chain::Command`. + - In a package `ci` a class named `MergeRequests::UpdateHeadPipelineService` would not be allowed because it would not match the package's namespace. + - This can be enforced easily with [Packwerk's based Rubocop Cops](https://github.com/rubyatscale/rubocop-packs/blob/main/lib/rubocop/cop/packs/root_namespace_is_pack_name.rb). +- `bounded context` is a top-level Packwerk package that represents a macro aspect of the domain. For example: `Ci::`, `MergeRequests::`, `Packages::`, etc. + - A bounded context is represented by a single Ruby module/namespace. For example, `Ci::` and not `Ci::JobArtifacts::`. + - A bounded context can be made of 1 or multiple Packwerk packages. Nested packages would be recommended if the domain is quite complex and we want to enforce privacy among all the implementation details. For example: `Ci::PipelineProcessing::` and `Ci::PipelineCreation::` could be separate packages of the same bounded context and expose their public API while keeping implementation details private. + - A new bounded context like `RemoteDevelopment::` can be represented a single package while large and complex bounded contexts like `Ci::` would need to be organized into smaller/nested packages. diff --git a/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/hexagonal_architecture.png b/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/hexagonal_architecture.png Binary files differnew file mode 100644 index 00000000000..a8d79e276a2 --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/hexagonal_architecture.png diff --git a/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/index.md b/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/index.md new file mode 100644 index 00000000000..eb4b428cf52 --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/hexagonal_monolith/index.md @@ -0,0 +1,132 @@ +--- +status: proposed +creation-date: "2023-05-22" +authors: [ "@fabiopitino" ] +coach: [ ] +approvers: [ ] +owning-stage: "" +--- + +# Hexagonal Rails Monolith + +## Summary + +**TL;DR:** Change the Rails monolith from a [big ball of mud](https://en.wikipedia.org/wiki/Big_ball_of_mud) state to +a [modular monolith](https://www.thereformedprogrammer.net/my-experience-of-using-modular-monolith-and-ddd-architectures) +that uses an [Hexagonal architecture](https://en.wikipedia.org/wiki/Hexagonal_architecture_(software)) (or ports and adapters architecture). +Extract cohesive functional domains into separate directory structure using Domain-Driven Design practices. +Extract infrastructure code (logging, database tools, instrumentation, etc.) into gems, essentially remove the need for `lib/` directory. +Define what parts of the functional domains (for example application services) are of public use for integration (the ports) +and what parts are instead private encapsulated details. +Define Web, Sidekiq, REST, GraphQL, and Action Cable as the adapters in the external layer of the architecture. +Use [Packwerk](https://github.com/Shopify/packwerk) to enforce privacy and dependency between modules of the monolith. + + + +## Details + +### Application domain + +The application core (functional domains) is divided into separate top-level bounded contexts called after the +[feature category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) they represent. +A bounded-context is represented in the form of a Ruby module. +This follows the existing [guideline on naming namespaces](../../../../development/software_design.md#use-namespaces-to-define-bounded-contexts) but puts more structure to it. + +Modules should: + +- Be deep enough to encapsulate a lot of the internal logic, state and data. +- Have a public interface that is as small as possible, safe to use by other bounded contexts and well documented. +- Be cohesive and represent the SSoT (single source of truth) of the feature it describes. + +Feature categories represent a product area that is large enough for the module to be deep, so we don't have a proliferation +of small top-level modules. It also helps the codebase to follow the +[ubiquitous language](../../../../development/software_design.md#use-ubiquitous-language-instead-of-crud-terminology). +A team can be responsible for multiple feature categories, hence owning the vision for multiple bounded contexts. +While feature categories can sometimes change ownership, this change of mapping the bounded context to new owners +is very cheap. +Using feature categories also helps new contributors, either as GitLab team members of members of the wider community, +to navigate the codebase. + +If multiple feature categories are strongly related, they may be grouped under a single bounded context. +If a feature category is only relevant in the context of a parent feature category, it may be included in the +parent's bounded context. For example: Build artifacts existing in the context of Continuous Integration feature category +and they may be merged under a single bounded context. + +### Application adapters + +>>> +_Adapters are the glue between components and the outside world._ +_They tailor the exchanges between the external world and the ports that represent the requirements of the inside_ +_of the application component. There can be several adapters for one port, for example, data can be provided by_ +_a user through a GUI or a command-line interface, by an automated data source, or by test scripts._ - +[Wikipedia](https://en.wikipedia.org/wiki/Hexagonal_architecture_(software)#Principle) +>>> + +Application adapters would be: + +- Web UI (Rails controllers, view, JS and Vue client) +- REST API endpoints +- GraphQL Endpoints +- Action Cable + +TODO: continue describing how adapters are organized and why they are separate from the domain code. + +### Platform code + +For platform code we consider any classes and modules that are required by the application domain and/or application +adapters to work. + +The Rails' `lib/` directory today contains multiple categories of code that could live somewhere else, +most of which is platform code: + +- REST API endpoints could be part of the [application adapters](#application-adapters). +- domain code (both large domain code such as `Gitlab::Ci` and small such as `Gitlab::JiraImport`) should be + moved inside the [application domain](#application-domain). +- The rest could be extracted as separate single-purpose gems under the `gems/` directory inside the monolith. + This can include utilities such as logging, error reporting and metrics, rate limiters, + infrastructure code like `Gitlab::ApplicationRateLimiter`, `Gitlab::Redis`, `Gitlab::Database` + and generic subdomains like `Banzai`. + +Base classes to extend Rails framework such as `ApplicationRecord` or `ApplicationWorker` as well as GitLab base classes +such as `BaseService` could be implemented as gem extensions. + +This means that aside from the Rails framework code, the rest of the platform code resides in `gems/`. + +Eventually all code inside `gems/` could potentially be extracted in a separate repository or open sourced. +Placing platform code inside `gems/` makes it clear that its purpose is to serve the application code. + +### Why Packwerk? + +TODO: + +- boundaries not enforced at runtime. Ruby code will still work as being all loaded in the same memory space. +- can be introduced incrementally. Not everything requires to be moved to packs for the Rails autoloader to work. + +Companies like Gusto have been developing and maintaining a list of [development and engineering tools](https://github.com/rubyatscale) +for organizations that want to move to using a Rails modular monolith around Packwerk. + +### EE and JH extensions + +TODO: + +## Challenges + +- Such changes require a shift in the development mindset to understand the benefits of the modular + architecture and not fallback into legacy practices. +- Changing the application architecture is a challenging task. It takes time, resources and commitment + but most importantly it requires buy-in from engineers. +- This may require us to have a medium-long term team of engineers or a Working Group that makes progresses + on the architecture evolution plan, foster discussions in various engineering channels and resolve adoption challenges. +- We need to ensure we build standards and guidelines and not silos. +- We need to ensure we have clear guidelines on where new code should be placed. We must not recreate junk drawer folders like `lib/`. + +## Opportunities + +The move to a modular monolith architecture enables a lot of opportunities that we could explore in the future: + +- We could align the concept of domain expert with explicitly owning specific modules of the monolith. +- The use of static analysis tool (such as Packwerk, Rubocop) can catch design violations in development and CI, ensuring + that best practices are honored. +- By defining dependencies between modules explicitly we could speed up CI by testing only the parts that are affected by + the changes. +- Such modular architecture could help to further decompose modules into separate services if needed. diff --git a/doc/architecture/blueprints/modular_monolith/index.md b/doc/architecture/blueprints/modular_monolith/index.md new file mode 100644 index 00000000000..ef50be643a6 --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/index.md @@ -0,0 +1,112 @@ +--- +status: proposed +creation-date: "2023-05-22" +authors: [ "@grzesiek", "@fabiopitino" ] +coach: [ ] +approvers: [ ] +owning-stage: "" +participating-stages: [] +--- + +<!-- vale gitlab.FutureTense = NO --> + +# GitLab Modular Monolith + +## Summary + +The main [GitLab Rails](https://gitlab.com/gitlab-org/gitlab) +project has been implemented as a large monolithic application, using +[Ruby on Rails](https://rubyonrails.org/) framework. It has over 2.2 million +lines of Ruby code and hundreds of engineers contributing to it every day. + +The application has been growing in complexity for more than a decade. The +monolithic architecture has served us well during this time, making it possible +to keep high development velocity and great engineering productivity. + +Even though we strive for having [an approachable open-core architecture](https://about.gitlab.com/blog/2022/07/14/open-core-is-worse-than-plugins/) +we need to strengthen the boundaries between domains to retain velocity and +increase development predictability. + +As we grow as an engineering organization, we want to explore a slightly +different, but related, architectural paradigm: +[a modular monolith design](https://en.wikipedia.org/wiki/Modular_programming), +while still using a [monolithic architecture](https://en.wikipedia.org/wiki/Monolithic_application) +with satellite services. + +This should allow us to increase engineering efficiency, reduce the cognitive +load, and eventually decouple internal components to the extend that will allow +us to deploy and run them separately if needed. + +## Motivation + +Working with a large and tightly coupled monolithic application is challenging: + +Engineering: + +- Onboarding engineers takes time. It takes a while before engineers feel + productive due to the size of the context and the amount of coupling. +- We need to use `CODEOWNERS` file feature for several domains but + [these rules are complex](https://gitlab.com/gitlab-org/gitlab/-/blob/409228f064a950af8ff2cecdd138fc9da41c8e63/.gitlab/CODEOWNERS#L1396-1457). +- It is difficult for engineers to build a mental map of the application due to its size. + Even apparently isolated changes can have [far-reaching repercussions](https://about.gitlab.com/handbook/engineering/development/#reducing-the-impact-of-far-reaching-work) + on other parts of the monolith. +- Attrition/retention of engineering talent. It is fatiguing and demoralizing for + engineers to constantly deal with the obstacles to productivity. + +Architecture: + +- There is little structure inside the monolith. We have attempted to enforce + the creation [of some modules](https://gitlab.com/gitlab-org/gitlab/-/issues/212156) + but have no company-wide strategy on what the functional parts of the + monolith should be, and how code should be organized. +- There is no isolation between existing modules. Ruby does not provide + out-of-the-box tools to effectively enforce boundaries. Everything lives + under the same memory space. +- We rarely build abstractions that can boost our efficiency. +- Moving stable parts of the application into separate services is impossible + due to high coupling. +- We are unable to deploy changes to specific domains separately and isolate + failures that are happening inside them. + +Productivity: + +- High median-time-to-production for complex changes. +- It can be overwhelming for the wider-community members to contribute. +- Reducing testing times requires diligent and persistent efforts. + +## Goals + +- Increase the development velocity and predicability through separation of concerns. +- Improve code quality by reducing coupling and introducing useful abstractions. +- Build abstractions required to deploy and run GitLab components separately. + +## How do we get there? + +While we do recognize that modularization is a significant technical endeavor, +we believe that the main challenge is organizational, rather than technical. We +not only need to design separation in a way that modules are decoupled in a +pragmatic way which works well on GitLab.com but also on self-managed +instances, but we need to align modularization with the way in which we want to +work at GitLab. + +There are many aspects and details required to make modularization of our +monolith successful. We will work on the aspects listed below, refine them, and +add more important details as we move forward towards the goal: + +1. [Deliver modularization proof-of-concepts that will deliver key insights](proof_of_concepts.md) +1. [Align modularization plans to the organizational structure](bounded_contexts.md) +1. Start a training program for team members on how to work with decoupled domains (TODO) +1. Build tools that will make it easier to build decoupled domains through inversion of control (TODO) +1. Separate domains into modules that will reflect organizational structure (TODO) +1. Build necessary services to align frontend and backend modularization (TODO) +1. [Introduce hexagonal architecture within the monolith](hexagonal_monolith/index.md) +1. Introduce clean architecture with one-way-dependencies and host application (TODO) +1. Build abstractions that will make it possible to run and deploy domains separately (TODO) + +## Status + +In progress. + +## References + +[List of references](references.md) diff --git a/doc/architecture/blueprints/modular_monolith/proof_of_concepts.md b/doc/architecture/blueprints/modular_monolith/proof_of_concepts.md new file mode 100644 index 00000000000..c215ffafbe4 --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/proof_of_concepts.md @@ -0,0 +1,134 @@ +--- +status: proposed +creation-date: "2023-07-05" +authors: [ "@grzesiek", "@fabiopitino" ] +coach: [ ] +owners: [ ] +--- + +# Modular Monolith: PoCs + +Modularization of our monolith is a complex project. There will be many +unknowns. One thing that can help us mitigate the risks and deliver key +insights are Proof-of-Concepts that we could deliver early on, to better +understand what will need to be done. + +## Inter-module communicaton + +A PoC that we plan to deliver is a PoC of inter-module communication. We do +recognize the need to separate modules, but still allow them to communicate +together using a well defined interface. Modules can communicate through a +facade classes (like libraries usually do), or through eventing system. Both +ways are important. + +The main question is: how do we want to define the interface and how to design +the communication channels? + +It is one of our goals to make it possible to plug modules out, and operate +some of them as separate services. This will make it easier deploy GitLab.com +in the future and scale key domains. One possible way to achieve this goal +would be to design the inter-module communication using a protobuf as an +interface and gRPC as a communication channel. When modules are plugged-in, we +would bypass gRPC and serialization and use in-process communication primitives +(while still using protobuf as an interface). When a module gets plugged-out, +gRPC would carry messages between modules. + +## Use Packwerk to enforce module boundaries + +Packwerk is a static analyzer that helps defining and enforcing module boundaries +in Ruby. + +[In this PoC merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98801) +we demonstrate a possible directory structure of the monolith broken down into separate +modules. + +The PoC also aims to solve the problem of EE extensions (and JH too) allowing the +Rails autoloader to be tweaked depending on whether to load only the Core codebase or +any extensions. + +The PoC also attempted to only move a small part of the `Ci::` namespace into a +`components/ci` Packwerk package. This seems to be the most iterative approach +explored so far. + +There are different approaches we could use to adopt Packwerk. Other PoC's also +explored are the [large extraction of CI package](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88899) +and [moving the 2 main CI classes into a package](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/90595). + +All 3 PoC's have a lot in common, from the introduction of Packwerk packages and configurations +to setting paths for the autoloader to work with any packages. What changes between the +various merge requests is the approach on choosing which files to move first. + +The main goals of the PoC were: + +- understand if Packwerk can be used on the GitLab codebase. +- understand the learning curve for developers. +- verify support for EE and JH extensions. +- allow gradual modularization. + +### Positive results + +- Using Packwerk would be pretty simple on GitLab since it's designed primarily to work + on Rails codebases. +- We can change the organization of the domain code to be module-oriented instead of following + the MVC pattern. It requires small initial changes to allow the Rails autoloading + to support the new directory structure, which is by the way not imposed by Packwerk. + After that, registering a new top-level package/bounded-context would be a 1 LOC change. +- Using the correct directory structure indicated in the [PoC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98801) + allows packages to contain all the code, including EE and JH extensions. +- Gradual modularization is possible and we can have any degree of modularization as we want, + from initial no enforcement down to complete isolation simulating an in-memory micro-service environment. +- Moving files into a Packwerk package doesn't necessarily mean renaming constants. + While this is not advisable long term, its an extra flexibility that the tool provides. + - For example: If we are extracting the `Ci::` module into a Packwerk package there can be + constants that belong to the CI domain but are not namespaced, like `CommitStatus` or + that have a different namespace, like `Gitlab::Ci::`. + Packwerk allows such constants to be moved inside the `ci` package and correctly flags + boundary violations. + - Packwerk enhancements from RubyAtScale tooling allow to enforce that all constants inside + a package share the same Ruby namespace. We eventually would want to leverage that. +- RubyAtScale provides also tools to track metrics about modularization and adoption which we + would need to monitor and drive as an engineering organization. +- Packwerk has IDE extensions (e.g. for VSCode) to provide realtime feedback on violations + (like Rubocop). It can also be run via CLI during the development workflow against a single + package. It could be integrated into pre-push Git hooks or Danger during code reviews. + +### Challenges + +Some of these challenges are not specific to Packwerk as tool/approach. They were observed +during the PoC and are more generically related to the process of modularization: + +- There is no right or wrong approach when introducing Packwerk packages. We need to define + clear guidelines to give developers the tools to make the best decision: + - Sometimes it could be creating an empty package and move files in it gradually. + - Sometimes it could be wrapping an already well designed and isolated part of the codebase. + - Sometimes it could be creating a new package from scratch. +- As we move code to a different directory structure we need to involve JiHu as they manage + extensions following the current directory structure. + We may have modules that are partially migrated and we need to ensure JiHu is up-to-date + with the current progresses. +- After privacy/dependency checks are enabled, Packwerk will log a lot of violations + (like Rubocop TODOs) since constant references in a Rails codebase are very entangled. + - The team owning the package needs to define a vision for the package. + What would the package look like once all violations have been fixed? + This may mean specifying where the package fits in the + [context map](https://www.oreilly.com/library/view/what-is-domain-driven/9781492057802/ch04.html) + of the system. How the current package should be used by another package `A` and how + it should use other packages. + - The vision above should tell developers how they should fix these violations over time. + Should they make a specific constant public? Should the package list another package as its + dependencies? Should events be used in some scenarios? + - Teams will likely need guidance in doing that. We may need to have a team of engineers, like + maintainers with a very broad understanding of the domains, that will support engineering + teams in this effort. +- Changes to CI configurations on tuning Knapsack and selective testing were ignored durign the + PoC. + +## Frontend sorting hat + +Frontend sorting-hat is a PoC for combining multiple domains to render a full +page of GitLab (with menus, and items that come from multiple separate +domains). + +## Frontend assets aggregation + +Frontend assets aggregation is a PoC for a possible separation of micro-frontends. diff --git a/doc/architecture/blueprints/modular_monolith/references.md b/doc/architecture/blueprints/modular_monolith/references.md new file mode 100644 index 00000000000..2c7d3dc972d --- /dev/null +++ b/doc/architecture/blueprints/modular_monolith/references.md @@ -0,0 +1,70 @@ +--- +status: proposed +creation-date: "2023-06-21" +authors: [ "@fabiopitino" ] +coach: [ ] +approvers: [ ] +owning-stage: "" +--- + +# References + +## Related design docs + +- [Composable codebase design doc](../composable_codebase_using_rails_engines/index.md) + +## Related Issues + +- [Split GitLab monolith into components](https://gitlab.com/gitlab-org/gitlab/-/issues/365293) +- [Make it simple to build and use "Decoupled Services"](https://gitlab.com/gitlab-org/gitlab/-/issues/31121) +- [Use nested structure to organize CI classes](https://gitlab.com/gitlab-org/gitlab/-/issues/209745) +- [Create new models / classes within a module / namespace](https://gitlab.com/gitlab-org/gitlab/-/issues/212156) +- [Make teams to be maintainers of their code](https://gitlab.com/gitlab-org/gitlab/-/issues/25872) +- [Add backend guide for Dependency Injection](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73644) + +## Internal Slack Channels + +- [`#modular_monolith`](https://gitlab.slack.com/archives/C03NTK6HZBM) +- [`#architecture`](https://gitlab.slack.com/archives/CJ4DB7517) + +## Reference Implementations / Guides + +Gusto / RubyAtScale: + +- [RubyAtScale toolchain for modularization](https://github.com/rubyatscale) +- [Gusto's engineering blog](https://engineering.gusto.com/laying-the-cultural-and-technical-foundation-for-big-rails/) +- [Gradual modularization](https://gradualmodularization.com/) (successor to CBRA) +- [Component-Based Rails Applications](https://cbra.info) ("deprecated") + +Shopify: + +- [Packwerk](https://github.com/Shopify/packwerk) +- [Shopify's jurney to modularization](https://shopify.engineering/shopify-monolith) +- [Internal GitLab doc transcript of an AMA with a Shopify engineer](https://docs.google.com/document/d/1uZbcaK8Aqs-D_n7_uQ5XE295r5UWDJEBwA6g5bTjcwc/edit#heading=h.d1tml5rlzrpa) + +Domain-Driven Rails / Rails Event Store: + +Rails Event Store is relevant because it is a mechanism to achieve many +of the goals discussed here, and is based upon patterns used by Arkency +to build production applications. + +This doesn't mean we need to use this specific framework or approach. + +However, the general concepts of DDD/ES/CQRS are important and in some +cases maybe necessary to achieve the goals of this blueprint, so it's +useful to have concrete production-proven implementations of those +concepts to look at as an example. + +- [Arkency's domain-driven Rails](https://products.arkency.com/domain-driven-rails/) +- [Arkency's Rails Event Store](https://railseventstore.org) + +App Continuum: + +An illustration of how an application can evolve from a small, unstructured app, through various +stages including a modular well-structured monolith, all the way to a microservices architecture. + +Includes discussion of why you might want to stop at various stages, and specifically the +challenges/concerns with making the jump to microservices, and why sticking with a +well-structured monolith may be preferable in many cases. + +- [App Continuum](https://www.appcontinuum.io) diff --git a/doc/architecture/blueprints/object_pools/index.md b/doc/architecture/blueprints/object_pools/index.md index d14e11b8d36..7b7f8d7d180 100644 --- a/doc/architecture/blueprints/object_pools/index.md +++ b/doc/architecture/blueprints/object_pools/index.md @@ -805,10 +805,10 @@ pools as it will always match the contents of the upstream repository. It has a number of downsides though: -- Normal repositories can now have different states, where some of the +- Repositories can now have different states, where some of the repositories are allowed to prune objects and others aren't. This introduces a source of uncertainty and makes it easy to accidentally delete objects in a - normal repository and thus corrupt its forks. + repository and thus corrupt its forks. - When upstream repositories go private we must stop updating objects which are supposed to be deduplicated across members of the fork network. This means diff --git a/doc/architecture/blueprints/observability_tracing/arch.png b/doc/architecture/blueprints/observability_tracing/arch.png Binary files differnew file mode 100644 index 00000000000..36ff23dc8a5 --- /dev/null +++ b/doc/architecture/blueprints/observability_tracing/arch.png diff --git a/doc/architecture/blueprints/observability_tracing/index.md b/doc/architecture/blueprints/observability_tracing/index.md new file mode 100644 index 00000000000..4291683f83f --- /dev/null +++ b/doc/architecture/blueprints/observability_tracing/index.md @@ -0,0 +1,171 @@ +--- +status: proposed +creation-date: "2023-06-20" +authors: [ "@mappelman" ] +approvers: [ "@hbenson", "@nicholasklick" ] +owning-stage: "~devops::monitor" +participating-stages: [] +--- + +# Distributed Tracing Feature + +## Summary + +GitLab already has distributed tracing as a feature. So this proposal focuses on the intended changes required to GA the feature. Given the strategic direction update which is covered more in the motivation section, we are deprecating the GitLab Observability UI (GOUI) in favor of building native UI for tracing in GitLab UI. + +This proposal covers the scope and technical approach to what will be released in GA, including the new UI, API changes and any backend changes to support the new direction. + +Distributed Tracing will GA as a premium feature, initially only available to premium and ultimate users. + +## Motivation + +In December 2021 GitLab acquired OpsTrace and kicked off work integrating Observability functionality into the DevOps platform. At that point the stated goal was to create an observability distribution that could be run independently of GitLab and which integrated well into the DevSecOps platform. See [Internal Only- Argus FAQ](https://docs.google.com/document/d/1eWZhbRdgQx74udzZjpSMgWnHfpYWETD7AWqnPVD5Sm8/edit) for more background on previous strategy. + +Since December 2021 there have been a lot of changes in the world and at GitLab. It is GitLabs belief that Observability should be natively built within GitLab UI to avoid fracturing capabilities and ensuring a singular UX. As such we are deprecating GitLab Observability UI which began life as a fork of Grafana in December 2021. + +Much of the GitLab Observability architecture and features were built around the fork of Grafana. As such, this proposal is part of a series of proposals that align us toward achieving the following high level objectives. + +## Observability Group Objectives + +The following group-level objectives are included for context. **The Objectives below are not completely covered by this design. This design focuses on distributed tracing. More design documents will be created for logging, metrics, auto-monitor, etc.** + +**Timeline**: Completion of the following before December 2024 + +**Objectives**: + +- GA of a Complete (Metrics, Logs, Traces) Observability Platform - Add on-by-default setup for tracing, metrics and logging including a GA for the service available on GitLab.com and for self-managed users. A user is able to trace micro-services or distributed systems using open-source tracers. Furthermore, users should be able to set sane defaults for sampling or use advanced techniques such as tail-based sampling. + +- Tailored Triage Workflow - Users need to connect the dots between Metrics, Logs, and Spans/Traces. Designing for the discovery, querying and the connection of all telemetry data, regardless of type, will aid users to resolve critical alerts and incidents more quickly. + +- Auto Monitor - When a developer starts a new project their application is automatically instrumented, alerts are set up and linked to GitLab alerts management, schedules are created and incidents are created for critical alerts. + +### Goals + +To release a generally available distributed tracing feature as part of GitLab.com SaaS with a minimum featureset such that it is valuable but can be iterated upon. + +Specific goals: + +- An HTTPS write API implemented in the [GitLab Observability Backend](https://GitLab.com/GitLab-org/opstrace/opstrace) project which receives spans sent to GitLab using [OTLP (OpenTelemetry Protocol)](https://opentelemetry.io/docs/specs/otel/protocol/). Users can collect and send distributed traces using either the [OpenTelemetry SDK](https://opentelemetry.io/docs/collector/deployment/no-collector/) or the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/). +- UI to list and filter/search for traces by ID, service, attributes or time +- UI to show a detail view of a trace and its corresponding spans +- Apply sensible ingestion and storage limits per top-level namespace for all GitLab tiers + +## Timeline + +In order to achieve the group objectives, the following timelines must be met for [GitLab phased rollout](https://about.GitLab.com/handbook/product/GitLab-the-product/#experiment-beta-ga) of Tracing. + +- **Tracing Experiment Release**: 16.2 +- **Tracing Beta Release**: 16.3 +- **Tracing GA Release**: 16.4 + +## Proposal + +Much of the proposed architecture already exists and is in operation for GitLab.com. Distributed tracing has already been in an internal **Beta** for quite some time and has internal users, with graduation to GA being blocked by UX requirements. These UX requirements resulted in the new UI strategy. + + The following diagram outlines the architecture for GitLab Observability Backend and how clients, including the GitLab UI, will interact with it. + +<img src="./arch.png"> + +### Key Components + +- Gatekeeper: Responsible for authentication, authorization and rate limit enforcement on all incoming requests. NGINX-Ingress interacts directly with Gatekeeper. +- Ingress: NGINX-Ingress is used to handle all incoming requests +- ClickHouse: ClickHouse is the backing store for all observability data +- Query Service: A horizontally scalable service that retrieves data from ClickHouse in response to a query +- GitLab UI: The UI hosted at GitLab.com +- Redis: An HA Redis cluster for caching GitLab API responses + +### Data Ingest + +One data ingestion pipeline will be deployed for each top level GitLab namespace. Currently we deploy one pipeline _per GitLab Group that enables observability_ and this architecture is now unnecessarily expensive and complex without the presence of the multi-tenant Grafana instances. This multi-tenant ingestion system has the following benefits: + +- Beyond rate limits, resource limits can be enforced per user such that no user can steal more system resources (memory, cpu) than allocated. +- Fine grained control of horizontal scaling for each user pipeline by adding more OTEL Collector instances +- Manage the users tenant in accordance to GitLab subscription tier, for example, quota, throughput, cold storage, shard to different databases +- Reduced complexity and enhanced security in the pipeline by leveraging off the shelf components like the [OpenTelemetry Collector](https://opentelemetry.io/docs/concepts/components/#collector) where data within that collector belongs to no more than a single user/customer. + +A pipeline is only deployed for the user upon enabling observability in the project settings, in the same way a user can enable error tracking for their project. When observability is enabled for any project in the users namespace, a pipeline will be deployed. This deployment is automated by our Kubernetes scheduler-operator and tenant-operator. Provisioning is currently managed through the iframe, but a preferred method would be to provision using a RESTful API. The GitLab UI would have a section in project settings that allow a user to "enable observability", much like they do for error tracking today. + +The opentelemetry collector is used as the core pipeline implementation for its excellent community development of receivers, processors and exporters. [An exporter for ClickHouse has emerged in the community](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter) which we intend to leverage and it currently has support for opentelemetry traces, metrics and logs. This will help accelerate the effort toward ingesting not just traces but also metrics and logs. + +### Limits + +In addition to the existing cpu and memory limits for each ingest pipeline, the following limits and quotas will also be enforced: + +- 100KB (possibly increase this to 1MB) total ingest rate of traces per second per top level namespace +- 30 day data retention +- TBD GB total storage + +All above limits are subject to change and will be driven by top level namespace configuration so scripts and future features can be built to make these more dynamic for each user or subscription tier. This configuration will be part of the tenant-operator custom resource. + +The ingest rate limit will utilize the internal Redis cluster to perform a simple, performant [sliding window rate limit like Cloudflare](https://blog.cloudflare.com/counting-things-a-lot-of-different-things/). The code for this will live in Gatekeeper, where a connection to Redis is already managed. + +The data retention and total storage limits will be enforced by a control loop in the tenant-operator that will periodically query ClickHouse and continuously delete the oldest whole day of data until quota is no longer exceeded. To do this efficiently, its important that ClickHouse tables are partitioned using `toDate(timestamp)` to partition by day. + +### Query API + +The query API, backed by the query service, will be a centralized, horizontally scalable component responsible for returning traces/spans back to the UI. A good starting point for this query service may be to leverage the Jaeger query service code and the [Jaeger query service swagger](https://github.com/Jaegertracing/Jaeger-idl/blob/main/swagger/api_v3/query_service.swagger.json). This query service will be extended to include support for metrics and logs in the future and will be queried directly by vue.js code in GitLab UI. + +The scope of effort for GA would include two APIs: + +- `/v1/traces` adhering to [this spec](https://github.com/Jaegertracing/Jaeger-idl/blob/main/swagger/api_v3/query_service.swagger.json#L64) +- `/v1/traces/{trace_ID}` adhering to [this spec](https://github.com/Jaegertracing/Jaeger-idl/blob/main/swagger/api_v3/query_service.swagger.json#L142) + +### Authentication and Authorization + +<!-- markdownlint-disable-next-line MD044 --> +GitLab Observability Backend utilizes an [instance-wide trusted GitLab OAuth](https://docs.GitLab.com/ee/integration/OAuth_provider.html#create-an-instance-wide-application) token to perform a seamless OAuth flow that authenticates the GitLab user against the GitLab Observability Backend (GOB). GOB creates an auth session and stores the session identifier in an http-only, secure cookie. This mechanism has already been examined and approved by AppSec. Now that the Observability UI will be native within the UI hosted at GitLab.com, a few small adjustments must be made for authentication to work against the new UI domain vs the embedded iframe that we previously relied upon (GitLab.com instead of observe.gitLab.com). + +A hidden iframe will be embedded in the GitLab UI only on pages where GOB authenticated APIs must be consumed. This allows GitLab.com UI to directly communicate with GOB APIs without the need for an intermediate proxy layer in rails and without relying on the less secure shared token between proxy and GOB. This iframe will be hidden and its sole purpose is to perform the OAuth flow and assign the http-only secure cookie containing the GOB user session. This flow is seamless and can be fully hidden from the user since its a **trusted** GitLab OAuth flow. Sessions currently expire after 30 days which is configurable in GOB deployment terraform. + +<!-- markdownlint-disable-next-line MD044 --> +In order to allow requests from GitLab.com directly to observe.gitLab.com using this method, all requests will have to include `{withCredentials: true}` in order to include cookies. For these "readonly" APIs that GitLab.com will query for tracing data, we must: + +- Configure Ingress with `"NGINX.Ingress.Kubernetes.io/cors-allow-credentials": "true"` and `"NGINX.Ingress.Kubernetes.io/cors-allow-origin": "GitLab.com"` +- Ensure we have effective CSRF protection enabled in our Gatekeeper component (Gatekeeper is responsible request authorization) + +<!-- markdownlint-disable-next-line MD044 --> +All requests from GitLab.com will then include the GOB session cookie for observe.gitLab.com to validate. Authorization is handled by the Gatekeeper component which checks group/project membership against GitLab and handles access appropriately. Anyone with inherited developer or above membership will have access to the tracing UI for that project. + +### Database Schema + +[The community developed OTEL exporter for ClickHouse](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter) has already implemented database schemas for storing traces and spans. [This blog post from ClickHouse](https://clickhouse.com/blog/storing-traces-and-spans-open-telemetry-in-clickhouse) further delves into the details of the community developed exporter and we intend to use the suggested schema design as a starting point for us to test during experiment and beta phases. It's recommended to read the blog post to learn more about the schemas and corresponding SQL queries we intend to try. + +### UI Design + +The new UI will be built using the Pajamas Design System in accordance with GitLab UX design standards. The UI will interact with the GOB query service directly from vue.js (see architecture diagram above) by sending a fetch to the subdomain `observe.gitLab.com/v1/query` with `{withCredentials: true}`. See the Authentication and Authorization section above for more details on how this is enabled. + +[**TODO Figma UI designs and commentary**] + +## Iterations + +16.2 + +- migrate all resources attached to the Group CR to the Tenant CR +- [fork and build Clickhouse exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter) +- add project_ID to all traces/spans +- gatekeeper: check membership at project level instead of group level +- basic query service for listing traces (no filtering/searching) +- implement hidden iframe-based OAuth mechanism (reuse/adapt what already done for GOUI) +- UI for traces list + +16.3 + +- filtering/searching query service (by traceID, service, status, duration min/max, start/end time, span attributes) +- add `read_observability` and `write_observability` scopes to Project access token and support Project access token for writing to project level ingest API +- provision API +- remove existing iframe provisioning +- UI for trace detail +- UI for filtering/searching traces +- basic e2e test for provision, send data, query in UI +- metrics, dashboards, alerts + +16.4 + +- UI settings page to "enable observability" (this would interact with provisioning API) +- production readiness review +- documentation complete +- alter GitLabNamespace CR to only represent a tenant (i.e. top level namespace) +- delete Group CR and corresponding controller +- e2e tests that haven't been added yet +- in cluster smoke test diff --git a/doc/architecture/blueprints/organization/index.md b/doc/architecture/blueprints/organization/index.md index be99211754d..2cfaf33ff50 100644 --- a/doc/architecture/blueprints/organization/index.md +++ b/doc/architecture/blueprints/organization/index.md @@ -16,23 +16,23 @@ This document is a work in progress and represents the current state of the Orga ## Glossary -- Organization: An Organization is the umbrella for one or multiple top-level groups. Organizations are isolated from each other by default meaning that cross-namespace features will only work for namespaces that exist in a single Organization. -- Top-level group: Top-level group is the name given to the topmost group of all other groups. Groups and projects are nested underneath the top-level group. +- Organization: An Organization is the umbrella for one or multiple top-level Groups. Organizations are isolated from each other by default meaning that cross-Namespace features will only work for Namespaces that exist in a single Organization. +- Top-level Group: Top-level Group is the name given to the topmost Group of all other Groups. Groups and Projects are nested underneath the top-level Group. - Cell: A Cell is a set of infrastructure components that contains multiple Organizations. The infrastructure components provided in a Cell are shared among Organizations, but not shared with other Cells. This isolation of infrastructure components means that Cells are independent from each other. -- User: An Organization has many users. Joining an Organization makes someone a user of that Organization. -- Member: Adding a user to a group or project within an Organization makes them a member. Members are always users, but users are not necessarily members of a group or project within an Organization. For instance, a user could just have accepted the invitation to join an Organization, but not be a member of any group or project it contains. -- Non-user: A non-user of an Organization means a user is not part of that specific Organization. +- User: An Organization has many Users. Joining an Organization makes someone a User of that Organization. +- Member: Adding a User to a Group or Project within an Organization makes them a Member. Members are always Users, but Users are not necessarily Members of a Group or Project within an Organization. For instance, a User could just have accepted the invitation to join an Organization, but not be a Member of any Group or Project it contains. +- Non-User: A Non-User of an Organization means a User is not part of that specific Organization. ## Summary Organizations solve the following problems: -1. Enables grouping of top-level groups. For example, the following top-level groups would belong to the Organization `GitLab`: +1. Enables grouping of top-level Groups. For example, the following top-level Groups would belong to the Organization `GitLab`: 1. `https://gitlab.com/gitlab-org/` 1. `https://gitlab.com/gitlab-com/` -1. Allows different Organizations to be isolated. Top-level groups of the same Organization can interact with each other but not with groups in other Organizations, providing clear boundaries for an Organization, similar to a self-managed instance. Isolation should have a positive impact on performance and availability as things like user dashboards can be scoped to Organizations. +1. Allows different Organizations to be isolated. Top-level Groups of the same Organization can interact with each other but not with Groups in other Organizations, providing clear boundaries for an Organization, similar to a self-managed instance. Isolation should have a positive impact on performance and availability as things like User dashboards can be scoped to Organizations. 1. Allows integration with Cells. Isolating Organizations makes it possible to allocate and distribute them across different Cells. -1. Removes the need to define hierarchies. An Organization is a container that could be filled with whatever hierarchy/entity set makes sense (Organization, top-level groups, etc.) +1. Removes the need to define hierarchies. An Organization is a container that could be filled with whatever hierarchy/entity set makes sense (Organization, top-level Groups, etc.) 1. Enables centralized control of user profiles. With an Organization-specific user profile, administrators can control the user's role in a company, enforce user emails, or show a graphical indicator that a user as part of the Organization. An example could be adding a "GitLab employee" stamp on comments. 1. Organizations bring an on-premise-like experience to SaaS (GitLab.com). The Organization admin will have access to instance-equivalent Admin Area settings with most of the configuration controlled on Organization level. @@ -43,19 +43,19 @@ Organizations solve the following problems: The Organization focuses on creating a better experience for Organizations to manage their GitLab experience. By introducing Organizations and [Cells](../cells/index.md) we can improve the reliability, performance and availability of our SaaS Platforms. - Wider audience: Many instance-level features are admin only. We do not want to lock out users of GitLab.com in that way. We want to make administrative capabilities that previously only existed for self-managed users available to our SaaS users as well. This also means we would give users of GitLab.com more independence from GitLab.com admins in the long run. Today, there are actions that self-managed admins can perform that GitLab.com users have to request from GitLab.com admins. -- Improved UX: Inconsistencies between the features available at the project and group levels create navigation and usability issues. Moreover, there isn't a dedicated place for Organization-level features. -- Aggregation: Data from all groups and projects in an Organization can be aggregated. -- An Organization includes settings, data, and features from all groups and projects under the same owner (including personal namespaces). -- Cascading behavior: Organization cascades behavior to all the projects and groups that are owned by the same Organization. It can be decided at the Organization level whether a setting can be overridden or not on the levels beneath. -- Minimal burden on customers: The addition of Organizations should not change existing group and project paths to minimize the impact of URL changes. +- Improved UX: Inconsistencies between the features available at the Project and Group levels create navigation and usability issues. Moreover, there isn't a dedicated place for Organization-level features. +- Aggregation: Data from all Groups and Projects in an Organization can be aggregated. +- An Organization includes settings, data, and features from all Groups and Projects under the same owner (including personal Namespaces). +- Cascading behavior: Organization cascades behavior to all the Projects and Groups that are owned by the same Organization. It can be decided at the Organization level whether a setting can be overridden or not on the levels beneath. +- Minimal burden on customers: The addition of Organizations should not change existing Group and Project paths to minimize the impact of URL changes. ### Non-Goals -Due to urgency of delivering Organizations as a prerequisite for Cells, it is currently not a goal to build Organization functionality on the namespace framework. +Due to urgency of delivering Organizations as a prerequisite for Cells, it is currently not a goal to build Organization functionality on the Namespace framework. ## Proposal -We create Organizations as a new lightweight entity, with just the features and workflows which it requires. We already have much of the functionality present in groups and projects, and groups themselves are essentially already the top-level entity. It is unlikely that we need to add significant features to Organizations outside of some key settings, as top-level groups can continue to serve this purpose at least on SaaS. +We create Organizations as a new lightweight entity, with just the features and workflows which it requires. We already have much of the functionality present in Groups and Projects, and Groups themselves are essentially already the top-level entity. It is unlikely that we need to add significant features to Organizations outside of some key settings, as top-level Groups can continue to serve this purpose at least on SaaS. From an infrastructure perspective, cluster-wide shared data must be both minimal (small in volume) and infrequently written. ```mermaid graph TD @@ -72,14 +72,30 @@ Self-managed instances would set a default Organization. ### Benefits -- No changes to URL's for groups moving under an Organization, which makes moving around top-level groups very easy. -- Low risk rollout strategy, as there is no conversion process for existing top-level groups. +- No changes to URL's for Groups moving under an Organization, which makes moving around top-level Groups very easy. +- Low risk rollout strategy, as there is no conversion process for existing top-level Groups. - Organization becomes the key for identifying what is part of an Organization, which is likely on its own table for performance and clarity. ### Drawbacks -- It is unclear right now how we would avoid continuing to spend effort to build instance (or not Organization) features, in particular much of the reporting. This is not an issue on SaaS as top-level groups already have this capability, however it is a challenge on self-managed. If we introduce a built-in Organization (or just none at all) for self-managed, it seems like we would need to continue to build instance/Organization level reporting features as we would not get that for free along with the work to add to groups. -- Billing may need to be moved from top-level groups to Organization level. +- It is unclear right now how we would avoid continuing to spend effort to build instance (or not Organization) features, in particular much of the reporting. This is not an issue on SaaS as top-level Groups already have this capability, however it is a challenge on self-managed. If we introduce a built-in Organization (or just none at all) for self-managed, it seems like we would need to continue to build instance/Organization level reporting features as we would not get that for free along with the work to add to Groups. +- Billing may need to be moved from top-level Groups to Organization level. + +## Data Exploration + +From an initial [data exploration](https://gitlab.com/gitlab-data/analytics/-/issues/16166#note_1353332877), we retrieved the following information about Users and Organizations: + +- For the users that are connected to an organization the vast majority of them (98%) are only associated with a single organization. This means we expect about 2% of Users to navigate across multiple Organizations. +- The majority of Users (78%) are only Members of a single top-level Group. +- 25% of current top-level Groups can be matched to an organization. + - Most of these top-level Groups (83%) are associated with an organization that has more than one top-level Group. + - Of the organizations with more than one top-level Group the (median) average number of top-level Groups is 3. + - Most top-level Groups that are matched to organizations with more than one top-level Group are assumed to be intended to be combined into a single organization (82%). + - Most top-level Groups that are matched to organizations with more than one top-level Group are using only a single pricing tier (59%). +- Most of the current top-level Groups are set to public visibility (85%). +- Less than 0.5% of top-level Groups share Groups with another top-level Group. However, this means we could potentially break 76,000 links between top-level Groups by introducing the Organization. + +Based on this analysis we expect to see similar behavior when rolling out Organizations. ## Design and Implementation Details @@ -88,40 +104,116 @@ Self-managed instances would set a default Organization. The Organization MVC will contain the following functionality: - Instance setting to allow the creation of multiple Organizations. This will be enabled by default on GitLab.com, and disabled for self-managed GitLab. -- Every instance will have a default organization. Initially, all users will be managed by this default Organization. -- Organization Owner. The creation of an Organization appoints that user as the Organization Owner. Once established, the Organization Owner can appoint other Organization Owners. -- Organization users. A user is managed by one Organization, but can be part of multiple Organizations. Users are able to navigate between the different Organizations they are part of. -- Setup settings. Containing the Organization name, ID, description, README, and avatar. Settings are editable by the Organization Owner. -- Setup flow. Users are able to build an Organization on top of an existing top-level group. New users are able to create an Organization from scratch and to start building top-level groups from there. -- Visibility. Options will be `public` and `private`. A nonuser of a specific Organization will not see private Organizations in the explore section. Visibility is editable by the Organization Owner. +- Every instance will have a default organization. Initially, all Users will be managed by this default Organization. +- Organization Owner. The creation of an Organization appoints that User as the Organization Owner. Once established, the Organization Owner can appoint other Organization Owners. +- Organization Users. A User is managed by one Organization, but can be part of multiple Organizations. Users are able to navigate between the different Organizations they are part of. +- Setup settings. Containing the Organization name, ID, description, and avatar. Settings are editable by the Organization Owner. +- Setup flow. Users are able to build an Organization on top of an existing top-level Group. New Users are able to create an Organization from scratch and to start building top-level Groups from there. +- Visibility. Options will be `public` and `private`. A Non-User of a specific Organization will not see private Organizations in the explore section. Visibility is editable by the Organization Owner. - Organization settings page with the added ability to remove an Organization. Deletion of the default Organization is prevented. -- Groups. This includes the ability to create, edit, and delete groups, as well as a Groups overview that can be accessed by the Organization Owner. -- Projects. This includes the ability to create, edit, and delete projects, as well as a Projects overview that can be accessed by the Organization Owner. +- Groups. This includes the ability to create, edit, and delete Groups, as well as a Groups overview that can be accessed by the Organization Owner. +- Projects. This includes the ability to create, edit, and delete Projects, as well as a Projects overview that can be accessed by the Organization Owner. ### Organization Access #### Organization Users -Organization Users can get access to groups and projects as: +Organization Users can get access to Groups and Projects as: -- A group member: this grants access to the group and all its projects, regardless of their visibility. -- A project member: this grants access to the project, and limited access to parent groups, regardless of their visibility. -- A non-member: this grants access to public and internal groups and projects of that Organization. To access a private group or project in an Organization, a user must become a member. +- A Group Member: this grants access to the Group and all its Projects, regardless of their visibility. +- A Project Member: this grants access to the Project, and limited access to parent Groups, regardless of their visibility. +- A Non-Member: this grants access to public and internal Groups and Projects of that Organization. To access a private Group or Project in an Organization, a User must become a Member. Organization Users can be managed in the following ways: -- As [Enterprise Users](../../../user/enterprise_user/index.md), managed by the Organization. This includes control over their user account and the ability to block the user. -- As Non-Enterprise Users, managed by the Default Organization. Non-Enterprise Users can be removed from an Organization, but the user keeps ownership of their user account. +- As [Enterprise Users](../../../user/enterprise_user/index.md), managed by the Organization. This includes control over their User account and the ability to block the User. +- As Non-Enterprise Users, managed by the Default Organization. Non-Enterprise Users can be removed from an Organization, but the User keeps ownership of their User account. Enterprise Users are only available to Organizations with a Premium or Ultimate subscription. Organizations on the free tier will only be able to host Non-Enterprise Users. +##### How do Users join an Organization? + +Users are visible across all Organizations. This allows Users to move between Organizations. Users can join an Organization by: + +1. Becoming a Member of a Namespace (Group, Subgroup, or Project) contained within an Organization. A User can become a Member of a Namespace by: + + - Being invited by username + - Being invited by email address + - Requesting access. This requires visibility of the Organization and Namespace and must be accepted by the owner of the Namespace. Access cannot be requested to private Groups or Projects. + +1. Becoming an Enterprise Users of an Organization. Bringing Enterprise Users to the Organization level is planned post MVC. + +##### When can Users see an Organization? + +##### What can Users see in an Organization? + +Users can see the things that they have access to in an Organization. For instance, an Organization User would be able to access only the private Groups and Projects that they are a Member of, but could see all public Groups and Projects. Actionable items such as issues, merge requests and the to-do list are seen in the context of the Organization. This means that a User might see 10 merge requests they created in `Organization A`, and 7 in `Organization B`, when in total they have created 17 merge requests across both Organizations. + +##### What is a Billable Member? + +How Billable Members are defined differs between GitLabs two main offerings: + +- Self-managed (SM): [Billable Members are Users who consume seats against the SM License](../../../subscriptions/self_managed/index.md#subscription-seats). Custom roles elevated above the Guest role are consuming seats. +- GitLab.com (SaaS): [Billable Members are Users who are Members of a Namespace (Group or Project) that consume a seat against the SaaS subscription for the top-level Group](../../../subscriptions/gitlab_com/index.md#how-seat-usage-is-determined). Currently, [Users with Minimal Access](../../../user/permissions.md#users-with-minimal-access) and Users without a Group count towards a licensed seat, but [that's changing](https://gitlab.com/gitlab-org/gitlab/-/issues/330663#note_1133361094). + +These differences and how they are calculated and displayed often cause confusion. For both SM and SaaS, we evaluate whether a User consumes a seat against the same core rule set: + +1. They are active users +1. They are not bot users +1. For the Ultimate tier, they are not a Guest + +For (1) this is determined differently per offering, in terms of both what classifies as active and also due to the underlying model that we refer to (User vs Member). +To help demonstrate the various associations used in GitLab relating to Billable Members, here is a relationship diagram: + +```mermaid +graph TD + A[Group] <-.type of.- B[Namespace] + C[Project] -.belongs to.-> A + + E[GroupMember] <-.type of.- D[Member] + G[User] -.has many.-> F + F -.belongs to.-> C + F[ProjectMember] <-.type of.- D + G -.has many.-> E -.belongs to.-> A + + GGL[GroupGroupLink<br/> See note 1] -.belongs_to.->A + PGL[ProjectGroupLink<br/> See note 2] -.belongs_to.->A + PGL -.belongs_to.->C +``` + +GroupGroupLink is the join table between two Group records, indicating that one Group has invited the other. +ProjectGroupLink is the join table between a Group and a Project, indicating the Group has been invited to the Project. + +SaaS has some additional complexity when it comes to the relationships that determine whether or not a User is considered a Billable Member, particularly relating to Group/Project membership that can often lead to confusion. An example of that are Members of a Group that have been invited into another Group or Project and therewith become billable. +There are two charts as the flow is different for each: [SaaS](https://mermaid.live/view#pako:eNqNVl1v2jAU_StXeS5M-3hCU6N2aB3SqKbSPkyAhkkuxFsSs9hpVUX899mxYxsnlOWFcH1877nnfkATJSzFaBLtcvaSZKQS8DhdlWCeijGxXBCygCeOFdzSPCfbHOGrRK9Ho2tlvUkEfcZmo97HXBCBG6AcSGuOj86ZA8No_BP5eHQNMz7HYovV8kuGyR-gOx1I3Qd9Ap-31btrtgORITxIPnBXsfoAGcWKVEn2uj4T4Z6pAPdMdKyX8t2mIG-5ex0LkCnBdO4OOrOhO-O3TDQzrkkSkN9izW-BCCUTCB-8hGU866Bl45FxKJ-GdGiDDYI7SOtOp7o0GW90rA20NYjXQxE6cWSaGr1Q2BnX9hCnIbZWc1reJAly3pisMsJ19vKEFiQHfQw5PmMenwqhPQ5Uxa-DjeAa5IJk_g3t-hvdZ8jFA8vxrpYvccfWHIA6aVmrLtMQj2rvuqPynSZYcnx8PWDzlAuZsay3MfouPJxl1c9hKFCIPedzSBuH5fV2X5FDBrT8Zadk2bbszJur_xsp9UznzZRWmIizV-Njx346X9TbPpwoVqO9xobebUZmF3gse0yk9wA-jDBkflTst2TS-EyMTcrTZmGz7hPrkG8HdChdv1n5TAWmGuxHLmXI9qgTza9aO93-TVfnobAh1M6V0VDtuk7E0w313tMUy3Swc_Tyll9VLUwMPcFxUJGBNdKYTTTwY-ByesC_qusx1Yk0bXtao9kk8Snzj8eLsX0lwqV2ujnUE5Bw7FT4g7QbQGM-4YWoXPRZ2C7BnT4TXZPSiAHFUIP3nVhGbiN3G9-OyKWsTvpSS60yMYZA5U_HtyQzdy7p7GCBon65OyXNWJwT9DSNMwF7YB3Xly1o--gqKrAqCE3l359GHa4iuQ8KXEUT-ZrijtS5WEWr8iihpBZs8Vom0WRHco5XUX1IZd9NKZETUxjr8R82ROYl) and [SM](https://mermaid.live/view#pako:eNqFk1FvwiAQx7_KhefVD-CDZo2JNdmcWe3DYpeI7alsLRgKLob0u48qtqxRx9Plz4-7-3NgSCZyJEOyLcRPtqdSwXKScnBLVyhXswrUHiGxMYSsKOimwPHnXwiCYNQAsaIKzXOm2BFh3ShrOGvjujvQghAMPrAaBCOITKRLyu9Rc9FAc6Gu9VPegVELLEKzkOILMwWhUH6yRdhCcWJilEeWXSz5VJzcqrWycWvc830rOmdwnmZ8KoU-vEnXU6-bf6noPmResdzYWxdboHDeAiHBbfqOuqifonX6Ym-CV7g8HfAhfZ0U2-2xUu-iwKm2wdg4BRoJWAUXufZH5JnqH-8ye42YpFCsbGbvRN-Tx7UmunfxqFCfvZfTNeS9AfJESpQlZbn9K6Y5lxL7KUpMydCGOZXfKUl5bTmqlYhPPCNDJTU-EX3IrZEJoztJy4tY_wJJwxFj). + +##### How can Users switch between different Organizations? + +Users can utilize a [context switcher](https://gitlab.com/gitlab-org/gitlab/-/issues/411637). This feature allows easy navigation and access to different Organizations' content and settings. By clicking on the context switcher and selecting a specific Organization from the provided list, Users can seamlessly transition their view and permissions, enabling them to interact with the resources and functionalities of the chosen Organization. + +##### What happens when a User is deleted? + +We've identified three different scenarios where a User can be removed from an Organization: + +1. Removal: The User is removed from the organization_users table. This is similar to the User leaving a company, but the User can join the Organization again after access approval. +1. Banning: The User is banned. This can happen in case of misconduct but the User cannot be added again to the Organization until they are unbanned. In this case, we keep the organization_users entry and change the permission to none. +1. Deleting: The User is deleted. We assign everything the User has authored to the Ghost User and delete the entry from the organization_users table. + +As part of the Organization MVC, Organization Owners can remove Organization Users. This means that the User's membership entries are deleted from all Groups and Projects that are contained within the Organization. In addition, the User entry is removed from the `organization_users` table. + +Actions such as banning and deleting a User will be added to the Organization at a later point. + #### Organization Non-Users -Non-users are external to the Organization and can only access the public resources of an Organization, such as public projects. +Non-Users are external to the Organization and can only access the public resources of an Organization, such as public Projects. ### Routing -Today only users, projects, namespaces and container images are considered routable entities which require global uniqueness on `https://gitlab.com/<path>/-/`. Initially, Organization routes will be [unscoped](../../../development/routing.md). Organizations will follow the path `https://gitlab.com/-/organizations/org-name/` as one of the design goals is that the addition of Organizations should not change existing group and project paths. +Today only Users, Projects, Namespaces and container images are considered routable entities which require global uniqueness on `https://gitlab.com/<path>/-/`. Initially, Organization routes will be [unscoped](../../../development/routing.md). Organizations will follow the path `https://gitlab.com/-/organizations/org-name/` as one of the design goals is that the addition of Organizations should not change existing Group and Project paths. + +### Impact of the Organization on Other Features + +We want a minimal amount of infrequently written tables in the shared database. If we have high write volume or large amounts of data in the shared database then this can become a single bottleneck for scaling and we lose the horizontal scalability objective of Cells. ## Iteration Plan @@ -129,48 +221,58 @@ The following iteration plan outlines how we intend to arrive at the Organizatio ### Iteration 1: Organization Prototype (FY24Q2) -In iteration 1, we introduce the concept of an Organization as a way to group top-level groups together. Support for Organizations does not require any [Cells](../cells/index.md) work, but having them will make all subsequent iterations of Cells simpler. The goal of iteration 1 will be to generate a prototype that can be used by GitLab teams to test moving functionality to the Organization. It contains everything that is necessary to move an Organization to a Cell: +In iteration 1, we introduce the concept of an Organization as a way to Group top-level Groups together. Support for Organizations does not require any [Cells](../cells/index.md) work, but having them will make all subsequent iterations of Cells simpler. The goal of iteration 1 will be to generate a prototype that can be used by GitLab teams to test moving functionality to the Organization. It contains everything that is necessary to move an Organization to a Cell: - The Organization can be named, has an ID and an avatar. -- Only non-enterprise user can be part of an Organization. -- A user can be part of multiple Organizations. +- Only a Non-Enterprise User can be part of an Organization. +- A User can be part of multiple Organizations. - A single Organization Owner can be assigned. - Groups can be created in an Organization. Groups are listed in the Groups overview. - Projects can be created in a Group. Projects are listed in the Projects overview. ### Iteration 2: Organization MVC Experiment (FY24Q3) -In iteration 2, an Organization MVC Experiment will be released. We will test the functionality with a select set of customers and improve the MVC based on these learnings. Users will be able to build an Organization on top of their existing top-level group. +In iteration 2, an Organization MVC Experiment will be released. We will test the functionality with a select set of customers and improve the MVC based on these learnings. Users will be able to build an Organization on top of their existing top-level Group. -- The Organization has a description and a README. +- The Organization has a description. +- Organizations can be deleted. +- Users can navigate between the different Organizations they are part of. ### Iteration 3: Organization MVC Beta (FY24Q4) In iteration 3, the Organization MVC Beta will be released. - Multiple Organization Owners can be assigned. -- Enterprise users can be added to an Organization. +- Organization Owners can change the visibility of an organization between `public` and `private`. A Non-User of a specific Organization will not see private Organizations in the explore section. ### Iteration 4: Organization MVC GA (FY25Q1) +In iteration 4, the Organization MVC will be rolled out. + ### Post-MVC Iterations After the initial rollout of Organizations, the following functionality will be added to address customer needs relating to their implementation of GitLab: 1. Internal visibility will be made available on Organizations that are part of GitLab.com. -1. Move billing from top-level group to Organization. +1. Enterprise Users will be made available at the Organization level. +1. Organizations are able to ban and delete Users. +1. Projects can be created from the Organization-level Projects overview. +1. Groups can be created from the Organization-level Groups overview. +1. Move billing from top-level Group to Organization. 1. Audit events at the Organization level. -1. Set merge request approval rules at the Organization level and cascade to all groups and projects. +1. Set merge request approval rules at the Organization level and cascade to all Groups and Projects. 1. Security policies at the Organization level. -1. Vulnerability reports at the Organization level. +1. Vulnerability Report and Dependency List at the Organization level. 1. Cascading Organization setting to enforce security scans. 1. Scan result policies at the Organization level. 1. Compliance frameworks. 1. [Support the agent for Kubernetes sharing at the Organization level](https://gitlab.com/gitlab-org/gitlab/-/issues/382731). +## Organization Rollout + ## Alternative Solutions -An alternative approach to building Organizations is to convert top-level groups into Organizations. The main advantage of this approach is that features could be built on top of the namespace framework and therewith leverage functionality that is already available at the group level. We would avoid building the same feature multiple times. However, Organizations have been identified as a critical driver of Cells. Due to the urgency of delivering Cells, we decided to opt for the quickest and most straightforward solution to deliver an Organization, which is the lightweight design described above. More details on comparing the two Organization proposals can be found [here](https://gitlab.com/gitlab-org/tenant-scale-group/group-tasks/-/issues/56). +An alternative approach to building Organizations is to convert top-level Groups into Organizations. The main advantage of this approach is that features could be built on top of the Namespace framework and therewith leverage functionality that is already available at the Group level. We would avoid building the same feature multiple times. However, Organizations have been identified as a critical driver of Cells. Due to the urgency of delivering Cells, we decided to opt for the quickest and most straightforward solution to deliver an Organization, which is the lightweight design described above. More details on comparing the two Organization proposals can be found [here](https://gitlab.com/gitlab-org/tenant-scale-group/group-tasks/-/issues/56). ## Decision Log diff --git a/doc/architecture/blueprints/rate_limiting/index.md b/doc/architecture/blueprints/rate_limiting/index.md index db6bb85d839..f42a70aa97a 100644 --- a/doc/architecture/blueprints/rate_limiting/index.md +++ b/doc/architecture/blueprints/rate_limiting/index.md @@ -108,7 +108,7 @@ quota and by a policy. - _Example:_ maximum artifact upload size of 1 GB - **Quota:** A global constraint in application usage that is aggregated across an entire namespace over the duration of their billing cycle. - - _Example:_ 400 units of compute per namespace per month + - _Example:_ 400 compute minutes per namespace per month - _Example:_ 10 GB transfer per namespace per month - **Policy:** A representation of business logic that is decoupled from application code. Decoupled policy definitions allow logic to be shared across multiple services diff --git a/doc/architecture/blueprints/remote_development/index.md b/doc/architecture/blueprints/remote_development/index.md index 16b71840f9e..ce55f23f828 100644 --- a/doc/architecture/blueprints/remote_development/index.md +++ b/doc/architecture/blueprints/remote_development/index.md @@ -215,7 +215,7 @@ end note note top of "Load Balancer IP" For local development, it includes all local loopback interfaces - e.g. 127.0.0.1, 172.16.123.1, 192.168.0.1, etc. + for example, 127.0.0.1, 172.16.123.1, 192.168.0.1, etc. end note @enduml @@ -439,7 +439,7 @@ Stopped -up-> Starting : status=Starting Terminated: Workspace has been deleted -Failed: Workspace is not ready due to\nvarious reasons(e.g. crashing container) +Failed: Workspace is not ready due to\nvarious reasons(for example, crashing container) Failed -up-> Starting : status=Starting\n(container\nnot crashing) Failed -right-> Stopped : status=Stopped Failed -down-> Terminated : status=Terminated diff --git a/doc/architecture/blueprints/repository_backups/index.md b/doc/architecture/blueprints/repository_backups/index.md new file mode 100644 index 00000000000..afd86e4979c --- /dev/null +++ b/doc/architecture/blueprints/repository_backups/index.md @@ -0,0 +1,268 @@ +--- +status: proposed +creation-date: "2023-04-26" +authors: [ "@proglottis" ] +coach: "@DylanGriffith" +approvers: [] +owning-stage: "~devops::systems" +participating-stages: [] +--- + +<!-- Blueprints often contain forward-looking statements --> +<!-- vale gitlab.FutureTense = NO --> + +# Repository Backups + +<!-- For long pages, consider creating a table of contents. The `[_TOC_]` +function is not supported on docs.gitlab.com. --> + +## Summary + +This proposal seeks to provide an out-of-a-box repository backup solution to +GitLab that gives more opportunities to apply Gitaly specific optimisations. It +will do this by moving repository backups out of `backup.rake` into a +coordination worker that enumerates repositories and makes per-repository +decisions to trigger repository backups that are streamed directly from Gitaly +to object-storage. + +The advantages of this approach are: + +- The backups are only transferred once, from the Gitaly hosting the physical + repository to object-storage. +- Smarter decisions can be made by leveraging specific repository access + patterns. +- Distributes backup and restore load. +- Since the entire process is run within Gitaly existing monitoring can be + used. +- Provides architecture for future WAL archiving and other optimisations. + +This should relieve the major pain points of the existing two strategies: + +- `backup.rake` - Repository backups are streamed from outside of Gitaly using + RPCs and stored in a single large tar file. Due to the amount of data + transferred these backups are limited to small installations. +- Snapshots - Cloud providers allow taking physical storage snapshots. These + are not an out-of-a-box solution as they are specific to the cloud provider. + +## Motivation + +### Goals + +- Improve time to create and restore repository backups. +- Improve monitoring of repository backups. + +### Non-Goals + +- Improving filesystem based snapshots. + +### Filesystem based Snapshots + +Snapshots rely on cloud platforms to be able to take physical snapshots of the +disks that Gitaly and Praefect use to store data. While never officially +recommended this strategy tends to be used once creating or restoring backups +using `backup.rake` takes too long. + +Gitaly and Git use lock files and fsync in order to prevent repository +corruption from concurrent processes and partial writes from a crash. This +generally means that if a file is written, then it will be valid. However, +because Git repositories are composed of many files and many write operations +may be taking place, it would be impossible to schedule a snapshot while no +file operations are ongoing. This means the consistency of a snapshot cannot be +guaranteed and restoring from a snapshot backup may require manual +intervention. + +[WAL](https://gitlab.com/groups/gitlab-org/-/epics/8911) may improve crash +resistance and so improve automatic recovery from snapshots, but each +repository will likely still require a majority of voting replicas in sync. + +Since each node in a Gitaly Cluster is not homogeneous, depending on +replication factor, in order to create a complete snapshot backup all nodes +would need to have snapshots taken. This means that snapshot backups have a lot +of repository data duplication. + +Snapshots are heavily dependent on the cloud provider and so they would not +provide an out-of-a-box experience. + +### Downtime + +An ideal repository backup solution would allow both backup and restore +operations to be done online. Specifically we would not want to shutdown or +pause writes to ensure that each node/repository is consistent. + +### Consistency + +Consistency in repository backups means: + +- That the Git repositories are valid after restore. There are no partially + applied operations. +- That all repositories in a cluster are healthy after restore, or are made + healthy automatically. + +Backups without consistency may result in data-loss or require manual +intervention on restore. + +Both types of consistency are difficult to achieve using snapshots as this +requires that snapshots of the filesystems on multiple hosts are taken +synchronously and without repositories on any of those hosts currently being +mutated. + +### Distribute Work + +We want to distribute the backup/restore work such that it isn't bottlenecked +on the machine running `backup.rake`, a single Gitaly node, or a single network +connection. + +On backup, `backup.rake` aggregates all repository backups onto its local +filesystem. This means that all repository data needs to be streamed from +Gitaly (possibly via Praefect) to where the Rake task is being run. If this is +CNG then it also requires a large volume on Kubernetes. The resulting backup +tar file then gets transferred to object storage. A similar process happens on +restore, the entire tar file needs to be downloaded and extracted on the local +filesystem, even for a partial restore when restoring a subset of repositories. +Effectively all repository data gets transferred, in full, multiple times +between multiple hosts. + +If each Gitaly could directly upload backups it would mean only transferring +repository data a single time, reducing the number of hosts and so the amount +of data transferred over all. + +### Gitaly Controlled + +Gitaly is looking to become self-contained and so should own its backups. + +`backup.rake` currently determines which repositories to backup and where those +backups are stored. This restricts the kind of optimisations that Gitaly could +apply and adds development/testing complexity. + +### Monitoring + +`backup.rake` is run in a variety of different environments. Historically +backups from Gitaly's perspective are a series of disconnected RPC calls. This +has resulted in backups having almost zero monitoring. Ideally the process +would run within Gitaly such that the process could be monitored using existing +metrics and log scraping. + +### Automatic Backups + +When `backup.rake` is set up on cron it can be difficult to tell if it has been +running successfully, if it is still running, how long it took, and how much +space it has taken. It is difficult to ensure that cron always has access to +the previous backup to allow for incremental backups or to determine if +updating the backup is required at all. + +Having a coordination process running continuously will allow moving from a +single-shot backup strategy to one where each repository determines its own +backup schedule based on usage patterns and priority. This way each repository +should be able to have a reasonably up-to-date backup without adding excess +load to any Gitaly node. + +### Updated Repositories Only + +`backup.rake` packages all repository backups into a tar file and generally has +no access to the previous backup. This makes it difficult to determine if the +repository has changed since last backup. + +Having access to previous backups on object-storage would mean that Gitaly +could more easily determine if a backup needs to be taken at all. This allows +us to waste less time backing up repositories that are no longer being +modified. + +### Point-in-time Restores + +There should be a mechanism by which a set of repositories can be restored to a +specific point in time. The identifier (backup ID) used should be able to be +determined by an admin and apply to all repositories. + +### WAL (write ahead log) + +We want to be able to provide infrastructure to allow continuous archiving of +the WAL. This means providing a central place to stream the archives to and +being able to match any full backup to a place in the log such that +repositories can be restored from the full backup, and the WAL applied up to a +specific point in time. + +### WORM + +Any Gitaly accessible storage should be WORM (write once, read many) in order +to prevent existing backups being modified in the case an attacker gains access +to a nodes object-storage credentials. + +[The pointer layout](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/gitaly-backup.md#pointer-layout) +currently used by repository backups relies on being able to overwrite the +pointer files, and as such would not be suitable for use on a WORM file store. + +WORM is likely object-storage provider specific: + +- [AWS object lock](https://aws.amazon.com/blogs/storage/protecting-data-with-amazon-s3-object-lock/) +- [Google Cloud WORM retention policy](https://cloud.google.com/blog/products/storage-data-transfer/protecting-cloud-storage-with-worm-key-management-and-more-updates). +- [MinIO object lock](https://min.io/docs/minio/linux/administration/object-management/object-retention.html) + +### `bundle-uri` + +Having direct access backup data may open the door for clone/fetch transfer +optimisations using bundle-uri. This allows us to point Git clients directly to +a bundle file instead of transferring packs from the repository itself. The +bulk repository transfer can then be faster and is offloaded to a plain http +server, rather than the Gitaly servers. + +## Proposal + +The proposal is broken down into an initial MVP and per-repository coordinator. + +### MVP + +The goal of the MVP is to validate that moving backup processing server-side +will improve the worst case, total-loss, scenario. That is, reduce the total +time to create and restore a full backup. + +The MVP will introduce backup and restore repository RPCs. There will be no +coordination worker. The RPCs will stream a backup directly from the +called Gitaly node to object storage. These RPCs will be called from +`backup.rake` via the `gitaly-backup` tool. `backup.rake` will no longer +package repository backups into the backup archive. + +This work is already underway, tracked by the [Server-side Backups MVP epic](https://gitlab.com/groups/gitlab-org/-/epics/10077). + +### Per-Repository Coordinator + +Instead of taking a backup of all repositories at once via `backup.rake`, a +backup coordination worker will be created. This worker will periodically +enumerate all repositories to decide if a backup needs to be taken. These +decisions could be determined by usage patterns or priority of the repository. + +When restoring, since each repository will have a different backup state, a +timestamp will be provided by the user. This timestamp will be used to +determine which backup to restore for each repository. Once WAL archiving is +implemented, the WAL could then be replayed up to the given timestamp. + +This wider effort is tracked in the [Server-side Backups epic](https://gitlab.com/groups/gitlab-org/-/epics/10826). + +## Design and implementation details + +### MVP + +There will be a pair of RPCs `BackupRepository` and `RestoreRepository`. These +RPCs will synchronously create/restore backups directly onto object storage. +`backup.rake` will continue to use `gitaly-backup` with a new `--server-side` +flag. Each Gitaly will need a backup configuration to specify the +object-storage service to use. + +Initially the structure of the backups in object-storage will be the same as +the existing [pointer layout](https://gitlab.com/gitlab-org/gitaly/-/blob/master/doc/gitaly-backup.md#pointer-layout). + +For MVP the backup ID must match an exact backup ID on object-storage. + +The configuration of object-storage will be controlled by a new config +`config.backup.go_cloud_url`. The [Go Cloud Development Kit](https://gocloud.dev) +tries to use a provider specific way to configure authentication. This can be +inferred from the VM or from environment variables. +See [Supported Storage Services](https://gocloud.dev/howto/blob/#services). + +## Alternative Solutions + +<!-- +It might be a good idea to include a list of alternative solutions or paths considered, although it is not required. Include pros and cons for +each alternative solution/path. + +"Do nothing" and its pros and cons could be included in the list too. +--> diff --git a/doc/architecture/blueprints/runner_admission_controller/index.md b/doc/architecture/blueprints/runner_admission_controller/index.md new file mode 100644 index 00000000000..d73ffb21ef3 --- /dev/null +++ b/doc/architecture/blueprints/runner_admission_controller/index.md @@ -0,0 +1,241 @@ +--- +status: proposed +creation-date: "2023-03-07" +authors: [ "@ajwalker" ] +coach: [ "@ayufan" ] +approvers: [ "@DarrenEastman", "@engineering-manager" ] +owning-stage: "~devops::<stage>" +participating-stages: [] +--- + +# GitLab Runner Admissions Controller + +The GitLab `admission controller` (inspired by the [Kubernetes admission controller concept](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/)) is a proposed technical solution to intercept jobs before they're persisted or added to the build queue for execution. + +An admission controller can be registered to the GitLab instance and receive a payload containing jobs to be created. Admission controllers can be _mutating_, _validating_, or both. + +- When _mutating_, mutatable job information can be modified and sent back to the GitLab instance. Jobs can be modified to conform to organizational policy, security requirements, or have, for example, their tag list modified so that they're routed to specific runners. +- When _validating_, a job can be denied execution. + +## Motivation + +To comply with the segregation of duties, organizational policy, or security requirements, customers in financial services, the US federal government market segment, or other highly regulated industries must ensure that only authorized users can use runners associated with particular CI job environments. + +In this context, using the term environments is not equivalent to the definition of the environment used in the GitLab CI environments and deployments documentation. Using the definition from the [SLSA guide](https://slsa.dev/spec/v0.1/terminology), an environment is the "machine, container, VM, or similar in which the job runs." + +An additional requirement comes from the Remote Computing Enablement (RCE) group at [Lawrence Livermore National Laboratory](https://hpc.llnl.gov/). In this example, users must have a user ID on the target Runner CI build environment for the CI job to run. To simplify administration across the entire user base, RCE needs to be able to associate a Runner with a GitLab user entity. + +### Current GitLab CI job handling mechanism + +Before going further, it is helpful to level-set the current job handling mechanism in GitLab CI and GitLab Runners. + +- First, a runner associated with a GitLab instance continuously queries the GitLab instance API to check if there is a new job that it could run. +- With every push to a project repository on GitLab with a `.gitlab-ci.yml` file present, the CI service present on the GitLab instance catches the event and triggers a new CI job. +- The CI job enters a pending state in the queue until a Runner requests a job from the instance. +- On the request from a runner to the API for a job, the database is queried to verify that the job parameters matches that of the runner. In other words, when runners poll a GitLab instance for a job to execute they're assigned a job if it matches the specified criteria. +- If the job matches the runner in question, then the GitLab instance connects the job to the runner and changes the job state to running. In other words, GitLab connects the `job` object with the `Runner` object. +- A runner can be configured to run un-tagged jobs. Tags are the primary mechanism used today to enable customers to have some control of which Runners run certain types of jobs. +- So while runners are scoped to the instance, group, or project, there are no additional access control mechanisms today that can easily be expanded on to deny access to a runner based on a user or group identifier. + +The current CI jobs queue logic is as follows. **Note - in the code ww still use the very old `build` naming construct, but we've migrated from `build` to `job` in the product and documentation. + +```ruby +jobs = + if runner.instance_type? + jobs_for_shared_runner + elsif runner.group_type? + jobs_for_group_runner + else + jobs_for_project_runner + end + +# select only jobs that have tags known to the runner +jobs = jobs.matches_tag_ids(runner.tags.ids) + +# select builds that have at least one tag if required +unless runner.run_untagged? + jobs = jobs.with_any_tags +end + +``` + +## Goals + +- Implement an initial solution that provides an easy-to-configure and use mechanism to `allow`, `deny` or `redirect` CI job execution on a specific runner entity based on some basic job details (like user, group or project membership). + +## Non-Goals + +- A re-design of the CI job queueing mechanism is not in the scope of this blueprint. + +## Proposal + +Implement a mechanism, `admission controllers`, to intercept CI jobs, allowing them to either mutate jobs, validate them or do both. An admission controller is a mutating webhook that can modify the CI job or reject the job according to a policy. The webhook is called before the job is inserted into the CI jobs queue. + +### Guiding principles + +- The webhook payload schema will be part of our public facing APIs. +- We must maintain backwards compatibility when extending the webhook payload. +- Controllers should be idempotent. + +### How will the admissions controller work? + +**Scenario 1**: I want to deny access to a certain runner. + +1. Configure an admissions controller to only accept jobs from specific projects. +1. When a job is created the `project information` (`project_id`, `job_id`, `api_token`) will be used to query GitLab for specific details. +1. If the `project information` matches the allow list, then the job payload is not modified and the job is able to run on the target runner. +1. If the `project information` does not match the allow list, then the job payload is not modified and the job is dropped. +1. The job tags are not changed. +1. Admission controller may optionally send back an arbitrary text description of why a decline decision was made. + +**Scenario 2**: Large runner fleet with using a common configuration and tags. + +Each runner has a tag such as `zone_a`, `zone_b`. In this scenario the customer does not know where a specific job can run as some users have access to `zone_a`, and some to `zone_b`. The customer does not want to fail a job that should run on `zone_a`, but instead redirect a job if it is not correctly tagged to run in `zone_a.` + +1. Configure an admissions controller to mutate jobs based on `user_id`. +1. When a job is created the `project information` (`project_id`, `job_id`, `api_token`) will be used to query GitLab for specific details. +1. If the `user_id` matches then the admissions controller modifies the job tag list. `zone_a` is added to the tag list as the controller has detected that the user triggering the job should have their jobs run IN `zone_a`. + +### MVC + +#### Admission controller + +1. A single admission controller can be registered at the instance level only. +1. The admission controller must respond within 30 seconds. +1. The admission controller will receive an array of individual jobs. These jobs may or may not be related to each other. The response must contain only responses to the jobs made as part of the request. + +#### Job Lifecycle + +1. The lifecycle of a job will be updated to include a new `validating` state. + + ```mermaid + stateDiagram-v2 + created --> validating + state validating { + [*] --> accept + [*] --> reject + } + reject --> failed + accept --> pending + pending --> running: picked by runner + running --> executed + state executed { + [*] --> failed + [*] --> success + [*] --> canceled + } + executed --> created: retry + ``` + +1. When the state is `validating`, the mutating webhook payload is sent to the admission controller. +1. For jobs where the webhook times out (30 seconds) their status should be set as though the admission was denied. This should +be rare in typical circumstances. +1. Jobs with denied admission can be retried. Retried jobs will be resent to the admission controller along with any mutations that they received previously. +1. [`allow_failure`](../../../ci/yaml/index.md#allow_failure) should be updated to support jobs that fail on denied admissions, for example: + + ```yaml + job: + script: + - echo "I will fail admission" + allow_failure: + on_denied_admission: true + ``` + +1. The UI should be updated to display the reason for any job mutations (if provided). +1. A table in the database should be created to store the mutations. Any changes that were made, like tags, should be persisted and attached to `ci_builds` with `acts_as_taggable :admission_tags`. + +#### Payload + +1. The payload is comprised of individual job entries consisting of: + - Job ID. + - [Predefined variables](../../../ci/variables/predefined_variables.md) + - Job tag list. +1. The response payload is comprised of individual job entries consisting of: + - Job ID. + - Admission state: `accepted` or `denied`. + - Mutations: Only `tags` is supported for now. The tags provided replaces the original tag list. + - Reason: A controller can provide a reason for admission and mutation. + +##### Example request + +```json +[ + { + "id": 123, + "variables": { + # predefined variables: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html + "CI_PROJECT_ID": 123, + "CI_PROJECT_NAME": "something", + "GITLAB_USER_ID": 98123, + ... + }, + "tags": [ "docker", "windows" ] + }, + { + "id": 245, + "variables": { + "CI_PROJECT_ID": 245, + "CI_PROJECT_NAME": "foobar", + "GITLAB_USER_ID": 98123, + ... + }, + "tags": [ "linux", "eu-west" ] + }, + { + "id": 666, + "variables": { + "CI_PROJECT_ID": 666, + "CI_PROJECT_NAME": "do-bad-things", + "GITLAB_USER_ID": 98123, + ... + }, + "tags": [ "secure-runner" ] + }, +] +``` + +##### Example response + +```json +[ + { + "id": 123, + "admission": "accepted", + "reason": "it's always-allow-day-wednesday" + }, + { + "id": 245, + "admission": "accepted", + "mutations": { + "tags": [ "linux", "us-west" ] + }, + "reason": "user is US employee: retagged region" + }, + { + "id": 666, + "admission": "rejected", + "reason": "you have no power here" + }, +] +``` + +### MVC + + +1. Multiple admissions controllers on groups and project levels. +1. Passing job definition through a chain of the controllers (starting on the project, through all defined group controllers up to the instance controller). +1. Each level gets the definition modified by the previous controller in the chain and makes decisions based on the current state. +1. Modification reasons, if reported by multiple controllers, are concatenated. +1. Usage of the admission controller is optional, so we can have a chain containing project+instance, project+group+parent group+instance, project+group, group+instance, etc + +### Implementation Details + +1. [placeholder for steps required to code the admissions controller MVC] + +## Technical issues to resolve + +| issue | resolution| +| ------ | ------ | +|We may have conflicting tag-sets as mutating controller can make it possible to define AND, OR and NONE logical definition of tags. This can get quite complex quickly. | | +|Rule definition for the queue web hook| +|What data to send to the admissions controller? Is it a subset or all of the [predefined variables](../../../ci/variables/predefined_variables.md)?| +|Is the `queueing web hook` able to run at GitLab.com scale? On GitLab.com we would trigger millions of webhooks per second and the concern is that would overload Sidekiq or be used to abuse the system. diff --git a/doc/architecture/blueprints/runner_scaling/index.md b/doc/architecture/blueprints/runner_scaling/index.md index de1203843aa..a6df58aa405 100644 --- a/doc/architecture/blueprints/runner_scaling/index.md +++ b/doc/architecture/blueprints/runner_scaling/index.md @@ -232,12 +232,12 @@ coupled in the current implementation so we will break them out here to consider them each separately. - **Virtual Machine (VM) shape**. The underlying provider of a VM requires configuration to - know what kind of machine to create. E.g. Cores, memory, failure domain, + know what kind of machine to create. For example, Cores, memory, failure domain, etc... This information is very provider specific. - **VM lifecycle management**. Multiple machines will be created and a system must keep track of which machines belong to this executor. Typically a cloud provider will have a way to manage a set of homogeneous machines. - E.g. GCE Instance Group. The basic operations are increase, decrease and + For example, GCE Instance Group. The basic operations are increase, decrease and usually delete a specific machine. - **VM autoscaling**. In addition to low-level lifecycle management, job-aware capacity decisions must be made to the set of machines to provide @@ -255,7 +255,7 @@ See also Glossary below. #### Current state The current architecture has several points of coupling between concerns. -Coupling reduces opportunities for abstraction (e.g. community supported +Coupling reduces opportunities for abstraction (for example, community supported plugins) and increases complexity, making the code harder to understand, test, maintain and extend. diff --git a/doc/architecture/blueprints/runner_tokens/index.md b/doc/architecture/blueprints/runner_tokens/index.md index c83586f851a..29d7c05c553 100644 --- a/doc/architecture/blueprints/runner_tokens/index.md +++ b/doc/architecture/blueprints/runner_tokens/index.md @@ -238,7 +238,7 @@ The new workflow looks as follows: 1. Creates a new runner in the `ci_runners` table (and corresponding `glrt-` prefixed authentication token); 1. Presents the user with instructions on how to configure this new runner on a machine, - with possibilities for different supported deployment scenarios (e.g. shell, `docker-compose`, Helm chart, etc.) + with possibilities for different supported deployment scenarios (for example, shell, `docker-compose`, Helm chart, etc.) This information contains a token which is available to the user only once, and the UI makes it clear to the user that the value shall not be shown again, as registering the same runner multiple times is discouraged (though not impossible). @@ -319,7 +319,7 @@ The respective `CiRunner` fields must return the values for the `ci_runner_machi #### Stale runner cleanup The functionality to -[clean up stale runners](../../../ci/runners/configure_runners.md#clean-up-stale-runners) needs +[clean up stale runners](../../../ci/runners/runners_scope.md#clean-up-stale-group-runners) needs to be adapted to clean up `ci_runner_machines` records instead of `ci_runners` records. At some point after the removal of the registration token support, we'll want to create a background diff --git a/doc/architecture/blueprints/work_items/index.md b/doc/architecture/blueprints/work_items/index.md index f067d9fab52..9924b0db9f4 100644 --- a/doc/architecture/blueprints/work_items/index.md +++ b/doc/architecture/blueprints/work_items/index.md @@ -30,11 +30,17 @@ Base type for issue, requirement, test case, incident and task (this list is pla A set of predefined types for different categories of work items. Currently, the available types are: -- Issue -- Incident -- Test case -- Requirement -- Task +- [Incident](/ee/operations/incident_management/incidents.md) +- [Test case](/ee/ci/test_cases/index.md) +- [Requirement](/ee/user/project/requirements/index.md) +- [Task](/ee/user/tasks.md) +- [OKRs](/ee/user/okrs.md) + +Work is underway to convert existing objects to Work Item Types or add new ones: + +- [Issue](https://gitlab.com/groups/gitlab-org/-/epics/9584) +- [Epic](https://gitlab.com/groups/gitlab-org/-/epics/9290) +- [Ticket](https://gitlab.com/groups/gitlab-org/-/epics/10419) #### Work Item properties @@ -58,7 +64,7 @@ All Work Item types share the same pool of predefined widgets and are customized ### Work Item widget types (updating) | widget type | feature flag | -|---|---|---| +|---|---| | assignees | | | description | | | hierarchy | | @@ -68,7 +74,7 @@ All Work Item types share the same pool of predefined widgets and are customized | start and due date | | | status\* | | | weight | | -| [notes](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) | work_items_mvc | +| [notes](https://gitlab.com/gitlab-org/gitlab/-/issues/378949) | | \* status is not currently a widget, but a part of the root work item, similar to title @@ -119,7 +125,7 @@ Work Items main goal is to enhance the planning toolset to become the most popul ### Scalability -Currently, different entities like issues, epics, merge requests etc share many similar features but these features are implemented separately for every entity type. This makes implementing new features or refactoring existing ones problematic: for example, if we plan to add new feature to issues and incidents, we would need to implement it separately on issue and incident types respectively. With work items, any new feature is implemented via widgets for all existing types which makes the architecture more scalable. +Currently, different entities like issues, epics, merge requests etc share many similar features but these features are implemented separately for every entity type. This makes implementing new features or refactoring existing ones problematic: for example, if we plan to add new feature to issues and incidents, we would need to implement it separately on issue and incident types. With work items, any new feature is implemented via widgets for all existing types which makes the architecture more scalable. ### Flexibility diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 1822e610dad..d14c9563642 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -167,7 +167,7 @@ job1: The order of caches extraction is: 1. Retrieval attempt for `cache:key` -1. Retrieval attemps for each entry in order in `fallback_keys` +1. Retrieval attempts for each entry in order in `fallback_keys` 1. Retrieval attempt for the global fallback key in `CACHE_FALLBACK_KEY` The cache extraction process stops after the first successful cache is retrieved. @@ -297,25 +297,20 @@ In this example, the job's cache policy is: ### Cache Node.js dependencies If your project uses [npm](https://www.npmjs.com/) to install Node.js -dependencies, the following example defines `cache` globally so that all jobs inherit it. +dependencies, the following example defines a default `cache` so that all jobs inherit it. By default, npm stores cache data in the home folder (`~/.npm`). However, you [can't cache things outside of the project directory](../yaml/index.md#cachepaths). Instead, tell npm to use `./.npm`, and cache it per-branch: ```yaml -# -# https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Nodejs.gitlab-ci.yml -# -image: node:latest - -# Cache modules in between jobs -cache: - key: $CI_COMMIT_REF_SLUG - paths: - - .npm/ - -before_script: - - npm ci --cache .npm --prefer-offline +default: + image: node:latest + cache: # Cache modules in between jobs + key: $CI_COMMIT_REF_SLUG + paths: + - .npm/ + before_script: + - npm ci --cache .npm --prefer-offline test_async: script: @@ -328,13 +323,13 @@ You can use [`cache:key:files`](../yaml/index.md#cachekeyfiles) to compute the c key from a lock file like `package-lock.json` or `yarn.lock`, and reuse it in many jobs. ```yaml -# Cache modules using lock file -cache: - key: - files: - - package-lock.json - paths: - - .npm/ +default: + cache: # Cache modules using lock file + key: + files: + - package-lock.json + paths: + - .npm/ ``` If you're using [Yarn](https://yarnpkg.com/), you can use [`yarn-offline-mirror`](https://classic.yarnpkg.com/blog/2016/11/24/offline-mirror/) @@ -358,26 +353,21 @@ job: ### Cache PHP dependencies If your project uses [Composer](https://getcomposer.org/) to install -PHP dependencies, the following example defines `cache` globally so that +PHP dependencies, the following example defines a default `cache` so that all jobs inherit it. PHP libraries modules are installed in `vendor/` and are cached per-branch: ```yaml -# -# https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/PHP.gitlab-ci.yml -# -image: php:7.2 - -# Cache libraries in between jobs -cache: - key: $CI_COMMIT_REF_SLUG - paths: - - vendor/ - -before_script: - # Install and run Composer - - curl --show-error --silent "https://getcomposer.org/installer" | php - - php composer.phar install +default: + image: php:7.2 + cache: # Cache libraries in between jobs + key: $CI_COMMIT_REF_SLUG + paths: + - vendor/ + before_script: + # Install and run Composer + - curl --show-error --silent "https://getcomposer.org/installer" | php + - php composer.phar install test: script: @@ -387,32 +377,24 @@ test: ### Cache Python dependencies If your project uses [pip](https://pip.pypa.io/en/stable/) to install -Python dependencies, the following example defines `cache` globally so that +Python dependencies, the following example defines a default `cache` so that all jobs inherit it. pip's cache is defined under `.cache/pip/` and is cached per-branch: ```yaml -# -# https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Python.gitlab-ci.yml -# -image: python:latest +default: + image: python:latest + cache: # Pip's cache doesn't store the python packages + paths: # https://pip.pypa.io/en/stable/topics/caching/ + - .cache/pip + before_script: + - python -V # Print out python version for debugging + - pip install virtualenv + - virtualenv venv + - source venv/bin/activate -# Change pip's cache directory to be inside the project directory since we can -# only cache local items. -variables: +variables: # Change pip's cache directory to be inside the project directory since we can only cache local items. PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip" -# Pip's cache doesn't store the python packages -# https://pip.pypa.io/en/stable/topics/caching/ -cache: - paths: - - .cache/pip - -before_script: - - python -V # Print out python version for debugging - - pip install virtualenv - - virtualenv venv - - source venv/bin/activate - test: script: - python setup.py test @@ -423,25 +405,20 @@ test: ### Cache Ruby dependencies If your project uses [Bundler](https://bundler.io) to install -gem dependencies, the following example defines `cache` globally so that all +gem dependencies, the following example defines a default `cache` so that all jobs inherit it. Gems are installed in `vendor/ruby/` and are cached per-branch: ```yaml -# -# https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Ruby.gitlab-ci.yml -# -image: ruby:2.6 - -# Cache gems in between builds -cache: - key: $CI_COMMIT_REF_SLUG - paths: - - vendor/ruby - -before_script: - - ruby -v # Print out ruby version for debugging - - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to - - bundle install -j $(nproc) # Install dependencies into ./vendor/ruby +default: + image: ruby:2.6 + cache: # Cache gems in between builds + key: $CI_COMMIT_REF_SLUG + paths: + - vendor/ruby + before_script: + - ruby -v # Print out ruby version for debugging + - bundle config set --local path 'vendor/ruby' # The location to install the specified gems to + - bundle install -j $(nproc) # Install dependencies into ./vendor/ruby rspec: script: @@ -456,13 +433,14 @@ For example, a testing job might not need the same gems as a job that deploys to production: ```yaml -cache: - key: - files: - - Gemfile.lock - prefix: $CI_JOB_NAME - paths: - - vendor/ruby +default: + cache: + key: + files: + - Gemfile.lock + prefix: $CI_JOB_NAME + paths: + - vendor/ruby test_job: stage: test @@ -572,18 +550,19 @@ stages: - build - test -before_script: - - echo "Hello" +default: + cache: + key: build-cache + paths: + - vendor/ + before_script: + - echo "Hello" job A: stage: build script: - mkdir vendor/ - echo "build" > vendor/hello.txt - cache: - key: build-cache - paths: - - vendor/ after_script: - echo "World" @@ -591,10 +570,6 @@ job B: stage: test script: - cat vendor/hello.txt - cache: - key: build-cache - paths: - - vendor/ ``` If one machine has one runner installed, then all jobs for your project @@ -602,6 +577,7 @@ run on the same host: 1. Pipeline starts. 1. `job A` runs. +1. The cache is extracted (if found). 1. `before_script` is executed. 1. `script` is executed. 1. `after_script` is executed. diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index 7be12d0c0fd..09f2758113f 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -49,18 +49,20 @@ When the job runs: ## Run a CI/CD job -You can run a CI/CD job from chat with the `/project-name run` -[slash command](../../integration/slash_commands.md). - -Prerequisites: +Prerequisite: - You must have at least the Developer role for the project. -To run a CI/CD job: +You can run a CI/CD job on the default branch from chat. To run a CI/CD job: + +- In the chat client, enter `/<project-name> run <job name> <arguments>` where: -- In the chat client, enter `/project-name run <job name> <arguments>`. + - `<project-name>` is the name of the project. + - `<job name>` is the name of the CI/CD job to run. + - `<arguments>` is the arguments to pass to the CI/CD job. ChatOps schedules a pipeline that contains only the specified job. +Other [slash commands](../../user/project/integrations/gitlab_slack_application.md#slash-commands) are also available. ### Exclude a job from ChatOps diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 6d14928389d..7f454cabcf4 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -33,7 +33,7 @@ To connect to an external repository: If the **Run CI/CD for external repository** option is not available, the GitLab instance might not have any import sources configured. Ask an administrator for your instance to check -the [import sources configuration](../../user/admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). +the [import sources configuration](../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources). ## Pipelines for external pull requests diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md index 9ea5f76733a..f7604689b78 100644 --- a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -79,7 +79,7 @@ and [Container Registry](../../../user/packages/container_registry/index.md).  -1. Visit **Packages and registries > Container Registry**. Make sure the application image has been +1. Visit **Deploy > Container Registry**. Make sure the application image has been pushed.  diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index ce03e9e3916..3e77e8c6c25 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -13,6 +13,12 @@ to AWS. You can reference these images in your CI/CD pipeline. If you're using GitLab.com and deploying to the [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (ECS), read about [deploying to ECS](ecs/deploy_to_aws_ecs.md). +NOTE: +If you are comfortable configuring a deployment yourself and just need to retrieve +AWS credentials, consider using [ID tokens and OpenID Connect](../cloud_services/aws/index.md). +ID tokens are more secure than storing credentials in CI/CD variables, but do not +work with the guidance on this page. + ## Authenticate GitLab with AWS To use GitLab CI/CD to connect to AWS, you must authenticate. diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md index 29d27e440ec..b921dabc4e2 100644 --- a/doc/ci/cloud_services/azure/index.md +++ b/doc/ci/cloud_services/azure/index.md @@ -107,6 +107,8 @@ You can find your subscription ID in: - The [Azure Portal](https://learn.microsoft.com/en-us/azure/azure-portal/get-subscription-tenant-id#find-your-azure-subscription). - The [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/manage-azure-subscriptions-azure-cli#get-the-active-subscription). +The command above grants read-only permissions to the entire subscription. For more information on applying the principle of least privilege in the context of your organization, read [Best practices for Azure AD roles](https://learn.microsoft.com/en-us/azure/active-directory/roles/best-practices). + ## Retrieve a temporary credential After you configure the Azure AD application and federated identity credentials, @@ -121,8 +123,11 @@ variables: AZURE_TENANT_ID: "<tenant-id>" auth: + id_tokens: + GITLAB_OIDC_TOKEN: + aud: https://gitlab.com script: - - az login --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID --federated-token $CI_JOB_JWT_V2 + - az login --service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID --federated-token $GITLAB_OIDC_TOKEN - az account show ``` @@ -131,7 +136,7 @@ The CI/CD variables are: - `AZURE_CLIENT_ID`: The [application client ID you saved earlier](#create-azure-ad-application-and-service-principal). - `AZURE_TENANT_ID`: Your Azure Active Directory. You can [find it by using the Azure CLI or Azure Portal](https://learn.microsoft.com/en-us/azure/active-directory/fundamentals/active-directory-how-to-find-tenant). -- `CI_JOB_JWT_V2`: The JSON web token is a [predefined CI/CD variable](../../variables/predefined_variables.md). +- `GITLAB_OIDC_TOKEN`: An OIDC [ID token](../../yaml/index.md#id_tokens). ## Troubleshooting diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md index 999c9b0c0fb..ae35b3779c3 100644 --- a/doc/ci/components/index.md +++ b/doc/ci/components/index.md @@ -11,7 +11,7 @@ type: reference FLAG: On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to enable the feature flag named `ci_namespace_catalog_experimental`. +To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `ci_namespace_catalog_experimental`. On GitLab.com, this feature is not available. This feature is not ready for production use. This feature is an experimental feature and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/9897) @@ -169,7 +169,7 @@ containing: For example, for a component repository located at `gitlab-org/dast` on `gitlab.com`: - The path `gitlab.com/gitlab-org/dast` tries to load the `template.yml` from the root directory. -- The path `gitalb.com/gitlab-org/dast/api-scan` tries to load the `template.yml` from the `/api-scan` directory. +- The path `gitlab.com/gitlab-org/dast/api-scan` tries to load the `template.yml` from the `/api-scan` directory. **Additional notes:** @@ -178,6 +178,167 @@ For example, for a component repository located at `gitlab-org/dast` on `gitlab. - If a tag is named the same as a commit SHA that exists, like `e3262fdd0914fa823210cdb79a8c421e2cef79d8`, the commit SHA takes precedence over the tag. +### Best practices + +#### Avoid using global keywords + +When using [global keywords](../yaml/index.md#global-keywords) all jobs in the +pipeline are affected. Using these keywords in a component affects all jobs in a +pipeline, whether they are directly defined in the main `.gitlab-ci.yml` or +in any included components. + +To make the composition of pipelines more deterministic, either: + +- Duplicate the default configuration for each job. +- Use [`extends`](../yaml/index.md#extends) feature within the component. + +```yaml +## +# BAD +default: + image: ruby:3.0 + +rspec: + script: bundle exec rspec +``` + +```yaml +## +# GOOD +rspec: + image: ruby:3.0 + script: bundle exec rspec +``` + +#### Replace hard-coded values with inputs + +A typical hard-coded value found in CI templates is `stage:` value. Such hard coded values may force the user +of the component to know and adapt the pipeline to such implementation details. + +For example, if `stage: test` is hard-coded for a job in a component, the pipeline using the component must +define the `test` stage. Additionally, if the user of the component want to customize the stage value it has +to override the configuration: + +```yaml +## +# BAD: In order to use different stage name you need to override all the jobs +# included by the component. +include: + - component: gitlab.com/gitlab-org/ruby-test@1.0 + +stages: [verify, deploy] + +unit-test: + stage: verify + +integration-test: + stage: verify +``` + +```yaml +## +# BAD: In order to use the component correctly you need to define the stage +# that is hard-coded in it. +include: + - component: gitlab.com/gitlab-org/ruby-test@1.0 + +stages: [test, deploy] +``` + +To improve this we can use [input parameters](../yaml/includes.md#define-input-parameters-with-specinputs) +allowing the user of a component to inject values that can be customized: + +```yaml +## +# GOOD: We don't need to know the implementation details of a component and instead we can +# rely on the inputs. +include: + - component: gitlab.com/gitlab-org/ruby-test@1.0 + inputs: + stage: verify + +stages: [verify, deploy] + +## +# inside the component YAML: +spec: + inputs: + stage: + default: test +--- +unit-test: + stage: $[[ inputs.stage ]] + script: echo unit tests + +integration-test: + stage: $[[ inputs.stage ]] + script: echo integration tests +``` + +#### Prefer inputs over variables + +If variables are only used for YAML evaluation (for example `rules`) and not by the Runner +execution, it's advised to use inputs instead. +Inputs are explicitly defined in the component's contract and they are better validated +than variables. + +For example, if a required input is not passed an error is returned as soon as the component +is being used. By contrast, if a variable is not defined, it's value is empty. + +```yaml +## +# BAD: you need to configure an environment variable for a custom value that doesn't need +# to be used on the Runner +unit-test: + image: $MY_COMPONENT_X_IMAGE + script: echo unit tests + +integration-test: + image: $MY_COMPONENT_X_IMAGE + script: echo integration tests + +## +# Usage: +include: + - component: gitlab.com/gitlab-org/ruby-test@1.0 + +variables: + MY_COMPONENT_X_IMAGE: ruby:3.2 +``` + +```yaml +## +# GOOD: we define a customizable value and accept it as input +spec: + inputs: + image: + default: ruby:3.0 +--- +unit-test: + image: $[[ inputs.image ]] + script: echo unit tests + +integration-test: + image: $[[ inputs.image ]] + script: echo integration tests + +## +# Usage: +include: + - component: gitlab.com/gitlab-org/ruby-test@1.0 + inputs: + image: ruby:3.2 +``` + +#### Use semantic versioning + +When tagging and releasing new versions of components we recommend using [semantic versioning](https://semver.org) +which is the standard for communicating bugfixes, minor and major or breaking changes. + +We recommend adopting at least the `MAJOR.MINOR` format. + +For example: `2.1`, `1.0.0`, `1.0.0-alpha`, `2.1.3`, `3.0.0-rc.1`. + ## CI/CD Catalog The CI/CD Catalog is a list of [components repositories](#components-repository), @@ -185,6 +346,8 @@ each containing resources that you can add to your CI/CD pipelines. ### Mark the project as a catalog resource +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407249) in GitLab 16.1. + After components are added to a components repository, they can immediately be [used](#use-a-component-in-a-cicd-configuration) to build pipelines in other projects. However, this repository is not discoverable. You must mark this project as a catalog resource to allow it to be visible in the CI Catalog @@ -199,3 +362,15 @@ To mark a project as a catalog resource: NOTE: This action is not reversible. + +## Convert a CI template to component + +Any existing CI template, that you share with other projects via `include:` syntax, can be converted to a CI component. + +1. Decide whether you want the component to be part of an existing [components repository](#components-repository), + if you want to logically group components together. Create and setup a [components repository](#components-repository) otherwise. +1. Create a YAML file in the components repository according to the expected [directory structure](#directory-structure). +1. Copy the content of the template YAML file into the new component YAML file. +1. Refactor the component YAML to follow the [best practices](#best-practices) for components. +1. Leverage the `.gitlab-ci.yml` in the components repository to [test changes to the component](#test-a-component). +1. Tag and [release the component](#release-a-component). diff --git a/doc/ci/docker/authenticate_registry.md b/doc/ci/docker/authenticate_registry.md index 224d0cdf7aa..52cc3071fda 100644 --- a/doc/ci/docker/authenticate_registry.md +++ b/doc/ci/docker/authenticate_registry.md @@ -17,14 +17,14 @@ In [`before_script`](../yaml/index.md#before_script), run `docker login`: ```yaml -image: docker:20.10.16 +default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind variables: DOCKER_TLS_CERTDIR: "/certs" -services: - - docker:20.10.16-dind - build: stage: build before_script: @@ -125,14 +125,14 @@ The following example shows [`before_script`](../yaml/index.md#before_script). The same commands apply for any solution you implement. ```yaml -image: docker:20.10.16 +default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind variables: DOCKER_TLS_CERTDIR: "/certs" -services: - - docker:20.10.16-dind - build: stage: build before_script: diff --git a/doc/ci/docker/docker_layer_caching.md b/doc/ci/docker/docker_layer_caching.md index 145ead9a212..861bea70cb7 100644 --- a/doc/ci/docker/docker_layer_caching.md +++ b/doc/ci/docker/docker_layer_caching.md @@ -28,19 +28,18 @@ with the `--cache-from` argument must be pulled This example `.gitlab-ci.yml` file shows how to use Docker caching: ```yaml -image: docker:20.10.16 - -services: - - docker:20.10.16-dind +default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind + before_script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY variables: # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled DOCKER_HOST: tcp://docker:2376 DOCKER_TLS_CERTDIR: "/certs" -before_script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - build: stage: build script: @@ -50,7 +49,7 @@ build: - docker push $CI_REGISTRY_IMAGE:latest ``` -In the `script` section for the `build` stage: +In the `script` section for the `build` job: 1. The first command tries to pull the image from the registry so that it can be used as a cache for the `docker build` command. diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 004da63476e..5e8f2b36620 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -61,8 +61,9 @@ the Docker commands, but needs permission to do so. 1. In GitLab, add `docker info` to `.gitlab-ci.yml` to verify that Docker is working: ```yaml - before_script: - - docker info + default: + before_script: + - docker info build_image: script: @@ -155,7 +156,12 @@ To use Docker-in-Docker with TLS enabled: `docker:20.10.16-dind` service: ```yaml - image: docker:20.10.16 + default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind + before_script: + - docker info variables: # When you use the dind service, you must instruct Docker to talk with @@ -174,12 +180,6 @@ To use Docker-in-Docker with TLS enabled: # container, thanks to volume mount from config.toml DOCKER_TLS_CERTDIR: "/certs" - services: - - docker:20.10.16-dind - - before_script: - - docker info - build: stage: build script: @@ -215,7 +215,12 @@ You can now use `docker` in the job script. You should include the `docker:20.10.16-dind` service: ```yaml -image: docker:20.10.16 +default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind + before_script: + - docker info variables: # When using dind service, you must instruct docker to talk with the @@ -235,12 +240,6 @@ variables: # This instructs Docker not to start over TLS. DOCKER_TLS_CERTDIR: "" -services: - - docker:20.10.16-dind - -before_script: - - docker info - build: stage: build script: @@ -280,7 +279,12 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: `docker:20.10.16-dind` service: ```yaml - image: docker:20.10.16 + default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind + before_script: + - docker info variables: # When using dind service, you must instruct Docker to talk with @@ -307,12 +311,6 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: DOCKER_TLS_VERIFY: 1 DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" - services: - - docker:20.10.16-dind - - before_script: - - docker info - build: stage: build script: @@ -352,9 +350,10 @@ Docker-in-Docker is the recommended configuration, but you should be aware of th To use Docker commands in your CI/CD jobs, you can bind-mount `/var/run/docker.sock` into the container. Docker is then available in the context of the image. -> If you bind the Docker socket and you are [using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -> you can no longer use `docker:20.10.16-dind` as a service. -> Volume bindings also affect services, making them incompatible. +If you bind the Docker socket and you are +[using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), +you can no longer use `docker:20.10.16-dind` as a service. Volume bindings also affect services, +making them incompatible. To make Docker available in the context of the image, you need to mount `/var/run/docker.sock` into the launched containers. To do this with the Docker @@ -390,11 +389,10 @@ sudo gitlab-runner register -n \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` -> If you want to use more complex Docker-in-Docker configurations, like it is necessary to run Code Quality checks with -> Code Climate, you need to ensure that the paths to the build directory are the same on the host as well as inside the -> Docker container. -> See section "[Improve Code Quality performance with private runners](../testing/code_quality.md#improve-code-quality-performance-with-private-runners)" -> in the Code Quality documentation. +To use more complex Docker-in-Docker configurations, such as is necessary to run Code Quality checks +with Code Climate, you need to ensure that the paths to the build directory are the same on the host +as well as inside the Docker container. For more details, see +[Improve Code Quality performance with private runners](../testing/code_quality.md#improve-code-quality-performance-with-private-runners). #### Enable registry mirror for `docker:dind` service @@ -558,10 +556,10 @@ You do not need to include the `docker:20.10.16-dind` service, like you do when you use the Docker-in-Docker executor: ```yaml -image: docker:20.10.16 - -before_script: - - docker info +default: + image: docker:20.10.16 + before_script: + - docker info build: stage: build @@ -639,8 +637,8 @@ and [using the OverlayFS storage driver](https://docs.docker.com/storage/storage To build Docker images without enabling privileged mode on the runner, you can use one of these alternatives: -- [`kaniko`](using_kaniko.md) -- [`buildah`](https://github.com/containers/buildah) +- [`kaniko`](using_kaniko.md). +- [`buildah`](https://github.com/containers/buildah). There is a [known issue](https://github.com/containers/buildah/issues/4049) with running as non-root, you might need this [workaround](https://docs.gitlab.com/runner/configuration/configuring_runner_operator.html#configure-setfcap) if you are using OpenShift Runner. For example, with `buildah`: @@ -703,10 +701,10 @@ This issue can occur when the service's image name [includes a registry hostname](../../ci/services/index.md#available-settings-for-services). For example: ```yaml -image: docker:20.10.16 - -services: - - registry.hub.docker.com/library/docker:20.10.16-dind +default: + image: docker:20.10.16 + services: + - registry.hub.docker.com/library/docker:20.10.16-dind ``` A service's hostname is [derived from the full image name](../../ci/services/index.md#accessing-the-services). @@ -714,14 +712,14 @@ However, the shorter service hostname `docker` is expected. To allow service resolution and access, add an explicit alias for the service name `docker`: ```yaml -image: docker:20.10.16 - -services: - - name: registry.hub.docker.com/library/docker:20.10.16-dind - alias: docker +default: + image: docker:20.10.16 + services: + - name: registry.hub.docker.com/library/docker:20.10.16-dind + alias: docker ``` -### Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password +### Error: `Error response from daemon: Get "https://registry-1.docker.io/v2/": unauthorized: incorrect username or password` This error appears when you use the deprecated variable, `CI_BUILD_TOKEN`. To prevent users from receiving this error, you should: diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 022aa7c3093..d3a2f4ece06 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -74,10 +74,8 @@ services that you want to use during runtime: ```yaml default: image: ruby:2.6 - services: - postgres:11.7 - before_script: - bundle install diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index 0ef37452cbb..2a9b381c517 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -163,9 +163,10 @@ require `Administrator` to approve every deployment job in `Production`.  -### Allow self-approval **(PREMIUM)** +### Allow self-approval -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381418) in GitLab 15.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381418) in GitLab 15.8. +> - Automatic approval [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124638) in GitLab 16.2 due to [usability issues](https://gitlab.com/gitlab-org/gitlab/-/issues/391258). By default, the user who triggers a deployment pipeline can't also approve the deployment job. To allow self-approval of a deployment job: @@ -175,9 +176,6 @@ To allow self-approval of a deployment job: 1. Expand **Protected environments**. 1. From the **Approval options**, select the **Allow pipeline triggerer to approve deployment** checkbox. -When a pipeline runs, deployment jobs are automatically approved in the pipeline if the user who -triggered the deployment is allowed to approve. - ## Approve or reject a deployment > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/) in GitLab 14.9 diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 8be46da3fa8..e15e09b27c1 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -170,23 +170,3 @@ For more information, see [Custom CI/CD configuration path](../pipelines/setting ## Require an approval before deploying Before promoting a deployment to a production environment, cross-verifying it with a dedicated testing group is an effective way to ensure safety. For more information, see [Deployment Approvals](deployment_approvals.md). - -## Troubleshooting - -### Pipelines jobs fail with `The deployment job is older than the previously succeeded deployment job...` - -This is caused by the [Prevent outdated deployment jobs](../pipelines/settings.md#prevent-outdated-deployment-jobs) feature. -If you have multiple jobs for the same environment (including non-deployment jobs), you might encounter this problem, for example: - -```yaml -build:service-a: - environment: - name: production - -build:service-b: - environment: - name: production -``` - -The [Prevent outdated deployment jobs](../pipelines/settings.md#prevent-outdated-deployment-jobs) might -not work well with this configuration, and must be disabled. diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index f2fb8eaa27e..689f92723ee 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -32,7 +32,7 @@ There are a few ways to view a list of environments for a given project: - On the project's overview page, if at least one environment is available (that is, not stopped).  -- On the left sidebar, select **Deployments > Environments**. +- On the left sidebar, select **Operate > Environments**. The environments are displayed.  @@ -159,10 +159,10 @@ deploy_review_app: environment: name: review/$CI_COMMIT_REF_SLUG url: https://$CI_ENVIRONMENT_SLUG.example.com - only: - - branches - except: - - main + rules: + - if: $CI_COMMIT_BRANCH == "main" + when: never + - if: $CI_COMMIT_BRANCH ``` #### Set a dynamic environment URL @@ -603,7 +603,7 @@ To stop an environment in the GitLab UI: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Operate > Environments**. 1. Next to the environment you want to stop, select **Stop**. -1. On the confirmation dialog box, select **Stop environment**. +1. On the confirmation dialog, select **Stop environment**. #### Multiple stop actions for an environment @@ -668,7 +668,7 @@ To delete an environment: 1. Select **Operate > Environments**. 1. Select the **Stopped** tab. 1. Next to the environment you want to delete, select **Delete environment**. -1. On the confirmation dialog box, select **Delete environment**. +1. On the confirmation dialog, select **Delete environment**. ### Access an environment for preparation or verification purposes diff --git a/doc/ci/environments/kubernetes_dashboard.md b/doc/ci/environments/kubernetes_dashboard.md index 7da48bed5d7..ee16294e654 100644 --- a/doc/ci/environments/kubernetes_dashboard.md +++ b/doc/ci/environments/kubernetes_dashboard.md @@ -7,7 +7,9 @@ type: reference # Dashboard for Kubernetes (Beta) **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../policy/experiment-beta-support.md#beta). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../policy/experiment-beta-support.md#beta). +> - Feature flag `environment_settings_to_graphql` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124177) in GitLab 16.2. +> - Feature flags `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125835) in GitLab 16.2. Use the Dashboard for Kubernetes to understand the status of your clusters with an intuitive visual interface. The dashboard works with every connected Kubernetes cluster, whether you deployed them @@ -22,9 +24,10 @@ Configure a dashboard to use it for a given environment. You can configure dashboard for an environment that already exists, or add one when you create an environment. -Prerequisite: +Prerequisites: - The agent for Kubernetes must be shared with the environment's project, or its parent group, using the [`user_access`](../../user/clusters/agent/user_access.md) keyword. +- Self-managed only. KAS is running on the GitLab subdomain. For example, `kas.example.com` and `example.com`. ### The environment already exists diff --git a/doc/ci/examples/deployment/composer-npm-deploy.md b/doc/ci/examples/deployment/composer-npm-deploy.md index 20083413f66..5d1ae0048af 100644 --- a/doc/ci/examples/deployment/composer-npm-deploy.md +++ b/doc/ci/examples/deployment/composer-npm-deploy.md @@ -84,8 +84,8 @@ stage_deploy: artifacts: paths: - build/ - only: - - dev + rules: + - if: $CI_COMMIT_BRANCH == "dev" script: - ssh-add <(echo "$STAGING_PRIVATE_KEY") - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp" @@ -96,7 +96,7 @@ stage_deploy: Here's the breakdown: -1. `only:dev` means that this build runs only when something is pushed to the `dev` branch. You can remove this block completely and have everything run on every push (but probably this is something you don't want). +1. `rules:if: $CI_COMMIT_BRANCH == "dev"` means that this build runs only when something is pushed to the `dev` branch. You can remove this block completely and have everything run on every push (but probably this is something you don't want). 1. `ssh-add ...` we add that private key you added on the web UI to the Docker container. 1. We connect via `ssh` and create a new `_tmp` folder. 1. We connect via `scp` and upload the `build` folder (which was generated by a `npm` script) to our previously created `_tmp` folder. @@ -131,28 +131,26 @@ Since this was a WordPress project, it includes real code snippets. Some further Our final `.gitlab-ci.yml` looks like this: ```yaml -image: tetraweb/php - -before_script: - - apt-get update - - apt-get install zip unzip - - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" - - php composer-setup.php - - php -r "unlink('composer-setup.php');" - - php composer.phar install - - npm install - - npm run deploy - - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' - - mkdir -p ~/.ssh - - eval $(ssh-agent -s) - - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - stage_deploy: + image: tetraweb/php artifacts: paths: - build/ - only: - - dev + rules: + - if: $CI_COMMIT_BRANCH == "dev" + before_script: + - apt-get update + - apt-get install zip unzip + - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" + - php composer-setup.php + - php -r "unlink('composer-setup.php');" + - php composer.phar install + - npm install + - npm run deploy + - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )' + - mkdir -p ~/.ssh + - eval $(ssh-agent -s) + - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' script: - ssh-add <(echo "$STAGING_PRIVATE_KEY") - ssh -p22 server_user@server_host "mkdir htdocs/wp-content/themes/_tmp" diff --git a/doc/ci/examples/deployment/index.md b/doc/ci/examples/deployment/index.md index a156cf1acb0..793e60d517c 100644 --- a/doc/ci/examples/deployment/index.md +++ b/doc/ci/examples/deployment/index.md @@ -69,8 +69,8 @@ staging: - apt-get install -y ruby-dev - gem install dpl - dpl heroku api --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY - only: - - main + rules: + - if: $CI_COMMIT_BRANCH == "main" environment: staging ``` @@ -93,8 +93,8 @@ staging: script: - gem install dpl - dpl heroku api --app=my-app-staging --api_key=$HEROKU_STAGING_API_KEY - only: - - main + rules: + - if: $CI_COMMIT_BRANCH == "main" environment: staging production: @@ -102,8 +102,8 @@ production: script: - gem install dpl - dpl heroku api --app=my-app-production --api_key=$HEROKU_PRODUCTION_API_KEY - only: - - tags + rules: + - if: $CI_COMMIT_TAG environment: production ``` diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 3385aca1ef2..ee8b6bb30b6 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -31,7 +31,7 @@ to write such end-to-end tests, and how to set up GitLab CI/CD to automatically against your new code, on a branch-by-branch basis. For the scope of this article, we will walk you through the process of setting up GitLab CI/CD for end-to-end testing JavaScript-based applications with WebdriverIO, but the general strategy should carry over to other languages. -We assume you are familiar with GitLab, [GitLab CI/CD](../../index.md), [Review Apps](../../review_apps/index.md), and running your app locally, e.g., on `localhost:8000`. +We assume you are familiar with GitLab, [GitLab CI/CD](../../index.md), [Review Apps](../../review_apps/index.md), and running your app locally, for example, on `localhost:8000`. ## What to test @@ -45,7 +45,7 @@ infrastructure is up and running, and that your units of code work well together ## Selenium and WebdriverIO -[Selenium](https://www.selenium.dev/) is a piece of software that can control web browsers, e.g., to make them +[Selenium](https://www.selenium.dev/) is a piece of software that can control web browsers, for example, to make them visit a specific URL or interact with elements on the page. It can be programmatically controlled from a variety of programming languages. In this article we're going to be using the [WebdriverIO](http://v4.webdriver.io/) JavaScript bindings, but the general concept should carry over @@ -115,7 +115,7 @@ easiest way to get started is to start with provides an overview of all available options. The two options that are going to be most relevant now are the `specs` option, which is an array of paths to your tests, and the `baseUrl` option, which points to where your app is running. And finally, we will need to tell WebdriverIO in which browsers we would like to run our -tests. This can be configured through the `capabilities` option, which is an array of browser names (e.g. +tests. This can be configured through the `capabilities` option, which is an array of browser names (for example, `firefox` or `chrome`). It is recommended to install [selenium-assistant](https://googlechromelabs.github.io/selenium-assistant/) to detect all installed browsers: @@ -130,7 +130,7 @@ But of course, a simple configuration of `config.capabilities = ['firefox']` wou If you've installed WebdriverIO as a dependency (`npm install --save-dev webdriverio`), you can add a line to the `scripts` property in your -`package.json` that runs `wdio` with the path to your configuration file as value, e.g.: +`package.json` that runs `wdio` with the path to your configuration file as value, for example: ```javascript "confidence-check": "wdio wdio.conf.js", @@ -152,10 +152,10 @@ For the scope of this article, we've defined an additional [CI/CD stage](../../y [Docker image](../../docker/using_docker_images.md). However, WebdriverIO fires up actual browsers to interact with your application, so we need to install and run them. Furthermore, WebdriverIO uses Selenium as a common interface to control different browsers, -so we need to install and run Selenium as well. Luckily, the Selenium project provides the Docker images +so we need to install and run Selenium as well. Luckily, the Selenium project provides the Docker images for Firefox [standalone-firefox](https://hub.docker.com/r/selenium/standalone-firefox/) and -[standalone-chrome](https://hub.docker.com/r/selenium/standalone-chrome/) that provide just that for -Firefox and Chrome, respectively. (Since Safari and Internet Explorer/Edge are not open source and +and for Chrome [standalone-chrome](https://hub.docker.com/r/selenium/standalone-chrome/). +(Since Safari and Internet Explorer/Edge are not open source and not available for Linux, we are unfortunately unable to use those in GitLab CI/CD). GitLab CI/CD makes it a breeze to link these images to our `confidence-check` jobs using the @@ -219,22 +219,27 @@ on GitLab CI/CD! To recap, our `.gitlab-ci.yml` configuration file looks something like this: ```yaml -image: node:8.10 +default: + image: node:8.10 + stages: - deploy - confidence-check + deploy_terraform: stage: deploy script: # Your Review App deployment scripts - for a working example please check https://gitlab.com/Flockademic/Flockademic/blob/5a45f1c2412e93810fab50e2dab8949e2d0633c7/.gitlab-ci.yml#L315 - echo environment: production + e2e:firefox: stage: confidence-check services: - selenium/standalone-firefox script: - npm run confidence-check --host=selenium__standalone-firefox + e2e:chrome: stage: confidence-check services: diff --git a/doc/ci/examples/index.md b/doc/ci/examples/index.md index a020f673fd7..514817adc91 100644 --- a/doc/ci/examples/index.md +++ b/doc/ci/examples/index.md @@ -102,7 +102,7 @@ to [the templates list](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/g ### Adding templates to your GitLab installation **(PREMIUM SELF)** You can add custom examples and templates to your self-managed GitLab instance. -Your GitLab administrator can [designate an instance template repository](../../user/admin_area/settings/instance_template_repository.md) +Your GitLab administrator can [designate an instance template repository](../../administration/settings/instance_template_repository.md) that contains examples and templates specific to your organization. ## Other resources diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index cf12943d279..14d8e0fc72f 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -481,10 +481,10 @@ In order to build and test our app with GitLab CI/CD, we need a file called `.gi Our `.gitlab-ci.yml` file will look like this: ```yaml -image: registry.gitlab.com/<USERNAME>/laravel-sample:latest - -services: - - mysql:5.7 +default: + image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + services: + - mysql:5.7 variables: MYSQL_DATABASE: homestead @@ -513,14 +513,13 @@ deploy_production: - ssh-add <(echo "$SSH_PRIVATE_KEY") - mkdir -p ~/.ssh - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - - ~/.composer/vendor/bin/envoy run deploy --commit="$CI_COMMIT_SHA" environment: name: production url: http://192.168.1.1 when: manual - only: - - main + rules: + - if: $CI_COMMIT_BRANCH == "main" ``` That's a lot to take in, isn't it? Let's run through it step by step. @@ -533,10 +532,10 @@ The `services` keyword defines additional images [that are linked to the main im Here we use the container image we created before as our main image and also use MySQL 5.7 as a service. ```yaml -image: registry.gitlab.com/<USERNAME>/laravel-sample:latest - -services: - - mysql:5.7 +default: + image: registry.gitlab.com/<USERNAME>/laravel-sample:latest + services: + - mysql:5.7 ... ``` @@ -592,7 +591,7 @@ If the SSH keys have added successfully, we can run Envoy. As mentioned before, GitLab supports [Continuous Delivery](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#continuous-delivery) methods as well. The [environment](../../yaml/index.md#environment) keyword tells GitLab that this job deploys to the `production` environment. The `url` keyword is used to generate a link to our application on the GitLab Environments page. -The `only` keyword tells GitLab CI/CD that the job should be executed only when the pipeline is building the `main` branch. +The `rules:if` keyword tells GitLab CI/CD that the job should be executed only when the pipeline is building the `main` branch. Lastly, `when: manual` is used to turn the job from running automatically to a manual action. ```yaml @@ -604,16 +603,14 @@ deploy_production: - ssh-add <(echo "$SSH_PRIVATE_KEY") - mkdir -p ~/.ssh - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config' - # Run Envoy - ~/.composer/vendor/bin/envoy run deploy - environment: name: production url: http://192.168.1.1 when: manual - only: - - main + rules: + - if: $CI_COMMIT_BRANCH == "main" ``` You may also want to add another job for [staging environment](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/), to final test your application before deploying to production. diff --git a/doc/ci/examples/php.md b/doc/ci/examples/php.md index a83bcf69491..244c1e379da 100644 --- a/doc/ci/examples/php.md +++ b/doc/ci/examples/php.md @@ -90,12 +90,12 @@ Finally, commit your files and push them to GitLab to see your build succeeding The final `.gitlab-ci.yml` should look similar to this: ```yaml -# Select image from https://hub.docker.com/_/php -image: php:5.6 - -before_script: - # Install dependencies - - bash ci/docker_install.sh > /dev/null +default: + # Select image from https://hub.docker.com/_/php + image: php:5.6 + before_script: + # Install dependencies + - bash ci/docker_install.sh > /dev/null test:app: script: @@ -108,9 +108,10 @@ Testing against multiple versions of PHP is super easy. Just add another job with a different Docker image version and the runner does the rest: ```yaml -before_script: - # Install dependencies - - bash ci/docker_install.sh > /dev/null +default: + before_script: + # Install dependencies + - bash ci/docker_install.sh > /dev/null # We test PHP5.6 test:5.6: @@ -206,10 +207,9 @@ Instead of PHPUnit, you can use any other tool to run unit tests. For example you can use [`atoum`](https://github.com/atoum/atoum): ```yaml -before_script: - - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar - test:atoum: + before_script: + - wget http://downloads.atoum.org/nightly/mageekguy.atoum.phar script: - php mageekguy.atoum.phar ``` @@ -224,18 +224,18 @@ To execute Composer before running your tests, add the following to your # Composer stores all downloaded packages in the vendor/ directory. # Do not use the following if the vendor/ directory is committed to # your git repository. -cache: - paths: - - vendor/ - -before_script: - # Install composer dependencies - - wget https://composer.github.io/installer.sig -O - -q | tr -d '\n' > installer.sig - - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" - - php -r "if (hash_file('SHA384', 'composer-setup.php') === file_get_contents('installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" - - php composer-setup.php - - php -r "unlink('composer-setup.php'); unlink('installer.sig');" - - php composer.phar install +default: + cache: + paths: + - vendor/ + before_script: + # Install composer dependencies + - wget https://composer.github.io/installer.sig -O - -q | tr -d '\n' > installer.sig + - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" + - php -r "if (hash_file('SHA384', 'composer-setup.php') === file_get_contents('installer.sig')) { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;" + - php composer-setup.php + - php -r "unlink('composer-setup.php'); unlink('installer.sig');" + - php composer.phar install ``` ## Access private packages or dependencies diff --git a/doc/ci/index.md b/doc/ci/index.md index 75e668290c8..a3106a2475c 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -99,10 +99,19 @@ GitLab CI/CD features, grouped by DevOps stage, include: | [Cloud deployment](cloud_deployment/index.md) | Deploy your application to a main cloud provider. | | **Secure** | | | [Code Quality](testing/code_quality.md) | Analyze your source code quality. | -| [Container Scanning](../user/application_security/container_scanning/index.md) | Check your Docker containers for known vulnerabilities. | +| [Container Scanning](../user/application_security/container_scanning/index.md) | Scan your container images for known vulnerabilities. | +| [Coverage-guided fuzz testing](../user/application_security/coverage_fuzzing/index.md) | Test your application's behavior by providing randomized input. | +| [Dynamic Application Security Testing](../user/application_security/dast/index.md) | Test your application's runtime behavior for vulnerabilities. | | [Dependency Scanning](../user/application_security/dependency_scanning/index.md) | Analyze your dependencies for known vulnerabilities. | +| [Infrastructure as Code scanning](../user/application_security/iac_scanning/index.md) | Scan your IaC configuration files for known vulnerabilities. | | [License Compliance](../user/compliance/license_compliance/index.md) | Search your project dependencies for their licenses. | -| [Security Test reports](../user/application_security/index.md) | Check for app vulnerabilities. | +| [Secret Detection](../user/application_security/secret_detection/index.md) | Search your application's source code for secrets. | +| [Static Application Security Testing](../user/application_security/sast/index.md) | Test your application's source code for known vulnerabilities. | +| [Web API fuzz testing](../user/application_security/api_fuzzing/index.md) | Test your application's API behavior by providing randomized input. | +| **Govern** | | +| [Compliance frameworks](../user/group/compliance_frameworks.md) | Enforce a GitLab CI/CD configuration on all projects in a group. | +| [Scan execution policies](../user/application_security/policies/scan-execution-policies.md) | Enforce security scans run on a specified schedule or with the project pipeline. | +| [Scan results policies](../user/application_security/policies/scan-result-policies.md) | Enforce action based on results of a pipeline security scan. | ## Examples diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index a60aaa04ea1..c2fe3071b52 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -24,6 +24,8 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints to [trigger a multi-project pipeline](../pipelines/downstream_pipelines.md#trigger-a-multi-project-pipeline-by-using-the-api). - [Releases](../../api/releases/index.md) and [Release links](../../api/releases/links.md). - [Terraform plan](../../user/infrastructure/index.md). +- [Deployments](../../api/deployments.md). +- [Environments](../../api/environments.md). The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by taking action like pushing a commit, diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md index 95d939c7027..3dd43a9760c 100644 --- a/doc/ci/jobs/job_artifacts.md +++ b/doc/ci/jobs/job_artifacts.md @@ -61,7 +61,7 @@ pdf: expire_in: 1 week ``` -If `expire_in` is not defined, the [instance-wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration) +If `expire_in` is not defined, the [instance-wide setting](../../administration/settings/continuous_integration.md#default-artifacts-expiration) is used. To prevent artifacts from expiring, you can select **Keep** from the job details page. @@ -193,7 +193,7 @@ job: > - [Improved performance](https://gitlab.com/gitlab-org/gitlab/-/issues/387765) in GitLab 15.9. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/407475) in GitLab 16.0. Feature flag `artifacts_management_page` removed. -You can view all artifacts stored in a project from the **CI/CD > Artifacts** page. +You can view all artifacts stored in a project from the **Build > Artifacts** page. This list displays all jobs and their associated artifacts. Expand an entry to access all artifacts associated with a job, including: @@ -351,7 +351,7 @@ Artifacts in old pipelines continue to be kept until a new pipeline runs for the Then the artifacts in the earlier pipeline for that ref are allowed to expire too. You can disable this behavior for all projects on a self-managed instance in the -[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). +[instance's CI/CD settings](../../administration/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). When **Keep artifacts from most recent successful jobs** is enabled, artifacts are always kept for [blocked](job_control.md#types-of-manual-jobs) pipelines. These artifacts expire only after the blocking job is triggered and the pipeline completes. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index b17db47eef2..09c7500a883 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -271,7 +271,7 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedule` | For [scheduled pipelines](../pipelines/schedules.md). | | `trigger` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | -| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **CI/CD > Pipelines** section. | +| `web` | For pipelines created by using **Run pipeline** button in the GitLab UI, from the project's **Build > Pipelines** section. | | `webide` | For pipelines created by using the [WebIDE](../../user/project/web_ide/index.md). | The following example runs the job as a manual job in scheduled pipelines or in push @@ -338,7 +338,7 @@ You can use the `$` character for both variables and paths. For example, if the `$DOCKERFILES_DIR` variable exists, its value is used. If it does not exist, the `$` is interpreted as being part of a path. -## Reuse rules in different jobs +### Reuse rules in different jobs > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322992) in GitLab 14.3. @@ -369,6 +369,10 @@ job2: ## Specify when jobs run with `only` and `except` +NOTE: +`only` and `except` are not being actively developed. [`rules`](#specify-when-jobs-run-with-rules) +is the preferred keyword to control when to add jobs to pipelines. + You can use [`only`](../yaml/index.md#only--except) and [`except`](../yaml/index.md#only--except) to control when to add jobs to pipelines. @@ -377,14 +381,18 @@ to control when to add jobs to pipelines. ### `only:refs` / `except:refs` examples -`only` or `except` used without `refs` is the same as -[`only:refs` / `except/refs`](../yaml/index.md#onlyrefs--exceptrefs) +You can use `only` or `except` with: -In the following example, `job` runs only for: +- Specific keywords. See the full list in the [`only`/`except` syntax reference](../yaml/index.md#onlyrefs--exceptrefs). +- Branch names. Avoid branch names that are exactly the same as a specific keyword. + For example, jobs configured to run for the `tags` keyword (tag pipelines) + would also run for a branch named `tags`. +- Regex patterns to specify a range of branch names. -- Git tags -- [Triggers](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines) -- [Scheduled pipelines](../pipelines/schedules.md) +The following examples omit `refs` because `only` or `except` used without `refs` +is the same as [`only:refs` / `except/refs`](../yaml/index.md#onlyrefs--exceptrefs). + +For example: ```yaml job: @@ -395,6 +403,12 @@ job: - schedules ``` +In this example, `job` runs only for: + +- Git tags +- [Triggers](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines) +- [Scheduled pipelines](../pipelines/schedules.md) + To execute jobs only for the parent repository and not forks: ```yaml @@ -1101,5 +1115,5 @@ to change this behavior. To run protected manual jobs: - Add the administrator as a direct member of the private project (any role) -- [Impersonate a user](../../user/admin_area/index.md#user-impersonation) who is a +- [Impersonate a user](../../administration/admin_area.md#user-impersonation) who is a direct member of the project. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index 17cdb4f7e3e..a05fe87013c 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -16,7 +16,7 @@ We have collected several resources that you may find useful before starting to The [Quick Start Guide](../quick_start/index.md) is a good overview of how GitLab CI/CD works. You may also be interested in [Auto DevOps](../../topics/autodevops/index.md) which can be used to build, test, and deploy your applications with little to no configuration needed at all. -For advanced CI/CD teams, [custom project templates](../../user/admin_area/custom_project_templates.md) can enable the reuse of pipeline configurations. +For advanced CI/CD teams, [custom project templates](../../administration/custom_project_templates.md) can enable the reuse of pipeline configurations. If you have questions that are not answered here, the [GitLab community forum](https://forum.gitlab.com/) can be a great resource. @@ -249,18 +249,14 @@ jobs: Example of the same pipeline using `cache` in GitLab CI/CD: ```yaml -image: node:latest - -# Cache modules in between jobs -cache: - key: $CI_COMMIT_REF_SLUG - paths: - - .npm/ - -before_script: - - npm ci --cache .npm --prefer-offline - test_async: + image: node:latest + cache: # Cache modules in between jobs + key: $CI_COMMIT_REF_SLUG + paths: + - .npm/ + before_script: + - npm ci --cache .npm --prefer-offline script: - node ./specs/start.js ./specs/async.spec.js ``` diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index dc1944e65c6..f3f14da037f 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -209,10 +209,10 @@ Refer to this index if these features aren't working as expected, or if you'd li For advanced CI/CD teams, project templates can enable the reuse of pipeline configurations, as well as encourage inner sourcing. -In self-managed GitLab instances, you can build an [Instance Template Repository](../../user/admin_area/settings/instance_template_repository.md). +In self-managed GitLab instances, you can build an [Instance Template Repository](../../administration/settings/instance_template_repository.md). Development teams across the whole organization can select templates from a dropdown list. A group maintainer or a group owner is able to set a group to use as the source for the -[custom project templates](../../user/admin_area/custom_project_templates.md). This can +[custom project templates](../../administration/custom_project_templates.md). This can be used by all projects in the group. An instance administrator can set a group as the source for [instance project templates](../../user/group/custom_project_templates.md), which can be used by projects in that instance. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index fcacc3b5d97..b6f30627b7e 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -11,7 +11,7 @@ type: reference > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/270059) in GitLab 13.10. The pipeline editor is the primary place to edit the GitLab CI/CD configuration in -the `.gitlab-ci.yml` file in the root of your repository. To access the editor, go to **CI/CD > Editor**. +the `.gitlab-ci.yml` file in the root of your repository. To access the editor, go to **Build > Pipeline editor**. From the pipeline editor page you can: @@ -45,7 +45,7 @@ The **Lint** tab is replaced with the **Validate** tab in GitLab 15.3. The lint in a successful [pipeline simulation](#simulate-a-cicd-pipeline). To test the validity of your GitLab CI/CD configuration before committing the changes, -you can use the CI lint tool. To access it, go to **CI/CD > Editor** and select the **Lint** tab. +you can use the CI lint tool. To access it, go to **Build > Pipeline editor** and select the **Lint** tab. This tool checks for syntax and logical errors but goes into more detail than the automatic [validation](#validate-ci-configuration) in the editor. @@ -77,11 +77,11 @@ for review. ## Visualize CI configuration > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241722) in GitLab 13.5. -> - [Moved to **CI/CD > Editor**](https://gitlab.com/gitlab-org/gitlab/-/issues/263141) in GitLab 13.7. +> - [Moved to **Build > Pipeline editor**](https://gitlab.com/gitlab-org/gitlab/-/issues/263141) in GitLab 13.7. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/290117) in GitLab 13.12. To view a visualization of your `.gitlab-ci.yml` configuration, in your project, -go to **CI/CD > Editor**, and then select the **Visualize** tab. The +go to **Build > Pipeline editor**, and then select the **Visualize** tab. The visualization shows all stages and jobs. Any [`needs`](../yaml/index.md#needs) relationships are displayed as lines connecting jobs together, showing the hierarchy of execution: diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index c29d23cfff7..6c7b00c827a 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -7,17 +7,17 @@ type: reference # Compute quota **(PREMIUM)** -> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" or "units of compute" in GitLab 16.1. +> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" or "compute minutes" in GitLab 16.1. NOTE: -The term `CI/CD minutes` is being renamed to `units of compute`. During this transition, you might see references in the UI and documentation to `CI/CD minutes`, `CI minutes`, `pipeline minutes`, `CI pipeline minutes`, `pipeline minutes quota`, and `units of compute`. For more information, see [epic 2150](https://gitlab.com/groups/gitlab-com/-/epics/2150). +The term `CI/CD minutes` is being renamed to `compute minutes`. During this transition, you might see references in the UI and documentation to `CI/CD minutes`, `CI minutes`, `pipeline minutes`, `CI pipeline minutes`, `pipeline minutes quota`, `compute credits`, `compute units`, and `compute minutes`. For more information, see [epic 2150](https://gitlab.com/groups/gitlab-com/-/epics/2150). Administrators can limit the amount of time that projects can use to run jobs on [shared runners](../runners/runners_scope.md#shared-runners) each month. This limit is tracked with a compute quota. By default, one minute of execution time by a single job uses -one unit of compute. The total execution time for a pipeline is +one compute minute. The total execution time for a pipeline is [the sum of all its jobs' durations](#how-compute-usage-is-calculated). Jobs can run concurrently, so the total usage can be higher than the end-to-end duration of a pipeline. @@ -25,17 +25,17 @@ end-to-end duration of a pipeline. On GitLab.com: - Compute quotas are enabled for all projects, but certain - projects [consume units of compute at a slower rate](#cost-factor). + projects [consume compute minutes at a slower rate](#cost-factor). - The base monthly compute quota for a GitLab.com [namespace](../../user/namespace/index.md) is determined by its [license tier](https://about.gitlab.com/pricing/). -- You can [purchase additional units of compute](#purchase-additional-units-of-compute) +- You can [purchase additional compute minutes](#purchase-additional-compute-minutes) if you need more than the amount of compute in your monthly quota. On self-managed GitLab instances: - Compute quotas are disabled by default. - When enabled, compute quotas apply to private projects only. -- Administrators can [assign more units of compute](#set-the-compute-quota-for-a-specific-namespace) +- Administrators can [assign more compute minutes](#set-the-compute-quota-for-a-specific-namespace) if a namespace uses all its monthly quota. [Project runners](../runners/runners_scope.md#project-runners) are not subject to a compute quota. @@ -80,7 +80,7 @@ To set a compute quota for a namespace: 1. Select **Admin Area**. 1. On the left sidebar, select **Overview > Groups**. 1. For the group you want to update, select **Edit**. -1. In the **Compute quota** box, enter the maximum number of units of compute. +1. In the **Compute quota** box, enter the maximum number of compute minutes. 1. Select **Save changes**. You can also use the [update group API](../../api/groups.md#update-group) or the @@ -134,64 +134,64 @@ The projects list shows [personal projects](../../user/project/working_with_proj with compute usage or shared runners usage in the current month only. The list is sorted in descending order of compute usage. -## Purchase additional units of compute **(FREE SAAS)** +## Purchase additional compute minutes **(FREE SAAS)** -If you're using GitLab SaaS, you can purchase additional packs of units of compute. -These additional units of compute: +If you're using GitLab SaaS, you can purchase additional packs of compute minutes. +These additional compute minutes: - Are used only after the monthly quota included in your subscription runs out. - Are carried over to the next month, if any remain at the end of the month. -- Are valid for 12 months from date of purchase or until all units of compute are consumed, whichever comes first. Expiry of units of compute is not enforced. +- Are valid for 12 months from date of purchase or until all compute minutes are consumed, whichever comes first. Expiry of compute minutes is not enforced. For example, with a GitLab SaaS Premium license: -- You have `10,000` monthly units of compute. -- You purchase an additional `5,000` units of compute. -- Your total limit is `15,000` units of compute. +- You have `10,000` monthly compute minutes. +- You purchase an additional `5,000` compute minutes. +- Your total limit is `15,000` compute minutes. -If you use `13,000` units of compute during the month, the next month your additional units of compute become -`2,000`. If you use `9,000` units of compute during the month, your additional units of compute remain the same. +If you use `13,000` compute minutes during the month, the next month your additional compute minutes become +`2,000`. If you use `9,000` compute minutes during the month, your additional compute minutes remain the same. -If you bought additional units of compute while on a trial subscription, those units of compute are available after the trial ends or you upgrade to a paid plan. +If you bought additional compute minutes while on a trial subscription, those compute minutes are available after the trial ends or you upgrade to a paid plan. -You can find pricing for additional units of compute on the +You can find pricing for additional compute minutes on the [GitLab Pricing page](https://about.gitlab.com/pricing/). -### Purchase units of compute for a group **(FREE SAAS)** +### Purchase compute minutes for a group **(FREE SAAS)** Prerequisite: - You must have the Owner role for the group. -You can purchase additional units of compute for your group. -You cannot transfer purchased units of compute from one group to another, +You can purchase additional compute minutes for your group. +You cannot transfer purchased compute minutes from one group to another, so be sure to select the correct group. 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Settings > Usage Quotas**. 1. Select **Pipelines**. -1. Select **Buy additional units of compute**. +1. Select **Buy additional compute minutes**. 1. Complete the details of the transaction. -After your payment is processed, the additional units of compute are added to your group +After your payment is processed, the additional compute minutes are added to your group namespace. -### Purchase units of compute for a personal namespace **(FREE SAAS)** +### Purchase compute minutes for a personal namespace **(FREE SAAS)** Prerequisite: - The namespace must be your personal namespace. -To purchase additional units of compute for your personal namespace: +To purchase additional compute minutes for your personal namespace: 1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Usage Quotas**. -1. Select **Buy additional units of compute**. GitLab redirects you to the Customers Portal. -1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more units of compute**, +1. Select **Buy additional compute minutes**. GitLab redirects you to the Customers Portal. +1. Locate the subscription card that's linked to your personal namespace on GitLab SaaS, select **Buy more compute minutes**, and complete the details of the transaction. -After your payment is processed, the additional units of compute are added to your personal +After your payment is processed, the additional compute minutes are added to your personal namespace. ## How compute usage is calculated @@ -206,17 +206,17 @@ Job duration * Cost factor not including time spent in the `created` or `pending` statuses. - [**Cost factor**](#cost-factor): A number based on project visibility. -The value is transformed into units of compute and added to the count of used units +The value is transformed into compute minutes and added to the count of used units in the job's top-level namespace. For example, if a user `alice` runs a pipeline: -- Under the `gitlab-org` namespace, the units of compute used by each job in the pipeline are +- Under the `gitlab-org` namespace, the compute minutes used by each job in the pipeline are added to the overall consumption for the `gitlab-org` namespace, not the `alice` namespace. -- For one of the personal projects in their namespace, the units of compute are added +- For one of the personal projects in their namespace, the compute minutes are added to the overall consumption for the `alice` namespace. -The compute used by one pipeline is the total units of compute used by all the jobs +The compute used by one pipeline is the total compute minutes used by all the jobs that ran in the pipeline. Jobs can run concurrently, so the total compute usage can be higher than the end-to-end duration of a pipeline. @@ -228,19 +228,19 @@ The cost factors for jobs running on shared runners on GitLab.com are: - Exceptions for public projects: - `0.5` for projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). - `0.008` for forks of projects in the [GitLab for Open Source program](../../subscriptions/community_programs.md#gitlab-for-open-source). For every 125 minutes of job execution time, - you use 1 unit of compute. + you use 1 compute minute. - Discounted dynamically for [community contributions to GitLab projects](#cost-factor-for-community-contributions-to-gitlab-projects). The cost factors on self-managed instances are: -- `0` for public projects, so they do not consume units of compute. +- `0` for public projects, so they do not consume compute minutes. - `1` for internal and private projects. #### Cost factor for community contributions to GitLab projects Community contributors can use up to 300,000 minutes on shared runners when contributing to open source projects maintained by GitLab. The maximum of 300,000 minutes would only be possible if contributing exclusively to projects [part of the GitLab product](https://about.gitlab.com/handbook/engineering/metrics/#projects-that-are-part-of-the-product). The total number of minutes available on shared runners -is reduced by the units of compute used by pipelines from other projects. +is reduced by the compute minutes used by pipelines from other projects. The 300,000 minutes applies to all SaaS tiers, and the cost factor calculation is: - `Monthly compute quota / 300,000 job duration minutes = Cost factor` @@ -279,27 +279,27 @@ On the first day of each calendar month, the accumulated compute usage is reset for all namespaces that use shared runners. This means your full quota is available, and calculations start again from `0`. -For example, if you have a monthly quota of `10,000` units of compute: +For example, if you have a monthly quota of `10,000` compute minutes: -- On **April 1**, you have `10,000` units of compute. -- During April, you use only `6,000` of the `10,000` units of compute. -- On **May 1**, the accumulated compute usage resets to `0`, and you have `10,000` units of compute to use again +- On **April 1**, you have `10,000` compute minutes. +- During April, you use only `6,000` of the `10,000` compute minutes. +- On **May 1**, the accumulated compute usage resets to `0`, and you have `10,000` compute minutes to use again during May. Usage data for the previous month is kept to show historical view of the consumption over time. -### Monthly rollover of purchased units of compute +### Monthly rollover of purchased compute minutes -If you purchase additional units of compute and don't use the full amount, the remaining amount rolls over to +If you purchase additional compute minutes and don't use the full amount, the remaining amount rolls over to the next month. For example: -- On **April 1**, you purchase `5,000` additional units of compute. -- During April, you use only `3,000` of the `5,000` additional units of compute. -- On **May 1**, the unused units of compute roll over, so you have `2,000` additional units of compute available for May. +- On **April 1**, you purchase `5,000` additional compute minutes. +- During April, you use only `3,000` of the `5,000` additional compute minutes. +- On **May 1**, the unused compute minutes roll over, so you have `2,000` additional compute minutes available for May. -Additional units of compute are a one-time purchase and do not renew or refresh each month. +Additional compute minutes are a one-time purchase and do not renew or refresh each month. ## What happens when you exceed the quota @@ -311,7 +311,7 @@ processing new jobs. - Any running job can be dropped at any point if the overall namespace usage goes over-quota by a grace period. -The grace period for running jobs is `1,000` units of compute. +The grace period for running jobs is `1,000` compute minutes. Jobs on project runners are not affected by the compute quota. @@ -319,8 +319,8 @@ Jobs on project runners are not affected by the compute quota. On GitLab SaaS an email notification is sent to the namespace owners when: -- The remaining units of compute is below 30% of the quota. -- The remaining units of compute is below 5% of the quota. +- The remaining compute minutes is below 30% of the quota. +- The remaining compute minutes is below 5% of the quota. - All the compute quota has been used. ### Special quota limits @@ -358,12 +358,12 @@ An administrator can reset the compute usage for a namespace for the current mon ### Reset usage for a personal namespace -1. Find the [user in the admin area](../../user/admin_area/index.md#administering-users). +1. Find the [user in the admin area](../../administration/admin_area.md#administering-users). 1. Select **Edit**. 1. In **Limits**, select **Reset compute usage**. ### Reset usage for a group namespace -1. Find the [group in the admin area](../../user/admin_area/index.md#administering-groups). +1. Find the [group in the admin area](../../administration/admin_area.md#administering-groups). 1. Select **Edit**. 1. In **Permissions and group features**, select **Reset compute usage**. diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index fdc03daf7ad..686020fc17a 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -449,6 +449,8 @@ pass `CI_MERGE_REQUEST_REF_PATH` to the downstream pipeline using [variable inhe ```yaml build_artifacts: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' stage: build script: - echo "This is a test artifact!" >> artifact.txt @@ -457,6 +459,8 @@ pass `CI_MERGE_REQUEST_REF_PATH` to the downstream pipeline using [variable inhe - artifact.txt upstream_job: + rules: + - if: $CI_PIPELINE_SOURCE == 'merge_request_event' variables: UPSTREAM_REF: $CI_MERGE_REQUEST_REF_PATH trigger: diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index c99df5b15e6..45af40a4cea 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -88,7 +88,7 @@ This table lists the refspecs injected for each pipeline type: |--------------- |---------------------------------------- | | pipeline for branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | | pipeline for tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | -| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+<sha>:refs/pipelines/<id>` | +| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+refs/pipelines/<id>:refs/pipelines/<id>` | The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a @@ -100,7 +100,7 @@ that might run pipelines after branch deletion. ### View pipelines You can find the current and historical pipeline runs under your project's -**CI/CD > Pipelines** page. You can also access pipelines for a merge request by navigating +**Build > Pipelines** page. You can also access pipelines for a merge request by navigating to its **Pipelines** tab.  @@ -290,7 +290,7 @@ pipelines. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24851) in GitLab 12.7. Users with the Owner role for a project can delete a pipeline -by selecting the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** +by selecting the pipeline in the **Build > Pipelines** to get to the **Pipeline Details** page, then selecting **Delete**.  @@ -433,7 +433,7 @@ If a stage contains more than 100 jobs, only the first 100 jobs are listed in th pipeline graph. The remaining jobs still run as usual. To see the jobs: - Select the pipeline, and the jobs are listed on the right side of the pipeline details page. -- On the left sidebar, select **CI/CD > Jobs**. +- On the left sidebar, select **Build > Jobs**. ### View job dependencies in the pipeline graph diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md deleted file mode 100644 index c2630d6810d..00000000000 --- a/doc/ci/pipelines/job_artifacts.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../jobs/job_artifacts.md' -remove_date: '2023-07-04' ---- - -This document was moved to [another location](../jobs/job_artifacts.md). - -<!-- This redirect file can be deleted after <2023-07-04>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index 7bff13aa390..c2fdbe3f6e5 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -222,7 +222,7 @@ You cannot use [auto-merge](../../user/project/merge_requests/merge_when_pipelin (formerly **Merge when pipeline succeeds**) to skip the merge train, when merge trains are enabled. See [issue 12267](https://gitlab.com/gitlab-org/gitlab/-/issues/12267) for more information. -### Cannot retry merge train pipeline cannot +### Cannot retry merge train pipeline When a merge train pipeline fails, the merge request is dropped from the train and the pipeline can't be retried. Merge train pipelines run on the merged result of the changes in the merge request and diff --git a/doc/ci/pipelines/pipeline_architectures.md b/doc/ci/pipelines/pipeline_architectures.md index dadf631e2bb..ac4c8c1a731 100644 --- a/doc/ci/pipelines/pipeline_architectures.md +++ b/doc/ci/pipelines/pipeline_architectures.md @@ -13,7 +13,7 @@ some of the important concepts related to them. You can structure your pipelines with different methods, each with their own advantages. These methods can be mixed and matched if needed: -- [Basic](#basic-pipelines): Good for straightforward projects where all the configuration is in one easy-to-find place. +- [Basic](#basic-pipelines): Good for straightforward projects where all the configuration is in one place. - [Directed Acyclic Graph](#directed-acyclic-graph-pipelines): Good for large, complex projects that need efficient execution. - [Parent-child pipelines](#parent-child-pipelines): Good for monorepos and projects with lots of independently defined components. @@ -31,9 +31,9 @@ own advantages. These methods can be mixed and matched if needed: <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see the [Multi-project pipelines demo](https://www.youtube.com/watch?v=g_PIwBM1J84). -## Basic Pipelines +## Basic pipelines -This is the simplest pipeline in GitLab. It runs everything in the build stage concurrently, +Basic pipelines are the simplest pipelines in GitLab. It runs everything in the build stage concurrently, and once all of those finish, it runs everything in the test and subsequent stages the same way. It's not the most efficient, and if you have lots of steps it can grow quite complex, but it's easier to maintain: @@ -66,7 +66,8 @@ stages: - test - deploy -image: alpine +default: + image: alpine build_a: stage: build @@ -132,7 +133,8 @@ stages: - test - deploy -image: alpine +default: + image: alpine build_a: stage: build @@ -251,7 +253,8 @@ stages: - test - deploy -image: alpine +default: + image: alpine build_a: stage: build @@ -281,7 +284,8 @@ stages: - test - deploy -image: alpine +default: + image: alpine build_b: stage: build diff --git a/doc/ci/pipelines/pipeline_security.md b/doc/ci/pipelines/pipeline_security.md index f035779e665..d499da21b88 100644 --- a/doc/ci/pipelines/pipeline_security.md +++ b/doc/ci/pipelines/pipeline_security.md @@ -13,18 +13,18 @@ Secrets management is the systems that developers use to securely store sensitiv in a secure environment with strict access controls. A **secret** is a sensitive credential that should be kept confidential, and includes: -- Passwords -- SSH keys -- Access tokens -- Other types of credentials +- Passwords. +- SSH keys. +- Access tokens. +- Any other types of credentials where exposure would be harmful to an organization. ## Secrets storage ### Secrets management providers Secrets that are the most sensitive and under the strictest policies should be stored -in a separate secrets management provider such as [Vault](https://www.vaultproject.io). -The secrets are stored outside of the GitLab instance, which is the safest option. +in a secrets management. [Vault](https://www.vaultproject.io) is one provider in this space. +When using Vault, secrets are stored outside of the GitLab instance. You can use the GitLab [Vault integration](../secrets/index.md#use-vault-secrets-in-a-ci-job) to retrieve those secrets in CI/CD pipelines when they are needed. @@ -39,7 +39,7 @@ Variable values: to the settings have access to the variables. - Can be [overridden](../variables/index.md#override-a-defined-cicd-variable), making it hard to determine which value was used. -- Are more easily exposed by accidental pipeline misconfiguration. +- Can be exposed by accidental pipeline misconfiguration. Sensitive data should be stored in a secrets management solution. If there is low sensitivity data that you want to store in a CI/CD variable, be sure to always: diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 38cdc5ed578..fe6c88c9c4d 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -31,7 +31,7 @@ To change the visibility of your pipelines and related features: When it is selected, pipelines and related features are visible: - For [**Public**](../../user/public_access.md) projects, to everyone. - - For **Internal** projects, to all authenticated users except [external users](../../user/admin_area/external_users.md). + - For **Internal** projects, to all authenticated users except [external users](../../administration/external_users.md). - For **Private** projects, to all project members (Guest or higher). When it is cleared: @@ -40,7 +40,7 @@ To change the visibility of your pipelines and related features: and the **CI/CD** menu items are visible only to project members (Reporter or higher). Other users, including guest users, can only view the status of pipelines and jobs, and only when viewing merge requests or commits. - - For **Internal** projects, pipelines are visible to all authenticated users except [external users](../../user/admin_area/external_users.md). + - For **Internal** projects, pipelines are visible to all authenticated users except [external users](../../administration/external_users.md). Related features are visible only to project members (Reporter or higher). - For **Private** projects, pipelines and related features are visible to project members (Reporter or higher) only. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index ec58491e604..80bb9f63413 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -68,7 +68,7 @@ In this file, you define: To create a `.gitlab-ci.yml` file: -1. On the left sidebar, select **Repository > Files**. +1. On the left sidebar, select **Code > Repository**. 1. Above the file list, select the branch you want to commit to. If you're not sure, leave `master` or `main`. Then select the plus icon (**{plus}**) and **New file**: @@ -117,7 +117,7 @@ The pipeline starts and runs the jobs you defined in the `.gitlab-ci.yml` file. Now take a look at your pipeline and the jobs within. -1. Go to **CI/CD > Pipelines**. A pipeline with three stages should be displayed: +1. Go to **Build > Pipelines**. A pipeline with three stages should be displayed:  diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index ddb8f0aaa61..e1be6118bab 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -204,7 +204,7 @@ and is planned for removal in 17.0. This change is a breaking change. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `anonymous_visual_review_feedback`. +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `anonymous_visual_review_feedback`. With Visual Reviews, members of any team (Product, Design, Quality, and so on) can provide feedback comments through a form in your review apps. The comments are added to the merge request that triggered the review app. @@ -255,7 +255,7 @@ to replace those values at runtime when each review app is created: variable. - `data-merge-request-id` is the merge request ID, which can be found by the `CI_MERGE_REQUEST_IID` variable. `CI_MERGE_REQUEST_IID` is available only if - [`only: [merge_requests]`](../pipelines/merge_request_pipelines.md) + [`rules:if: $CI_PIPELINE_SOURCE == "merge_request_event`](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs) is used and the merge request is created. - `data-mr-url` is the URL of the GitLab instance and is the same for all review apps. diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index c365cc934db..9424f8ea846 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -140,45 +140,6 @@ and then [register](https://docs.gitlab.com/runner/commands/#gitlab-runner-regis To verify that the previous authentication token has been revoked, use the [Runners API](../../api/runners.md#verify-authentication-for-a-registered-runner). -## Determine the IP address of a runner - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/17286) in GitLab 10.6. - -It may be useful to know the IP address of a runner so you can troubleshoot -issues with that runner. GitLab stores and displays the IP address by viewing -the source of the HTTP requests it makes to GitLab when polling for jobs. The -IP address is always kept up to date so if the runner IP changes it -automatically updates in GitLab. - -The IP address for shared runners and project runners can be found in -different places. - -### Determine the IP address of a shared runner - -Prerequisite: - -- You must have administrator access to the instance. - -To determine the IP address of a shared runner: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. On the left sidebar, select **CI/CD > Runners**. -1. Find the runner in the table and view the **IP Address** column. - - - -### Determine the IP address of a project runner - -To can find the IP address of a runner for a project project, -you must have the Owner role for the -project. - -1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. On the details page you should see a row for **IP Address**. - - - ## Use tags to control which jobs a runner can run You must set up a runner to be able to run all the different types of jobs @@ -313,17 +274,6 @@ variables: - echo "Hello runner selector feature" ``` -## Runner statuses - -A runner can have one of the following statuses. - -| Status | Description | -|---------|-------------| -| `online` | The runner has contacted GitLab within the last 2 hours and is available to run jobs. | -| `offline` | The runner has not contacted GitLab in more than 2 hours and is not available to run jobs. Check the runner to see if you can bring it online. | -| `stale` | The runner has not contacted GitLab in more than 3 months. If the runner was created more than 3 months ago, but it never contacted the instance, it is also considered **stale**. | -| `never_contacted` | The runner has never contacted GitLab. To make the runner contact GitLab, run `gitlab-runner run`. | - ## Configure runner behavior with variables You can use [CI/CD variables](../variables/index.md) to configure runner Git behavior @@ -948,91 +898,6 @@ setting. be read from concurrency, so no additional memory is allocated in addition to what the decompressor requires. This defaults to the number of CPUs available. -## Clean up stale runners **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363012) in GitLab 15.1. - -You can clean up group runners that have been inactive for more than three months. - -Group runners are those that were created at the group level. - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Settings > CI/CD**. -1. Expand **Runners**. -1. Turn on the **Enable stale runner cleanup** toggle. - -### View stale runner cleanup logs - -You can check the [Sidekiq logs](../../administration/logs/index.md#sidekiq-logs) to see the cleanup result. In Kibana you can use the following query: - -```json -{ - "query": { - "match_phrase": { - "json.class.keyword": "Ci::Runners::StaleGroupRunnersPruneCronWorker" - } - } -} -``` - -Filter entries where stale runners were removed: - -```json -{ - "query": { - "range": { - "json.extra.ci_runners_stale_group_runners_prune_cron_worker.total_pruned": { - "gte": 1, - "lt": null - } - } - } -} -``` - -## Determine which runners need to be upgraded **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365078) in GitLab 15.3. - -The version of GitLab Runner used by your runners should be -[kept up-to-date](https://docs.gitlab.com/runner/index.html#gitlab-runner-versions). - -To determine which runners need to be upgraded: - -1. View the list of runners: - - For a group: - 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. - 1. Select **Build > Runners**. - - For the instance: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. Select **CI/CD > Runners**. - -1. Above the list of runners, view the status: - - **Outdated - recommended**: The runner does not have the latest `PATCH` version, which may make it vulnerable - to security or high severity bugs. Or, the runner is one or more `MAJOR` versions behind your GitLab instance, so some features may not be available or work properly. - - **Outdated - available**: Newer versions are available but upgrading is not critical. - -1. Filter the list by status to view which individual runners need to be upgraded. - -## View statistics for runner performance **(ULTIMATE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377963) in GitLab 15.8. - -As an administrator, you can view runner statistics to learn about the performance of your runner fleet. - -1. Select **Main menu > Admin**. -1. On the left sidebar, select **CI/CD > Runners**. -1. Select **View metrics**. - -The **Median job queued time** value is calculated by sampling the queue duration of the -most recent 100 jobs that were run by Instance runners. Jobs from only the latest 5000 -runners are considered. - -The median is a value that falls into the 50th percentile: half of the jobs -queued for longer than the median value, and half of the jobs queued for less than the -median value. - ## Authentication token security > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30942) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `enforce_runner_token_expires_at`. Disabled by default. diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 19c5be88c1b..5427911e1ce 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -17,15 +17,10 @@ Your jobs can run on: - [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)) - [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/experiment-beta-support.md#beta)) -Refer to the Compute [cost factor](../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. -The number of minutes you can use on these runners depends on the [maximum number of units of compute](../pipelines/cicd_minutes.md) +Refer to the compute minutes [cost factor](../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the machine type based on size. +The number of minutes you can use on these runners depends on the [maximum number of compute minutes](../pipelines/cicd_minutes.md) in your [subscription plan](https://about.gitlab.com/pricing/). -[Untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) jobs automatically run in containers -on the `small` Linux runners. - -The objective is to make 90% of CI jobs start executing in 120 seconds or less. The error rate should be less than 0.5%. - ## How SaaS runners work When you use SaaS runners: @@ -35,6 +30,9 @@ When you use SaaS runners: - The virtual machine where your job runs has `sudo` access with no password. - The storage is shared by the operating system, the image with pre-installed software, and a copy of your cloned repository. This means that the available free disk space for your jobs to use is reduced. +- [Untagged](../../ci/runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run) jobs automatically run in containers +on the `small` Linux runners. +- The objective is to make 90% of CI jobs start executing in 120 seconds or less. The error rate target will be less than 0.5%. NOTE: Jobs handled by SaaS runners on GitLab.com **time out after 3 hours**, regardless of the timeout configured in a project. diff --git a/doc/ci/runners/new_creation_workflow.md b/doc/ci/runners/new_creation_workflow.md index 0c43fd21d1d..fce4b57b2a1 100644 --- a/doc/ci/runners/new_creation_workflow.md +++ b/doc/ci/runners/new_creation_workflow.md @@ -61,7 +61,7 @@ workflow will break. To avoid a broken workflow, you must: -1. [Create a shared runner](register_runner.md#for-a-shared-runner) and obtain the authentication token. +1. [Create a shared runner](runners_scope.md#create-a-shared-runner-with-an-authentication-token) and obtain the authentication token. 1. Replace the registration token in your runner registration workflow with the authentication token. @@ -80,10 +80,25 @@ The `gitlab-runner register` command will stop accepting registration tokens and authentication tokens generated in the GitLab runners administration page. These authentication tokens are recognizable by their `glrt-` prefix. +When you create a runner in the GitLab UI, you specify configuration values that were previously command-line options +prompted by the `gitlab-runner register` command. +These command-line options have been [deprecated](../../update/deprecations.md#registration-tokens-and-server-side-runner-arguments-in-post-apiv4runners-endpoint). + +If you specify an authentication token with: + +- the `--token` command-line option, the `gitlab-runner register` command does not accept the configuration values. +- the `--registration-token` command-line option, the `gitlab-runner register` command ignores the configuration values. + +Authentication tokens have the prefix, `glrt-`. + +To ensure minimal disruption to your automation workflow, +[legacy-compatible registration processing](https://docs.gitlab.com/runner/register/#legacy-compatible-registration-processing) +triggers if an authentication token is specified in the legacy parameter `--registration-token`. + Example command for GitLab 15.9: ```shell -gitlab-runner register +gitlab-runner register \ --non-interactive \ --executor "shell" \ --url "https://gitlab.com/" \ @@ -101,7 +116,7 @@ In GitLab 15.11 and later, these attributes are no longer accepted as arguments The following example shows the new command: ```shell -gitlab-runner register +gitlab-runner register \ --non-interactive \ --executor "shell" \ --url "https://gitlab.com/" \ diff --git a/doc/ci/runners/register_runner.md b/doc/ci/runners/register_runner.md index 9c63850e38a..bca11682258 100644 --- a/doc/ci/runners/register_runner.md +++ b/doc/ci/runners/register_runner.md @@ -1,128 +1,11 @@ --- -stage: Verify -group: Runner -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'runners_scope.md' +remove_date: '2023-09-21' --- -# Generate runner tokens **(FREE)** +This document was moved to [another location](runners_scope.md). -To register a runner, you can use either: - -- An authentication token assigned to the runner when you create the runner in the UI. The runner uses the token to authenticate with GitLab when picking up jobs from the job queue. -- A registration token (deprecated). - -## Generate an authentication token - -Registration with an authentication token is available for all runners. - -NOTE: -The token only displays in the UI for a short period of time during registration. - -### For a shared runner - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383139) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_admin` [flag](../../administration/feature_flags.md) -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/389269) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `create_runner_workflow_for_admin`. - -Prerequisites: - -- You must be an administrator. - -To generate an authentication token for a shared runner: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. On the left sidebar, select **CI/CD > Runners**. -1. Select **New instance runner**. -1. Select a platform. -1. Optional. Enter configurations for the runner. -1. Select **Submit**. -1. Follow the instructions to register the runner from the command line. - -You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. - -### For a group runner - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/393919) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `create_runner_workflow_for_namespace`. - -Prerequisites: - -- You must have the Owner role for the group. - -To generate an authentication token for a group runner: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Build > Runners**. -1. Select **New group runner**. -1. Select a platform. -1. Optional. Enter configurations for the runner. -1. Select **Submit**. -1. Follow the instructions to register the runner from the command line. - -You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. - -### For a project runner - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/393919) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `create_runner_workflow_for_namespace`. - -Prerequisites: - -- You must have the Maintainer role for the project. - -To generate an authentication token for a project runner: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > CI/CD**. -1. Expand the **Runners** section. -1. Select **New project runner**. -1. Select a platform. -1. Optional. Enter configurations for the runner. -1. Select **Submit**. -1. Follow the instructions to register the runner from the command line. - -You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. - -## Generate a registration token (deprecated) - -WARNING: -The ability to pass a runner registration token, and support for certain configuration arguments was -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6. Authentication tokens -should be used instead to register runners. Registration tokens, and support for certain configuration -arguments, will be removed in GitLab 17.0. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). -The configuration arguments disabled for `glrt-` tokens will be `--locked`, `--access-level`, -`--run-untagged`, `--maximum-timeout`, `--paused`, `--tag-list`, and `--maintenance-note`. This change is a breaking -change. - -### For a shared runner - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **CI/CD > Runners**. -1. Select **Register an instance runner**. -1. Copy the registration token. - -### For a group runner - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Build > Runners**. -1. Copy the registration token. - -### For a project runner - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > CI/CD**. -1. Expand the **Runners** section. -1. Copy the registration token. +<!-- This redirect file can be deleted after <YYYY-MM-DD>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index e7b764025c9..840eb7fe93b 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -5,9 +5,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# The scope of runners **(FREE)** +# Manage runners -Runners are available based on who you want to have access: +GitLab Runner has the following types of runners, which are available based on who you want to have access: - [Shared runners](#shared-runners) are available to all groups and projects in a GitLab instance. - [Group runners](#group-runners) are available to all projects and subgroups in a group. @@ -24,26 +24,71 @@ multiple projects. If you are using a self-managed instance of GitLab: -- Your administrator can install and register shared runners by - going to your project's **Settings > CI/CD**, expanding **Runners**, - and selecting **Show runner installation instructions**. - These instructions are also available [in the documentation](https://docs.gitlab.com/runner/install/index.html). +- Your administrator can [install GitLab Runner](https://docs.gitlab.com/runner/install/index.html) and register a shared runner. - The administrator can also configure a maximum number of shared runner - [units of compute for each group](../pipelines/cicd_minutes.md#set-the-compute-quota-for-a-specific-namespace). + [compute minutes for each group](../pipelines/cicd_minutes.md#set-the-compute-quota-for-a-specific-namespace). If you are using GitLab.com: - You can select from a list of [shared runners that GitLab maintains](index.md). -- The shared runners consume the [units of compute](../pipelines/cicd_minutes.md) +- The shared runners consume the [compute minutes](../pipelines/cicd_minutes.md) included with your account. +### Create a shared runner with an authentication token + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383139) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_admin` [flag](../../administration/feature_flags.md) +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/389269) in GitLab 16.0. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/415447) in GitLab 16.2. Feature flag `create_runner_workflow_for_admin` removed. + +Prerequisites: + +- You must be an administrator. + +When you create a runner, it is assigned an authentication token that you use to register it. The runner uses the token to authenticate with GitLab when picking up jobs from the job queue. + +To create a shared runner: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Select **New instance runner**. +1. Select a platform. +1. Optional. Enter configurations for the runner. +1. Select **Submit**. +1. Follow the on-screen instructions to register the runner from the command line. + +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + +NOTE: +The authentication token displays in the UI for only a short period of time during registration. + +### Create a shared runner with a registration token (deprecated) + +WARNING: +The ability to pass a runner registration token, and support for certain configuration arguments was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. Authentication tokens +should be used instead. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). + +Prerequisites: + +- You must be an administrator. + +To create a shared runner: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **CI/CD > Runners**. +1. Select **Register an instance runner**. +1. Copy the registration token. +1. [Register the runner](https://docs.gitlab.com/runner/register/). + ### Enable shared runners for a project On GitLab.com, [shared runners](index.md) are enabled in all projects by default. On self-managed instances of GitLab, an administrator can -[enable them for all new projects](../../user/admin_area/settings/continuous_integration.md#enable-shared-runners-for-new-projects). +[enable them for all new projects](../../administration/settings/continuous_integration.md#enable-shared-runners-for-new-projects). For existing projects, an administrator must [install](https://docs.gitlab.com/runner/install/index.html) and @@ -94,10 +139,6 @@ To disable shared runners for a group: 1. Optional. To allow shared runners to be enabled for individual projects or subgroups, select **Allow projects and subgroups to override the group setting**. -NOTE: -If you re-enable the shared runners for a group after you disable them, a user with the -Owner or Maintainer role must manually change this setting for each project subgroup or project. - ### How shared runners pick jobs Shared runners process jobs by using a fair usage queue. This queue prevents @@ -143,11 +184,43 @@ to have access to a set of runners. Group runners process jobs by using a first in, first out ([FIFO](https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics))) queue. -### Create a group runner +### Create a group runner with an authentication token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19819) in GitLab 14.10, path changed from **Settings > CI/CD > Runners**. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/393919) in GitLab 16.0. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/415447) in GitLab 16.2. Feature flag `create_runner_workflow_for_admin` removed. + +Prerequisites: + +- You must have the Owner role for the group. You can create a group runner for your self-managed GitLab instance or for GitLab.com. +When you create a runner, it is assigned an authentication token that you use to register it. The runner uses the token to authenticate with GitLab when picking up jobs from the job queue. + +To create a group runner: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Build > Runners**. +1. Select **New group runner**. +1. Select a platform. +1. Optional. Enter configurations for the runner. +1. Select **Submit**. +1. Follow the on-screen instructions to register the runner from the command line. + +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + +NOTE: +The authentication token displays in the UI for only a short period of time during registration. + +### Create a group runner with a registration token (deprecated) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19819) in GitLab 14.10, path changed from **Settings > CI/CD > Runners**. + +WARNING: +The ability to pass a runner registration token, and support for certain configuration arguments was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. Authentication tokens +should be used instead. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). + You must have the Owner role for the group. To create a group runner: @@ -224,6 +297,48 @@ You must have the Owner role for the group. You must remove it from each project first. 1. On the confirmation dialog, select **OK**. +### Clean up stale group runners **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363012) in GitLab 15.1. + +You can clean up group runners that have been inactive for more than three months. + +Group runners are those that were created at the group level. + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > CI/CD**. +1. Expand **Runners**. +1. Turn on the **Enable stale runner cleanup** toggle. + +#### View stale runner cleanup logs + +You can check the [Sidekiq logs](../../administration/logs/index.md#sidekiq-logs) to see the cleanup result. In Kibana you can use the following query: + +```json +{ + "query": { + "match_phrase": { + "json.class.keyword": "Ci::Runners::StaleGroupRunnersPruneCronWorker" + } + } +} +``` + +Filter entries where stale runners were removed: + +```json +{ + "query": { + "range": { + "json.extra.ci_runners_stale_group_runners_prune_cron_worker.total_pruned": { + "gte": 1, + "lt": null + } + } + } +} +``` + ## Project runners Use _project runners_ when you want to use runners for specific projects. For example, @@ -241,9 +356,40 @@ NOTE: Project runners do not get shared with forked projects automatically. A fork *does* copy the CI/CD settings of the cloned repository. -### Create a project runner +### Create a project runner with an authentication token + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383143) in GitLab 15.10. Deployed behind the `create_runner_workflow_for_namespace` [flag](../../administration/feature_flags.md). Disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/393919) in GitLab 16.0. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/415447) in GitLab 16.2. Feature flag `create_runner_workflow_for_admin` removed. + +Prerequisites: + +- You must have the Maintainer role for the project. + +You can create a project runner for your self-managed GitLab instance or for GitLab.com. When you create a runner, it is assigned an authentication token that you use to register to the runner. The runner uses the token to authenticate with GitLab when picking up jobs from the job queue. -You can create a project runner for your self-managed GitLab instance or for GitLab.com. +To create a project runner: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > CI/CD**. +1. Expand the **Runners** section. +1. Select **New project runner**. +1. Select a platform. +1. Optional. Enter configurations for the runner. +1. Select **Submit**. +1. Follow the on-screen instructions to register the runner from the command line. + +You can also [create a runner](../../api/users.md#create-a-runner) with the API to generate an authentication token. + +NOTE: +The authentication token displays in the UI for only a short period of time during registration. + +### Create a project runner with a registration token (deprecated) + +WARNING: +The ability to pass a runner registration token, and support for certain configuration arguments was +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/380872) in GitLab 15.6 and will be removed in GitLab 17.0. Authentication tokens +should be used instead. For more information, see [Migrating to the new runner registration workflow](new_creation_workflow.md). Prerequisite: @@ -284,7 +430,7 @@ You can edit a project runner from any of the projects it's enabled for. The modifications, which include unlocking and editing tags and the description, affect all projects that use the runner. -An administrator can [enable the runner for multiple projects](../../user/admin_area/settings/continuous_integration.md#enable-a-project-runner-for-multiple-projects). +An administrator can [enable the runner for multiple projects](../../administration/settings/continuous_integration.md#enable-a-project-runner-for-multiple-projects). ### Prevent a project runner from being enabled for other projects @@ -302,3 +448,95 @@ To lock or unlock a project runner: 1. Select **Edit** (**{pencil}**). 1. Select the **Lock to current projects** checkbox. 1. Select **Save changes**. + +## Runner statuses + +A runner can have one of the following statuses. + +| Status | Description | +|---------|-------------| +| `online` | The runner has contacted GitLab within the last 2 hours and is available to run jobs. | +| `offline` | The runner has not contacted GitLab in more than 2 hours and is not available to run jobs. Check the runner to see if you can bring it online. | +| `stale` | The runner has not contacted GitLab in more than 3 months. If the runner was created more than 3 months ago, but it never contacted the instance, it is also considered **stale**. | +| `never_contacted` | The runner has never contacted GitLab. To make the runner contact GitLab, run `gitlab-runner run`. | + +## View statistics for runner performance **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/377963) in GitLab 15.8. + +As an administrator, you can view runner statistics to learn about the performance of your runner fleet. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Select **View metrics**. + +The **Median job queued time** value is calculated by sampling the queue duration of the +most recent 100 jobs that were run by Instance runners. Jobs from only the latest 5000 +runners are considered. + +The median is a value that falls into the 50th percentile: half of the jobs +queued for longer than the median value, and half of the jobs queued for less than the +median value. + +## Determine which runners need to be upgraded **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/365078) in GitLab 15.3. + +The version of GitLab Runner used by your runners should be +[kept up-to-date](https://docs.gitlab.com/runner/index.html#gitlab-runner-versions). + +To determine which runners need to be upgraded: + +1. View the list of runners: + - For a group: + 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. + 1. Select **Build > Runners**. + - For the instance: + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. Select **CI/CD > Runners**. + +1. Above the list of runners, view the status: + - **Outdated - recommended**: The runner does not have the latest `PATCH` version, which may make it vulnerable + to security or high severity bugs. Or, the runner is one or more `MAJOR` versions behind your GitLab instance, so some features may not be available or work properly. + - **Outdated - available**: Newer versions are available but upgrading is not critical. + +1. Filter the list by status to view which individual runners need to be upgraded. + +## Determine the IP address of a runner + +It may be useful to know the IP address of a runner so you can troubleshoot +issues with that runner. GitLab stores and displays the IP address by viewing +the source of the HTTP requests it makes to GitLab when polling for jobs. The +IP address is always kept up to date so if the runner IP changes it +automatically updates in GitLab. + +The IP address for shared runners and project runners can be found in +different places. + +### Determine the IP address of a shared runner + +Prerequisite: + +- You must have administrator access to the instance. + +To determine the IP address of a shared runner: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **CI/CD > Runners**. +1. Find the runner in the table and view the **IP Address** column. + + + +### Determine the IP address of a project runner + +To can find the IP address of a runner for a project project, +you must have the Owner role for the +project. + +1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. +1. On the details page you should see a row for **IP Address**. + + diff --git a/doc/ci/runners/saas/gpu_saas_runner.md b/doc/ci/runners/saas/gpu_saas_runner.md index 7b83f6593a0..a05b162ac84 100644 --- a/doc/ci/runners/saas/gpu_saas_runner.md +++ b/doc/ci/runners/saas/gpu_saas_runner.md @@ -15,9 +15,9 @@ GitLab provides GPU-enabled runners only on Linux. For more information about ho The following machine types are available for GPU-enabled runners on Linux x86-64. -| Runner Tag | vCPUs | Memory | Storage | GPU | -|----------------------------------------|-------|--------|---------|------------------------------------| -| `saas-linux-medium-amd64-gpu-standard` | 4 | 16 GB | 50 GB | 1 Nvidia Tesla T4 GPU (or similar) | +| Runner Tag | vCPUs | Memory | Storage | GPU | GPU Memory | +|----------------------------------------|-------|--------|---------|--------------------------------|------------| +| `saas-linux-medium-amd64-gpu-standard` | 4 | 15 GB | 50 GB | 1 Nvidia Tesla T4 (or similar) | 16 GB | ## Container images with GPU drivers diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index d95fc701056..95917bbc300 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -81,16 +81,12 @@ If you want to [contribute to GitLab](https://about.gitlab.com/community/contrib `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners, dedicated for GitLab projects and related community forks. These runners are backed by the same machine type as our `small` runners. -Unlike the normal SaaS runners on Linux, each virtual machine is re-used up to 40 times. +Unlike the most commonly used SaaS runners on Linux, each virtual machine is re-used up to 40 times. As we want to encourage people to contribute, these runners are free of charge. -<!--- start_remove The following content will be removed on remove_date: '2023-08-22' --> - -## Pre-clone script (removed) +## Pre-clone script (deprecated) This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/391896) in GitLab 15.9 -and [removed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29405) in 16.0. +and [will be removed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29405) in 17.0. Use [`pre_get_sources_script`](../../../ci/yaml/index.md#hookspre_get_sources_script) instead. - -<!--- end_remove --> diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 836a14d7521..a559fc7d53e 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -45,7 +45,7 @@ Each image runs a specific version of macOS and Xcode. | (none, awaiting macOS 13) | `beta` | NOTE: -Each time you run a job that requires tooling or dependencies not available in the base image, those items must be added to the newly provisioned build VM increasing the total job duration. +If your job requires tooling or dependencies not available in our available images, those can only be installed in the job execution. ## Image update policy @@ -88,9 +88,6 @@ test: - echo "running scripts in the test job" ``` -NOTE: -You can specify a different Xcode image to run a job. To do so, replace the value for the `image` keyword with the value of the [virtual machine image name](#supported-macos-images) from the list of available images. The default value is our latest image. - ## Code signing iOS Projects with fastlane Before you can integrate GitLab with Apple services, install to a device, or deploy to the Apple App Store, you must [code sign](https://developer.apple.com/support/code-signing/) your application. @@ -106,22 +103,22 @@ Related topics: - [Code Signing Best Practice Guide](https://codesigning.guide/) - [fastlane authentication with Apple Services guide](https://docs.fastlane.tools/getting-started/ios/authentication/) -## Known Limitations and Usage Constraints - -- If the VM image does not include the specific software version you need for your job, then the job execution time will increase as the required software needs to be fetched and installed. -- At this time, it is not possible to bring your own OS image. -- The keychain for user `gitlab` is not publicly available. You must create a keychain instead. - ## Optimizing Homebrew By default, Homebrew checks for updates at the start of any operation. Homebrew has a -release cycle that may be more frequent than the GitLab MacOS image release cycle. This +release cycle that may be more frequent than the GitLab macOS image release cycle. This difference in release cycles may cause steps that call `brew` to take extra time to complete while Homebrew makes updates. -To reduce build time due to unintended Homebrew updates, set the `HOMEBREW_NO_AUTO_UPDATE` variable in `.gitlab-ci.yml` : +To reduce build time due to unintended Homebrew updates, set the `HOMEBREW_NO_AUTO_UPDATE` variable in `.gitlab-ci.yml`: ```yaml variables: HOMEBREW_NO_AUTO_UPDATE: 1 ``` + +## Known issues and usage constraints + +- If the VM image does not include the specific software version you need for your job, the required software must be fetched and installed. This causes an increase in job execution time. +- It is not possible to bring your own OS image. +- The keychain for user `gitlab` is not publicly available. You must create a keychain instead. diff --git a/doc/ci/runners/saas/windows_saas_runner.md b/doc/ci/runners/saas/windows_saas_runner.md index fa981bddff3..8ec44b8c275 100644 --- a/doc/ci/runners/saas/windows_saas_runner.md +++ b/doc/ci/runners/saas/windows_saas_runner.md @@ -12,7 +12,7 @@ the Google Cloud Platform. This solution uses an developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). These SaaS runners are in [Beta](../../../policy/experiment-beta-support.md#beta) -and aren't recomended for production workloads. +and aren't recommended for production workloads. We want to keep iterating to get Windows runners in a stable state and [generally available](../../../policy/experiment-beta-support.md#generally-available-ga). @@ -57,16 +57,15 @@ Below is a sample `.gitlab-ci.yml` file that shows how to start using the runner tags: - shared-windows - windows-1809 + before_script: + - Set-Variable -Name "time" -Value (date -Format "%H:%m") + - echo ${time} + - echo "started by ${GITLAB_USER_NAME}" stages: - build - test -before_script: - - Set-Variable -Name "time" -Value (date -Format "%H:%m") - - echo ${time} - - echo "started by ${GITLAB_USER_NAME}" - build: extends: - .shared_windows_runners diff --git a/doc/ci/secrets/convert-to-id-tokens.md b/doc/ci/secrets/convert-to-id-tokens.md new file mode 100644 index 00000000000..e9f0d0ef5b0 --- /dev/null +++ b/doc/ci/secrets/convert-to-id-tokens.md @@ -0,0 +1,203 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: tutorial +--- + +# Tutorial: Update HashiCorp Vault configuration to use ID Tokens **(PREMIUM)** + +This tutorial demonstrates how to convert your existing CI/CI secrets configuration to use [ID Tokens](../secrets/id_token_authentication.md). + +The `CI_JOB_JWT` variables are deprecated, but updating to ID tokens requires some important configuration changes to work with Vault. If you have more than a handful of jobs, converting everything at once is a daunting task. + +From GitLab 15.9 to 15.11, [enable the automatic ID token authentication](../secrets/id_token_authentication.md#enable-automatic-id-token-authentication-deprecated) +setting to enable ID Tokens and disable `CI_JOB_JWT` tokens. + +In GitLab 16.0 and later you can use ID tokens without any settings changes. +Jobs that use `secrets:vault` automatically do not have `CI_JOB_JWT` tokens available, +Jobs that don't use `secrets:vault` can still use `CI_JOB_JWT` tokens. + +This tutorial will focus on v16 onwards, if you are running a slightly older version you will need to toggle the `Limit JSON Web Token (JWT) access` setting as appropriate. + +To update your vault configuration to use ID tokens: + +1. [Create a second JWT authentication path in Vault](#create-a-second-jwt-authentication-path-in-vault) +1. [Recreate roles to use the new authentication path](#recreate-roles-to-use-the-new-authentication-path) +1. [Update your CI/CD Jobs](#update-your-cicd-jobs) + +## Prerequisites + +This tutorial assumes you are familiar with GitLab CI/CD and Vault. + +To follow along, you must have: + +- An instance running GitLab 15.9 or later, or be on GitLab.com. +- A Vault server that you are already using. +- CI/CD jobs retrieving secrets from Vault with `CI_JOB_JWT`. + +In the examples below, replace `vault.example.com` with the URL of your Vault server, +and `gitlab.example.com` with the URL of your GitLab instance. + +## Create a second JWT authentication path in Vault + +As part of the transition from `CI_JOB_JWT` to ID tokens, you must update the `bound_issuer` in Vault to include `https://`: + +```shell +$ vault write auth/jwt/config \ + jwks_url="https://gitlab.example.com/-/jwks" \ + bound_issuer="https://gitlab.example.com" +``` + +After you make this change, jobs that use `CI_JOB_JWT` start to fail. + +You can create multiple authentication paths in Vault, which enable you to transition to IT Tokens on a project by job basis without disruption. + +1. Configure a new authentication path with the name `jwt_v2`, run: + + ```shell + vault auth enable -path jwt_v2 jwt + ``` + + You can choose a different name, but the rest of these examples assume you used `jwt_v2`, so update the examples as needed. + +1. Configure the new authentication path for your instance: + + ```shell + $ vault write auth/jwt_v2/config \ + jwks_url="https://gitlab.example.com/-/jwks" \ + bound_issuer="https://gitlab.example.com" + ``` + +## Recreate roles to use the new authentication path + +Roles are bound to a specific authentication path so you need to add new roles for each job. + +1. Recreate the role for staging named `myproject-staging`: + + ```shell + $ vault write auth/jwt_v2/role/myproject-staging - <<EOF + { + "role_type": "jwt", + "policies": ["myproject-staging"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims": { + "project_id": "22", + "ref": "master", + "ref_type": "branch" + } + } + EOF + ``` + +1. Recreate the role for production named `myproject-production`: + + ```shell + $ vault write auth/jwt_v2/role/myproject-production - <<EOF + { + "role_type": "jwt", + "policies": ["myproject-production"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims_type": "glob", + "bound_claims": { + "project_id": "22", + "ref_protected": "true", + "ref_type": "branch", + "ref": "auto-deploy-*" + } + } + EOF + ``` + +You only need to update `jwt` to `jwt_v2` in the `vault` command, do not change the `role_type` inside the role. + +## Update your CI/CD Jobs + +Vault has two different [KV Secrets Engines](https://developer.hashicorp.com/vault/docs/secrets/kv) and the version you are using impacts how you define secrets in CI/CD. + +Check the [Which Version is my Vault KV Mount?](https://support.hashicorp.com/hc/en-us/articles/4404288741139-Which-Version-is-my-Vault-KV-Mount-) article on HashiCorp's support portal to check your Vault server. + +Also, if needed you can review the CI/CD documentation for: + +- [`secrets:`](../yaml/index.md#secrets) +- [`id_tokens:`](../yaml/index.md#id_tokens) + +The following examples show how to obtain the staging database password written to the `password` field in `secret/myproject/staging/db` + +### KV Secrets Engine v1 + +The [`secrets:vault`](../yaml/index.md#secretsvault) keyword defaults to v2 of the KV Mount, so you need to explicitly configure the job to use the v1 engine: + +```yaml +job: + variables: + VAULT_SERVER_URL: https://vault.example.com + VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_ROLE: myproject-staging + id_tokens: + VAULT_ID_TOKEN: + aud: https://gitlab.example.com + secrets: + PASSWORD: + vault: + engine: + name: kv-v1 + path: secret + field: password + path: myproject/staging/db + file: false +``` + +Both `VAULT_SERVER_URL` and `VAULT_AUTH_PATH` can be [defined as project or group CI/CD variables](../../ci/variables/index.md#define-a-cicd-variable-in-the-ui), +if preferred. + +We use [`secrets:file:false`](../../ci/yaml/index.md#secretsfile) because ID tokens place secrets in a file by default, but we need it to work as a regular variable to match the old behavior. + +### KV Secrets Engine v2 + +There are two formats you can use for the v2 engine. + +Long format: + +```yaml +job: + variables: + VAULT_SERVER_URL: https://vault.example.com + VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_ROLE: myproject-staging + id_tokens: + VAULT_ID_TOKEN: + aud: https://gitlab.example.com + secrets: + PASSWORD: + vault: + engine: + name: kv-v2 + path: secret + field: password + path: myproject/staging/db + file: false +``` + +This is the same as the example for the v1 engine but `secrets:vault:engine:name:` is set to `kv-v2` to match the engine. + +You can also use a short format: + +```yaml +job: + variables: + VAULT_SERVER_URL: https://vault.example.com + VAULT_AUTH_PATH: jwt_v2 + VAULT_AUTH_ROLE: myproject-staging + id_tokens: + VAULT_ID_TOKEN: + aud: https://gitlab.example.com + secrets: + PASSWORD: + vault: myproject/staging/db/password@secret + file: false +``` + +After you commit the updated CI/CD configuration, your jobs will be fetching secrets with ID Tokens, congratulations! diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md index 16b94fed4d3..509bb6f07cf 100644 --- a/doc/ci/secrets/id_token_authentication.md +++ b/doc/ci/secrets/id_token_authentication.md @@ -74,7 +74,8 @@ The token also includes custom claims provided by GitLab: | `runner_id` | Always | ID of the runner executing the job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | | `runner_environment` | Always | The type of runner used by the job. Can be either `gitlab-hosted` or `self-hosted`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | | `sha` | Always | The commit SHA for the job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.0. | -| `ci_config_ref_uri` | Always | The ref path to the top-level pipeline definition, for example, `gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.1 behind the `ci_jwt_v2_ref_uri_claim` feature flag. This claim is `null` unless the pipeline definition is located in the same project. | +| `ci_config_ref_uri` | Always | The ref path to the top-level pipeline definition, for example, `gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.2. This claim is `null` unless the pipeline definition is located in the same project. | +| `ci_config_sha` | Always | Git commit SHA for the `ci_config_ref_uri`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404722) in GitLab 16.2. This claim is `null` unless the pipeline definition is located in the same project. | ```json { @@ -103,6 +104,7 @@ The token also includes custom claims provided by GitLab: "runner_environment": "self-hosted", "sha": "714a629c0b401fdce83e847fc9589983fc6f46bc", "ci_config_ref_uri": "gitlab.example.com/my-group/my-project//.gitlab-ci.yml@refs/heads/main", + "ci_config_sha": "714a629c0b401fdce83e847fc9589983fc6f46bc", "jti": "235b3a54-b797-45c7-ae9a-f72d7bc6ef5b", "iss": "https://gitlab.example.com", "iat": 1681395193, @@ -139,6 +141,9 @@ manual_authentication: You can use ID tokens to automatically fetch secrets from HashiCorp Vault with the [`secrets`](../yaml/index.md#secrets) keyword. +If you previously used `CI_JOB_JWT` to fetch secrets from Vault, learn how to switch +to ID tokens with the [Update HashiCorp Vault configuration to use ID Tokens](convert-to-id-tokens.md) tutorial. + ### Configure automatic ID Token authentication If one ID token is defined, the `secrets` keyword automatically uses it to authenticate with Vault. For example: diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index 2e7e89fc601..f4c90934e06 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -145,13 +145,11 @@ default: image: name: ruby:2.6 entrypoint: ["/bin/bash"] - services: - name: my-postgres:11.7 alias: db-postgres entrypoint: ["/usr/local/bin/db-postgres"] command: ["start"] - before_script: - bundle install @@ -240,18 +238,17 @@ variables: PGDATA: "/var/lib/postgresql/data" POSTGRES_INITDB_ARGS: "--encoding=UTF8 --data-checksums" -services: - - name: postgres:11.7 - alias: db - entrypoint: ["docker-entrypoint.sh"] - command: ["postgres"] - -image: - name: ruby:2.6 - entrypoint: ["/bin/bash"] - -before_script: - - bundle install +default: + services: + - name: postgres:11.7 + alias: db + entrypoint: ["docker-entrypoint.sh"] + command: ["postgres"] + image: + name: ruby:2.6 + entrypoint: ["/bin/bash"] + before_script: + - bundle install test: script: @@ -321,28 +318,26 @@ Before the new extended Docker configuration options, you would need to: - Create your own image based on the `super/sql:latest` image. - Add the default command. -- Use the image in the job's configuration: +- Use the image in the job's configuration. - ```dockerfile - # my-super-sql:latest image's Dockerfile + - `my-super-sql:latest` image's Dockerfile: - FROM super/sql:latest - CMD ["/usr/bin/super-sql", "run"] - ``` + ```dockerfile + FROM super/sql:latest + CMD ["/usr/bin/super-sql", "run"] + ``` - ```yaml - # .gitlab-ci.yml + - In the job in the `.gitlab-ci.yml`: - services: - - my-super-sql:latest - ``` + ```yaml + services: + - my-super-sql:latest + ``` After the new extended Docker configuration options, you can set a `command` in the `.gitlab-ci.yml` file instead: ```yaml -# .gitlab-ci.yml - services: - name: super/sql:latest command: ["/usr/bin/super-sql", "run"] @@ -414,7 +409,7 @@ To enable service logging, add the `CI_DEBUG_SERVICES` variable to the project's ```yaml variables: - CI_DEBUG_SERVICES: "true" + CI_DEBUG_SERVICES: "true" ``` Accepted values are: diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index afb14bd976f..ab38d2ce934 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -23,8 +23,9 @@ For more information, see [GitLab CI/CD variables](../variables/index.md). First, in your `.gitlab-ci.yml` add: ```yaml -services: - - postgres:12.2-alpine +default: + services: + - postgres:12.2-alpine variables: POSTGRES_DB: $POSTGRES_DB diff --git a/doc/ci/ssh_keys/index.md b/doc/ci/ssh_keys/index.md index ab767697cbc..15b731f88c8 100644 --- a/doc/ci/ssh_keys/index.md +++ b/doc/ci/ssh_keys/index.md @@ -93,7 +93,7 @@ to access it. In this case, you can use an SSH key pair. # - git config --global user.name "User name" ``` - The [`before_script`](../yaml/index.md#before_script) can be set globally + The [`before_script`](../yaml/index.md#before_script) can be set as a default or per-job. 1. Make sure the private server's [SSH host keys are verified](#verifying-the-ssh-host-keys). @@ -156,7 +156,7 @@ trusted network (ideally, from the private server itself): ssh-keyscan example.com ## Or use an IP -ssh-keyscan 1.2.3.4 +ssh-keyscan 10.0.2.2 ``` Create a new [file type CI/CD variable](../variables/index.md#use-file-type-cicd-variables) diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index edc962c9921..9667daf7501 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -8,9 +8,6 @@ type: reference # Test cases **(ULTIMATE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233479) in GitLab 13.6. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/241983) in GitLab 13.7. - Test cases in GitLab can help your teams create testing scenarios in their existing development platform. Now your Implementation and Testing teams can collaborate better, as they no longer have to @@ -30,7 +27,8 @@ Prerequisite: To create a test case in a GitLab project: -1. Go to **CI/CD > Test cases**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Test cases**. 1. Select **New test case**. You are taken to the new test case form. Here you can enter the new case's title, [description](../../user/markdown.md), attach a file, and assign [labels](../../user/project/labels.md). 1. Select **Submit test case**. You are taken to view the new test case. @@ -40,16 +38,15 @@ To create a test case in a GitLab project: You can view all test cases in the project in the test cases list. Filter the issue list with a search query, including labels or the test case's title. -Prerequisite: - -Whether you can view an test case depends on the [project visibility level](../../user/public_access.md): +Prerequisites: -- Public project: You don't have to be a member of the project. -- Private project: You must have at least the Guest role for the project. +- In a public project: You don't have to be a member of the project. +- In a private project: You must have at least the Guest role for the project. To view a test case: -1. In a project, go to **CI/CD > Test cases**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Test cases**. 1. Select the title of the test case you want to view. You are taken to the test case page.  @@ -58,11 +55,11 @@ To view a test case: You can edit a test case's title and description. -Prerequisite: +Prerequisites: - You must have at least the Reporter role. - Users demoted to the Guest role can continue to edit the test cases they created -when they were in the higher role. + when they were in the higher role. To edit a test case: @@ -83,13 +80,19 @@ To archive a test case, on the test case's page, select **Archive test case**. To view archived test cases: -1. Go to **CI/CD > Test cases**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Build > Test cases**. 1. Select **Archived**. ## Reopen an archived test case If you decide to start using an archived test case again, you can reopen it. -You must have at least the Reporter role. +Prerequisites: -To reopen an archived test case, on the test case's page, select **Reopen test case**. +- You must have at least the Reporter role. + +To reopen an archived test case: + +1. [View a test case](#view-a-test-case). +1. Select **Reopen test case**. diff --git a/doc/ci/testing/accessibility_testing.md b/doc/ci/testing/accessibility_testing.md index b03e4a23153..f6d0468c876 100644 --- a/doc/ci/testing/accessibility_testing.md +++ b/doc/ci/testing/accessibility_testing.md @@ -4,14 +4,7 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> -# Accessibility testing (deprecated) **(FREE)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25144) in GitLab 12.8. - -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/390424) in GitLab 15.9 -and is planned for removal in 17.0. This change is a breaking change. +# Accessibility testing **(FREE)** If your application offers a web interface, you can use [GitLab CI/CD](../index.md) to determine the accessibility @@ -19,7 +12,7 @@ impact of pending code changes. [Pa11y](https://pa11y.org/) is a free and open source tool for measuring the accessibility of web sites. GitLab integrates Pa11y into a -[CI job template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml). +[CI/CD job template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml). The `a11y` job analyzes a defined set of web pages and reports accessibility violations, warnings, and notices in a file named `accessibility`. @@ -29,9 +22,6 @@ As of [GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73309) ## Accessibility merge request widget -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39425) in GitLab 13.0 behind the disabled [feature flag](../../administration/feature_flags.md) `:accessibility_report_view`. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217372) in GitLab 13.1. - GitLab displays an **Accessibility Report** in the merge request widget area:  @@ -41,7 +31,7 @@ GitLab displays an **Accessibility Report** in the merge request widget area: You can run Pa11y with GitLab CI/CD using the [GitLab Accessibility Docker image](https://gitlab.com/gitlab-org/ci-cd/accessibility). -To define the `a11y` job for GitLab 12.9 and later: +To define the `a11y` job: 1. [Include](../yaml/index.md#includetemplate) the [`Accessibility.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml) @@ -71,13 +61,7 @@ The `a11y` job in your CI/CD pipeline generates these files: You can [view job artifacts in your browser](../jobs/job_artifacts.md#download-job-artifacts). NOTE: -For GitLab versions earlier than 12.9, use `include:remote` and -link to the [current template in the default branch](https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Verify/Accessibility.gitlab-ci.yml) - -NOTE: The job definition provided by the template does not support Kubernetes. You cannot pass configurations into Pa11y via CI configuration. To change the configuration, edit a copy of the template in your CI file. - -<!--- end_remove -->
\ No newline at end of file diff --git a/doc/ci/testing/browser_performance_testing.md b/doc/ci/testing/browser_performance_testing.md index 600b1a2cf4b..059cd637f9e 100644 --- a/doc/ci/testing/browser_performance_testing.md +++ b/doc/ci/testing/browser_performance_testing.md @@ -4,15 +4,8 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> # Browser Performance Testing **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3507) in GitLab 10.3. - -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/388719) in GitLab 15.9 -and is planned for removal in 17.0. This change is a breaking change. - If your application offers a web interface and you're using [GitLab CI/CD](../index.md), you can quickly determine the rendering performance impact of pending code changes in the browser. @@ -79,18 +72,13 @@ using Docker-in-Docker. URL: https://example.com ``` -WARNING: -In GitLab 13.12 and earlier, the job [was named](https://gitlab.com/gitlab-org/gitlab/-/issues/225914) `performance`. - The above example: - Creates a `browser_performance` job in your CI/CD pipeline and runs sitespeed.io against the webpage you defined in `URL` to gather key metrics. - Uses a template that doesn't work with Kubernetes clusters. If you are using a Kubernetes cluster, use [`template: Jobs/Browser-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Browser-Performance-Testing.gitlab-ci.yml) - instead. -- Uses a CI/CD template that is included in all GitLab installations since 12.4. If you are using - GitLab 12.3 or earlier, you must [add the configuration manually](#gitlab-versions-132-and-earlier). + instead. The template uses the [GitLab plugin for sitespeed.io](https://gitlab.com/gitlab-org/gl-performance), and it saves the full HTML sitespeed.io report as a [Browser Performance report artifact](../yaml/artifacts_reports.md#artifactsreportsbrowser_performance) @@ -120,8 +108,6 @@ browser_performance: ### Configuring degradation threshold -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27599) in GitLab 13.0. - You can configure the sensitivity of degradation alerts to avoid getting alerts for minor drops in metrics. This is done by setting the `DEGRADATION_THRESHOLD` CI/CD variable. In the example below, the alert only shows up if the `Total Score` metric degrades by 5 points or more: @@ -147,9 +133,9 @@ be extended for dynamic environments, but a few extra steps are required: 1. In the `review` job: 1. Generate a URL list file with the dynamic URL. 1. Save the file as an artifact, for example with `echo $CI_ENVIRONMENT_URL > environment_url.txt` - in your job's `script`. + in your job's `script`. 1. Pass the list as the URL environment variable (which can be a URL or a file containing URLs) - to the `browser_performance` job. + to the `browser_performance` job. 1. You can now run the sitespeed.io container against the desired hostname and paths. @@ -174,10 +160,10 @@ review: artifacts: paths: - environment_url.txt - only: - - branches - except: - - master + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + when: never + - if: $CI_COMMIT_BRANCH browser_performance: dependencies: @@ -185,63 +171,3 @@ browser_performance: variables: URL: environment_url.txt ``` - -### GitLab versions 13.2 and earlier - -Browser Performance Testing has gone through several changes since its introduction. -In this section we detail these changes and how you can run the test based on your -GitLab version: - -- In 13.2 the feature was renamed from `Performance` to `Browser Performance` with additional - template CI/CD variables. -- In GitLab 12.4 [a job template was made available](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Browser-Performance.gitlab-ci.yml). -- For 11.5 to 12.3 no template is available and the job has to be defined manually as follows: - - ```yaml - performance: - stage: performance - image: docker:git - variables: - URL: https://example.com - SITESPEED_VERSION: 14.1.0 - SITESPEED_OPTIONS: '' - services: - - docker:stable-dind - script: - - mkdir gitlab-exporter - - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/1.1.0/index.js - - mkdir sitespeed-results - - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:$SITESPEED_VERSION --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL $SITESPEED_OPTIONS - - mv sitespeed-results/data/performance.json performance.json - artifacts: - paths: - - performance.json - - sitespeed-results/ - reports: - performance: performance.json - ``` - -- For 11.4 and earlier the job should be defined as follows: - - ```yaml - performance: - stage: performance - image: docker:git - variables: - URL: https://example.com - services: - - docker:stable-dind - script: - - mkdir gitlab-exporter - - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/1.1.0/index.js - - mkdir sitespeed-results - - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:6.3.1 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $URL - - mv sitespeed-results/data/performance.json performance.json - artifacts: - paths: - - performance.json - - sitespeed-results/ - ``` - -Upgrading to the latest version and using the templates is recommended, to ensure -you receive the latest updates, including updates to the sitespeed.io versions. diff --git a/doc/ci/testing/code_quality.md b/doc/ci/testing/code_quality.md index 5f6af4cb8a9..fcb5a23742a 100644 --- a/doc/ci/testing/code_quality.md +++ b/doc/ci/testing/code_quality.md @@ -51,7 +51,8 @@ Code Quality results are shown in the: > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) to GitLab Free in 13.2. Code Quality analysis results display in the merge request widget area if a report from the target -branch is available for comparison. +branch is available for comparison. The merge request widget displays Code Quality findings and resolutions that +were introduced by the changes made in the merge request.  @@ -70,13 +71,14 @@ issues are marked by an indicator beside the gutter. Hover over the marker for d ### Pipeline details view **(PREMIUM)** The full list of Code Quality violations generated by a pipeline is shown in the **Code Quality** -tab of the pipeline's details page. +tab of the pipeline's details page. The pipeline details view displays all Code Quality findings +that were found on the branch it was run on.  ### Project quality view **(ULTIMATE)** -The project quality view displays an overview of the code quality findings. The view can be found under **Analytics > CI/CD**, and requires [`project_quality_summary_page`](../../user/feature_flags.md) feature flag to be enabled for this particular project. +The project quality view displays an overview of the code quality findings. The view can be found under **Analyze > CI/CD analytics**, and requires [`project_quality_summary_page`](../../user/feature_flags.md) feature flag to be enabled for this particular project.  @@ -541,8 +543,8 @@ for more details. ### The code cannot be found and the pipeline runs always with default configuration You are probably using a private runner with the Docker-in-Docker socket-binding configuration. -You should configure Code Quality checks to run on your worker as documented in section -"[Improve Code Quality performance with private runners](#improve-code-quality-performance-with-private-runners)". +You should configure Code Quality checks to run on your worker as documented in +[Improve Code Quality performance with private runners](#improve-code-quality-performance-with-private-runners)". ### Changing the default configuration has no effect diff --git a/doc/ci/testing/load_performance_testing.md b/doc/ci/testing/load_performance_testing.md index 2897d7fe0ab..dac4dd555b0 100644 --- a/doc/ci/testing/load_performance_testing.md +++ b/doc/ci/testing/load_performance_testing.md @@ -4,15 +4,10 @@ group: Pipeline Execution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -<!--- start_remove The following content will be removed on remove_date: '2024-05-22' --> -# Load Performance Testing (deprecated) **(PREMIUM)** +# Load Performance Testing **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in GitLab 13.2. -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/388723) in GitLab 15.9 -and is planned for removal in 17.0. This change is a breaking change. - With Load Performance Testing, you can test the impact of any pending code changes to your application's backend in [GitLab CI/CD](../index.md). @@ -204,5 +199,3 @@ load_performance: rules: - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. ``` - -<!--- end_remove --> diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md index 1b1600b3d99..8527d0b712d 100644 --- a/doc/ci/testing/unit_test_report_examples.md +++ b/doc/ci/testing/unit_test_report_examples.md @@ -295,3 +295,23 @@ run unittests: junit: - report.xml ``` + +## Helm + +This example uses [Helm Unittest](https://github.com/helm-unittest/helm-unittest#docker-usage) plugin, with the `-t junit` flag to format the output to a JUnit report in XML format. + +```yaml +helm: + image: helmunittest/helm-unittest:latest + stage: test + script: + - '-t JUnit -o report.xml -f tests/*[._]test.yaml .' + artifacts: + reports: + junit: report.xml +``` + +The `-f tests/*[._]test.yaml` flag configures `helm-unittest` to look for files in the `tests/` directory that end in either: + +- `.test.yaml` +- `_test.yaml` diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 412394a24e7..a47eaaf0381 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -183,7 +183,7 @@ A part of the trigger's token displays on the right of the page, under the job d  In pipelines triggered with a trigger token, jobs are labeled as `triggered` in -**CI/CD > Jobs**. +**Build > Jobs**. ## Troubleshooting diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index e94ccfa6b2b..623589e2cbc 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -164,7 +164,7 @@ blocked the pipeline, or allowed the wrong pipeline type. ### Pipeline with many jobs fails to start -A Pipeline that has more jobs than the instance's defined [CI/CD limits](../user/admin_area/settings/continuous_integration.md#set-cicd-limits) +A Pipeline that has more jobs than the instance's defined [CI/CD limits](../administration/settings/continuous_integration.md#set-cicd-limits) fails to start. To reduce the number of jobs in your pipeline, you can split your `.gitlab-ci.yml` diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 7b6ba36e35d..bd066797ada 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -246,7 +246,7 @@ malicious code can compromise both masked and protected variables. Variable values are encrypted using [`aes-256-cbc`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) and stored in the database. This data can only be read and decrypted with a -valid [secrets file](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost). +valid [secrets file](../../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost). ### Mask a CI/CD variable @@ -882,3 +882,22 @@ WARNING: If you add `CI_DEBUG_TRACE` as a local variable to runners, debug logs generate and are visible to all users with access to job logs. The permission levels are not checked by the runner, so you should only use the variable in GitLab itself. + +## Known issues and workarounds + +These are some know issues with CI/CD variables, and where applicable, known workarounds. + +### "argument list too long" + +This issue occurs when the combined length of all CI/CD variables defined for a job exceeds the limit imposed by the +shell where the job executes. This includes the names and values of pre-defined and user defined variables. This limit +is typically referred to as `ARG_MAX`, and is shell and operating system dependent. This issue also occurs when the +content of a single [File-type](#use-file-type-cicd-variables) variable exceeds `ARG_MAX`. + +For more information, see [issue 392406](https://gitlab.com/gitlab-org/gitlab/-/issues/392406#note_1414219596). + +As a workaround you can either: + +- Use [File-type](#use-file-type-cicd-variables) CI/CD variables for large environment variables where possible. +- If a single large variable is larger than `ARG_MAX`, try using [Secure Files](../secure_files/index.md), or +bring the file to the job through some other mechanism. diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 1656b2147d5..73aaafe46c0 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -105,7 +105,7 @@ as it can cause the pipeline to behave unexpectedly. | `CI_PROJECT_DESCRIPTION` | 15.1 | all | The project description as displayed in the GitLab web interface. | | `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address of the project. | | `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility. Can be `internal`, `private`, or `public`. | -| `CI_PROJECT_CLASSIFICATION_LABEL` | 14.2 | all | The project [external authorization classification label](../../user/admin_area/settings/external_authorization.md). | +| `CI_PROJECT_CLASSIFICATION_LABEL` | 14.2 | all | The project [external authorization classification label](../../administration/settings/external_authorization.md). | | `CI_REGISTRY_IMAGE` | 8.10 | 0.5 | The address of the project's Container Registry. Only available if the Container Registry is enabled for the project. | | `CI_REGISTRY_PASSWORD` | 9.0 | all | The password to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. This password value is the same as the `CI_JOB_TOKEN` and is valid only as long as the job is running. Use the `CI_DEPLOY_PASSWORD` for long-lived access to the registry | | `CI_REGISTRY_USER` | 9.0 | all | The username to push containers to the project's GitLab Container Registry. Only available if the Container Registry is enabled for the project. | diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 69595b62de2..3b6419dbff2 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -129,7 +129,8 @@ Content of `.gitlab-ci.yml`: ```yaml include: 'https://company.com/autodevops-template.yml' -image: alpine:latest +default: + image: alpine:latest variables: POSTGRES_USER: root @@ -418,7 +419,8 @@ these keywords: ### `include` with `rules:if` -> Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. +> - Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. +> - Support for `when: never` and `when:always` [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/414517) in GitLab 16.2. Feature flag `ci_support_include_rules_when_never` removed. Use [`rules:if`](index.md#rulesif) to conditionally include other configuration files based on the status of CI/CD variables. For example: @@ -447,7 +449,8 @@ test: ### `include` with `rules:exists` -> Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. +> - Support for `when: never` and `when:always` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348146) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `ci_support_include_rules_when_never`. Disabled by default. +> - Support for `when: never` and `when:always` [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/414517) in GitLab 16.2. Feature flag `ci_support_include_rules_when_never` removed. Use [`rules:exists`](index.md#rulesexists) to conditionally include other configuration files based on the existence of files. For example: @@ -627,4 +630,4 @@ limit is reached. You can remove one included file at a time to try to narrow do which configuration file is the source of the loop or excessive included files. In [GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/207270) self-managed users can -change the [maximum includes](../../user/admin_area/settings/continuous_integration.md#maximum-includes) value. +change the [maximum includes](../../administration/settings/continuous_integration.md#maximum-includes) value. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 3dfc98c043a..9a060c35721 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -163,7 +163,7 @@ The time limit to resolve all files is 30 seconds. pipeline run, the new pipeline uses the changed configuration. - You can have up to 150 includes per pipeline by default, including [nested](includes.md#use-nested-includes). Additionally: - In [GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/207270) self-managed users can - change the [maximum includes](../../user/admin_area/settings/continuous_integration.md#maximum-includes) value. + change the [maximum includes](../../administration/settings/continuous_integration.md#maximum-includes) value. - In [GitLab 15.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/367150) you can have up to 150 includes. In nested includes, the same file can be included multiple times, but duplicated includes count towards the limit. @@ -353,7 +353,7 @@ The order of the items in `stages` defines the execution order for jobs: If a pipeline contains only jobs in the `.pre` or `.post` stages, it does not run. There must be at least one other job in a different stage. `.pre` and `.post` stages -can be used in [required pipeline configuration](../../user/admin_area/settings/continuous_integration.md#required-pipeline-configuration) +can be used in [required pipeline configuration](../../administration/settings/continuous_integration.md#required-pipeline-configuration) to define compliance jobs that must run before or after project pipeline jobs. **Keyword type**: Global keyword. @@ -843,7 +843,7 @@ they expire and are deleted. The `expire_in` setting does not affect: - Artifacts from the latest job, unless keeping the latest job artifacts is disabled [at the project level](../jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). - or [instance-wide](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). + or [instance-wide](../../administration/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). After their expiry, artifacts are deleted hourly by default (using a cron job), and are not accessible anymore. @@ -875,7 +875,7 @@ job: **Additional details**: - The expiration time period begins when the artifact is uploaded and stored on GitLab. - If the expiry time is not defined, it defaults to the [instance wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration). + If the expiry time is not defined, it defaults to the [instance wide setting](../../administration/settings/continuous_integration.md#default-artifacts-expiration). - To override the expiration date and protect artifacts from being automatically deleted: - Select **Keep** on the job page. - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/22761), set the value of @@ -963,7 +963,7 @@ job: WARNING: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `non_public_artifacts`. On +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `non_public_artifacts`. On GitLab.com, this feature is not available. Due to [issue 413822](https://gitlab.com/gitlab-org/gitlab/-/issues/413822), the keyword can be used when the feature flag is disabled, but the feature does not work. Do not attempt to use this feature when the feature flag is disabled, and always test @@ -2039,11 +2039,17 @@ job_with_id_tokens: aud: - https://gcp.com - https://aws.com + SIGSTORE_ID_TOKEN: + aud: sigstore script: - command_to_authenticate_with_gitlab $ID_TOKEN_1 - command_to_authenticate_with_aws $ID_TOKEN_2 ``` +**Related topics**: + +- [Keyless signing with Sigstore](signing_examples.md). + ### `image` Use `image` to specify a Docker image that the job runs in. @@ -2718,7 +2724,7 @@ when to add jobs to pipelines. | `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | | `tags` | When the Git reference for a pipeline is a tag. | | `triggers` | For pipelines created by using a [trigger token](../triggers/index.md#configure-cicd-jobs-to-run-in-triggered-pipelines). | - | `web` | For pipelines created by selecting **Run pipeline** in the GitLab UI, from the project's **CI/CD > Pipelines** section. | + | `web` | For pipelines created by selecting **Run pipeline** in the GitLab UI, from the project's **Build > Pipelines** section. | **Example of `only:refs` and `except:refs`**: @@ -2896,6 +2902,12 @@ in the project. Use `pages` to define a [GitLab Pages](../../user/project/pages/index.md) job that uploads static content to GitLab. The content is then published as a website. +You must: + +- Define [`artifacts`](#artifacts) with a path to the content directory, which is + `public` by default. +- Use [`publish`](#pagespublish) if want to use a different content directory. + **Keyword type**: Job name. **Example of `pages`**: @@ -2904,9 +2916,7 @@ uploads static content to GitLab. The content is then published as a website. pages: stage: deploy script: - - mkdir .public - - cp -r * .public - - mv .public public + - mv my-html-content public artifacts: paths: - public @@ -2915,15 +2925,38 @@ pages: environment: production ``` -This example moves all files from the root of the project to the `public/` directory. -The `.public` workaround is so `cp` does not also copy `public/` to itself in an infinite loop. +This example moves all files from a `my-html-content/` directory to the `public/` directory. +This directory is exported as an artifact and published with GitLab Pages. -**Additional details**: +#### `pages:publish` -You must: +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/415821) in GitLab 16.1. + +Use `publish` to configure the content directory of a [`pages` job](#pages). + +**Keyword type**: Job keyword. You can use it only as part of a `pages` job. + +**Possible inputs**: A path to a directory containing the Pages content. + +**Example of `publish`**: + +```yaml +pages: + stage: deploy + script: + - npx @11ty/eleventy --input=path/to/eleventy/root --output=dist + artifacts: + paths: + - dist + publish: dist + rules: + - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH + environment: production +``` -- Place any static content in a `public/` directory. -- Define [`artifacts`](#artifacts) with a path to the `public/` directory. +This example uses [Eleventy](https://www.11ty.dev) to generate a static website and +output the generated HTML files into a the `dist/` directory. This directory is exported +as an artifact and published with GitLab Pages. ### `parallel` @@ -2955,6 +2988,11 @@ This example creates 5 jobs that run in parallel, named `test 1/5` to `test 5/5` - Every parallel job has a `CI_NODE_INDEX` and `CI_NODE_TOTAL` [predefined CI/CD variable](../variables/index.md#predefined-cicd-variables) set. +- A pipeline with jobs that use `parallel` might: + - Create more jobs running in parallel than available runners. Excess jobs are queued + and marked `pending` while waiting for an available runner. + - Create too many jobs, and the pipeline fails with a `job_activity_limit_exceeded` error. + The maximum number of jobs that can exist in active pipelines is [limited at the instance-level](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). **Related topics**: @@ -3076,7 +3114,7 @@ release_job: This example creates a release: - When you push a Git tag. -- When you add a Git tag in the UI at **Repository > Tags**. +- When you add a Git tag in the UI at **Code > Tags**. **Additional details**: @@ -3120,8 +3158,7 @@ CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitl To create a release when a new tag is added to the project: - Use the `$CI_COMMIT_TAG` CI/CD variable as the `tag_name`. -- Use [`rules:if`](#rulesif) or [`only: tags`](#onlyrefs--exceptrefs) to configure - the job to run only for new tags. +- Use [`rules:if`](#rulesif) to configure the job to run only for new tags. ```yaml job: @@ -3133,7 +3170,7 @@ job: - if: $CI_COMMIT_TAG ``` -To create a release and a new tag at the same time, your [`rules`](#rules) or [`only`](#only--except) +To create a release and a new tag at the same time, your [`rules`](#rules) should **not** configure the job to run only for new tags. A semantic versioning example: ```yaml @@ -3702,7 +3739,8 @@ If the rule matches, then the job is a manual job with `allow_failure: true`. #### `rules:needs` -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31581) in GitLab 16.0 [with a flag](../../user/feature_flags.md) named `introduce_rules_with_needs`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31581) in GitLab 16.0 [with a flag](../../user/feature_flags.md) named `introduce_rules_with_needs`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/408871) in GitLab 16.2. Feature flag `introduce_rules_with_needs` removed. Use `needs` in rules to update a job's [`needs`](#needs) for specific conditions. When a condition matches a rule, the job's `needs` configuration is completely replaced with the `needs` in the rule. @@ -4752,6 +4790,6 @@ important to describe those, too. Think of things that may go wrong and include This is important to minimize requests for support, and to avoid doc comments with questions that you know someone might ask. -Each scenario can be a third-level heading, e.g. `### Getting error message X`. +Each scenario can be a third-level heading, for example, `### Getting error message X`. If you have none to add when creating a doc, leave this section in place but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/yaml/signing_examples.md b/doc/ci/yaml/signing_examples.md new file mode 100644 index 00000000000..5609bd2374e --- /dev/null +++ b/doc/ci/yaml/signing_examples.md @@ -0,0 +1,310 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Use Sigstore for keyless signing and verification **(FREE SAAS)** + +The [Sigstore](https://www.sigstore.dev/) project provides a CLI called +[Cosign](https://docs.sigstore.dev/cosign/overview/) which can be used for keyless signing of container images built +with GitLab CI/CD. Keyless signing has many advantages, including eliminating the need to manage, safeguard, and rotate a private +key. Cosign requests a short-lived key pair to use for signing, records it on a certificate transparency log, and +then discards it. The key is generated through a token obtained from the GitLab server using the OIDC identity of the user who +ran the pipeline. This token includes unique claims that certify the token was generated by a CI/CD pipeline. To learn more, +see Cosign [documentation](https://docs.sigstore.dev/cosign/overview/#example-working-with-containers) on keyless signatures. + +For details on the mapping between GitLab OIDC claims and Fulcio certificate extensions, see the GitLab column of +[Mapping OIDC token claims to Fulcio OIDs](https://github.com/sigstore/fulcio/blob/main/docs/oid-info.md#mapping-oidc-token-claims-to-fulcio-oids). + +**Requirements:** + +- You must be using GitLab.com. +- Your project's CI/CD configuration must be located in the project. +- You must use a version of Cosign that is `>= 2.0.1`. + +## Container images + +### Sign a container image with Cosign + +GitLab [ID tokens](../secrets/id_token_authentication.md) can be used by Cosign for +[keyless signing](https://docs.sigstore.dev/cosign/overview/#example-working-with-containers). The token must have `sigstore` set as the +[`aud`](../secrets/id_token_authentication.md#token-payload) claim. The token can be used by Cosign automatically when +it is set in the `SIGSTORE_ID_TOKEN` environment variable. + +**Best practices**: + +- Build and sign a container image in the same job to prevent the image from being tampered with before it is + signed. + +See [Cosign documentation](https://docs.sigstore.dev/cosign/signing_with_containers/) to learn more about signing. + +```yaml +build_and_sign: + stage: build + image: docker:latest + services: + - docker:dind + variables: + COSIGN_YES: "true" + id_tokens: + SIGSTORE_ID_TOKEN: + aud: sigstore + before_script: + - apk add --update cosign + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY + script: + - docker build --pull -t "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" . + - docker push "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" + - IMAGE_DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA) + - cosign sign $IMAGE_DIGEST +``` + +### Verify a container image with Cosign + +```yaml +verify: + image: alpine:3.18 + stage: verify + before_script: + - apk add --update cosign docker + - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY + script: + - cosign verify "$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA" --certificate-identity "https://gitlab.com/my-group/my-project//path/to/.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "https://gitlab.com" +``` + +## Use Sigstore and npm to generate keyless provenance + +You can use Sigstore and npm, together with GitLab CI/CD, to digitally sign build artifacts without the overhead of key management. + +### About npm provenance + +[npm CLI](https://docs.npmjs.com/cli) allows package maintainers to provide users with provenance attestations. Using npm +CLI provenance generation allows users to trust and verify that the package they are downloading and using is from you and the +build system that built it. + +For more information on how to publish npm packages, see [GitLab npm Package Registry](../../user/packages/npm_registry/index.md). + +### Sigstore + +[Sigstore](https://www.sigstore.dev/) is a set of tools that package managers and security experts can use to secure their software +supply chains against attacks. Bringing together free-to-use open source technologies like Fulcio, Cosign, and Rekor, it +handles digital signing, verification, and checks for provenance +needed to make it safer to distribute and use open source software. + +**Related topics**: + +- [SLSA Provenance definition](https://slsa.dev/provenance/v1) +- [npm Docs](https://docs.npmjs.com/generating-provenance-statements) +- [npm Provenance RFC](https://github.com/npm/rfcs/blob/main/accepted/0049-link-packages-to-source-and-build.md#detailed-steps-to-publish) + +### Generating provenance in GitLab CI/CD + +Now that Sigstore supports GitLab OIDC as described above, you can use npm provenance together with GitLab CI/CD and Sigstore to +generate and sign provenance for your npm packages in a GitLab CI/CD pipeline. + +#### Prerequisites + +1. Set your GitLab [ID token](../secrets/id_token_authentication.md) `aud` to `sigstore`. +1. Add the `--provenance` flag to have npm publish. + +Example content to be added to `.gitlab-ci.yml` file: + +```yaml +image: node:latest + +build: + id_tokens: + SIGSTORE_ID_TOKEN: + aud: sigstore + script: + - npm publish --provenance --access public +``` + +The npm GitLab template provides this functionality as well, the example is in +the [templates documentation](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/npm.gitlab-ci.yml). + +## Verifying npm provenance + +npm CLI also provides functionality for end users to verify the provenance of packages. + +```plaintext +npm audit signatures +audited 1 package in 0s +1 package has a verified registry signature +``` + +### Inspecting the provenance metadata + +The Rekor transparency log stores certificates and attestations for every package that is published with provenance. +For example, here is the [entry for the below example](https://search.sigstore.dev/?logIndex=21076013). + +An example provenance document generated by npm: + +```yaml +_type: https://in-toto.io/Statement/v0.1 +subject: + - name: pkg:npm/%40strongjz/strongcoin@0.0.13 + digest: + sha512: >- + 924a134a0fd4fe6a7c87b4687bf0ac898b9153218ce9ad75798cc27ab2cddbeff77541f3847049bd5e3dfd74cea0a83754e7686852f34b185c3621d3932bc3c8 +predicateType: https://slsa.dev/provenance/v0.2 +predicate: + buildType: https://github.com/npm/CLI/gitlab/v0alpha1 + builder: + id: https://gitlab.com/strongjz/npm-provenance-example/-/runners/12270835 + invocation: + configSource: + uri: git+https://gitlab.com/strongjz/npm-provenance-example + digest: + sha1: 6e02e901e936bfac3d4691984dff8c505410cbc3 + entryPoint: deploy + parameters: + CI: 'true' + CI_API_GRAPHQL_URL: https://gitlab.com/api/graphql + CI_API_V4_URL: https://gitlab.com/api/v4 + CI_COMMIT_BEFORE_SHA: 7d3e913e5375f68700e0c34aa90b0be7843edf6c + CI_COMMIT_BRANCH: main + CI_COMMIT_REF_NAME: main + CI_COMMIT_REF_PROTECTED: 'true' + CI_COMMIT_REF_SLUG: main + CI_COMMIT_SHA: 6e02e901e936bfac3d4691984dff8c505410cbc3 + CI_COMMIT_SHORT_SHA: 6e02e901 + CI_COMMIT_TIMESTAMP: '2023-05-19T10:17:12-04:00' + CI_COMMIT_TITLE: trying to publish to gitlab reg + CI_CONFIG_PATH: .gitlab-ci.yml + CI_DEFAULT_BRANCH: main + CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX: gitlab.com:443/strongjz/dependency_proxy/containers + CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX: gitlab.com:443/strongjz/dependency_proxy/containers + CI_DEPENDENCY_PROXY_SERVER: gitlab.com:443 + CI_DEPENDENCY_PROXY_USER: gitlab-ci-token + CI_JOB_ID: '4316132595' + CI_JOB_NAME: deploy + CI_JOB_NAME_SLUG: deploy + CI_JOB_STAGE: deploy + CI_JOB_STARTED_AT: '2023-05-19T14:17:23Z' + CI_JOB_URL: https://gitlab.com/strongjz/npm-provenance-example/-/jobs/4316132595 + CI_NODE_TOTAL: '1' + CI_PAGES_DOMAIN: gitlab.io + CI_PAGES_URL: https://strongjz.gitlab.io/npm-provenance-example + CI_PIPELINE_CREATED_AT: '2023-05-19T14:17:21Z' + CI_PIPELINE_ID: '872773336' + CI_PIPELINE_IID: '40' + CI_PIPELINE_SOURCE: push + CI_PIPELINE_URL: https://gitlab.com/strongjz/npm-provenance-example/-/pipelines/872773336 + CI_PROJECT_CLASSIFICATION_LABEL: '' + CI_PROJECT_DESCRIPTION: '' + CI_PROJECT_ID: '45821955' + CI_PROJECT_NAME: npm-provenance-example + CI_PROJECT_NAMESPACE: strongjz + CI_PROJECT_NAMESPACE_ID: '36018' + CI_PROJECT_PATH: strongjz/npm-provenance-example + CI_PROJECT_PATH_SLUG: strongjz-npm-provenance-example + CI_PROJECT_REPOSITORY_LANGUAGES: javascript,dockerfile + CI_PROJECT_ROOT_NAMESPACE: strongjz + CI_PROJECT_TITLE: npm-provenance-example + CI_PROJECT_URL: https://gitlab.com/strongjz/npm-provenance-example + CI_PROJECT_VISIBILITY: public + CI_REGISTRY: registry.gitlab.com + CI_REGISTRY_IMAGE: registry.gitlab.com/strongjz/npm-provenance-example + CI_REGISTRY_USER: gitlab-ci-token + CI_RUNNER_DESCRIPTION: 3-blue.shared.runners-manager.gitlab.com/default + CI_RUNNER_ID: '12270835' + CI_RUNNER_TAGS: >- + ["gce", "east-c", "linux", "ruby", "mysql", "postgres", "mongo", + "git-annex", "shared", "docker", "saas-linux-small-amd64"] + CI_SERVER_HOST: gitlab.com + CI_SERVER_NAME: GitLab + CI_SERVER_PORT: '443' + CI_SERVER_PROTOCOL: https + CI_SERVER_REVISION: 9d4873fd3c5 + CI_SERVER_SHELL_SSH_HOST: gitlab.com + CI_SERVER_SHELL_SSH_PORT: '22' + CI_SERVER_URL: https://gitlab.com + CI_SERVER_VERSION: 16.1.0-pre + CI_SERVER_VERSION_MAJOR: '16' + CI_SERVER_VERSION_MINOR: '1' + CI_SERVER_VERSION_PATCH: '0' + CI_TEMPLATE_REGISTRY_HOST: registry.gitlab.com + GITLAB_CI: 'true' + GITLAB_FEATURES: >- + elastic_search,ldap_group_sync,multiple_ldap_servers,seat_link,usage_quotas,zoekt_code_search,repository_size_limit,admin_audit_log,auditor_user,custom_file_templates,custom_project_templates,db_load_balancing,default_branch_protection_restriction_in_groups,extended_audit_events,external_authorization_service_api_management,geo,instance_level_scim,ldap_group_sync_filter,object_storage,pages_size_limit,project_aliases,password_complexity,enterprise_templates,git_abuse_rate_limit,required_ci_templates,runner_maintenance_note,runner_performance_insights,runner_upgrade_management,runner_jobs_statistics + GITLAB_USER_ID: '31705' + GITLAB_USER_LOGIN: strongjz + environment: + name: 3-blue.shared.runners-manager.gitlab.com/default + architecture: linux/amd64 + server: https://gitlab.com + project: strongjz/npm-provenance-example + job: + id: '4316132595' + pipeline: + id: '872773336' + ref: .gitlab-ci.yml + metadata: + buildInvocationId: https://gitlab.com/strongjz/npm-provenance-example/-/jobs/4316132595 + completeness: + parameters: true + environment: true + materials: false + reproducible: false + materials: + - uri: git+https://gitlab.com/strongjz/npm-provenance-example + digest: + sha1: 6e02e901e936bfac3d4691984dff8c505410cbc3 +``` + +## Build artifacts + +You can use Cosign to both sign build artifacts and verify artifacts that were signed with Cosign. + +### Sign a build artifact with Cosign + +GitLab [ID tokens](../secrets/id_token_authentication.md) can be used by Cosign for keyless signing of build artifacts. +The token must have `sigstore` set as the [`aud`](../secrets/id_token_authentication.md#token-payload) claim. The token can be used by Cosign automatically when +it is set in the `SIGSTORE_ID_TOKEN` environment variable. + +**Best practices**: + +- Build and sign an artifact in the same job to prevent the artifact from being tampered with before it is + signed. + +To learn more about signing artifacts, see [Cosign documentation](https://docs.sigstore.dev/cosign/signing_with_blobs/#keyless-signing-of-blobs-and-files). + +```yaml +sign_artifact: + stage: build + image: alpine:latest + variables: + COSIGN_YES: "true" + id_tokens: + SIGSTORE_ID_TOKEN: + aud: sigstore + before_script: + - apk add --update cosign + script: + - echo "This is a build artifact" > artifact.txt + - cosign sign-blob artifact.txt --bundle cosign.bundle + artifacts: + paths: + - artifact.txt + - cosign.bundle +``` + +### Verify a build artifact with Cosign + +Verifying an artifact requires both the artifact itself and the `cosign.bundle` file produced by `cosign sign-blob`. +The `--certificate-identity` option should reference the project and branch where the artifact was signed. The +`--certificate-oidc-issuer` option should reference the GitLab instance where the artifact was signed. + +To learn more about verifying signed artifacts, see [Cosign documentation](https://docs.sigstore.dev/cosign/verify/). + +```yaml +verify_artifact: + stage: verify + image: alpine:latest + before_script: + - apk add --update cosign + script: + - cosign verify-blob artifact.txt --bundle cosign.bundle --certificate-identity "https://gitlab.com/my-group/my-project//path/to/.gitlab-ci.yml@refs/heads/main" --certificate-oidc-issuer "https://gitlab.com" +``` diff --git a/doc/development/advanced_search.md b/doc/development/advanced_search.md index 73a8191b789..30e1874f1ed 100644 --- a/doc/development/advanced_search.md +++ b/doc/development/advanced_search.md @@ -167,7 +167,7 @@ These proxy objects would talk to Elasticsearch server directly (see top half of  -In the planned new design, each model would have a pair of corresponding sub-classed proxy objects, in which model-specific logic is located. For example, `Snippet` would have `SnippetClassProxy` and `SnippetInstanceProxy` (being subclass of `Elasticsearch::Model::Proxy::ClassMethodsProxy` and `Elasticsearch::Model::Proxy::InstanceMethodsProxy`, respectively). +In the planned new design, each model would have a pair of corresponding sub-classed proxy objects, in which model-specific logic is located. For example, `Snippet` would have `SnippetClassProxy` being a subclass of `Elasticsearch::Model::Proxy::ClassMethodsProxy`. `Snippet` would have `SnippetInstanceProxy` being a subclass of `Elasticsearch::Model::Proxy::InstanceMethodsProxy`. `__elasticsearch__` would represent another layer of proxy object, keeping track of multiple actual proxy objects. It would forward method calls to the appropriate index. For example: diff --git a/doc/development/ai_architecture.md b/doc/development/ai_architecture.md index ac62f50baf5..f497047ccce 100644 --- a/doc/development/ai_architecture.md +++ b/doc/development/ai_architecture.md @@ -40,7 +40,7 @@ package "AI API" as AIF { node "Vertex AI" } -package GitLab { +package GitLab { node "Web IDE" package "Web" { @@ -106,3 +106,24 @@ The following models have been approved for use: The following vector stores have been approved for use: - [`pgvector`](https://github.com/pgvector/pgvector) is a Postgres extension adding support for storing vector embeddings and calculating ANN (approximate nearest neighbor). + +### Indexing Update + +We are currently using sequential scan, which provides perfect recall. We are considering adding an index if we can ensure that it still produces accurate results, as noted in the `pgvector` indexing [documentation](https://github.com/pgvector/pgvector#indexing). + +Given that the table contains thousands of entries, indexing with these updated settings would likely improve search speed while maintaining high accuracy. However, more testing may be needed to verify the optimal configuration for this dataset size before deploying to production. + +A [draft MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/122035) has been created to update the index. + +The index function has been updated to improve search quality. This was tested locally by setting the `ivfflat.probes` value to `10` with the following SQL command: + +```ruby +Embedding::TanukiBotMvc.connection.execute("SET ivfflat.probes = 10") +``` + +Setting the `probes` value for indexing improves results, as per the neighbor [documentation](https://github.com/ankane/neighbor#indexing). + +For optimal `probes` and `lists` values: + +- Use `lists` equal to `rows / 1000` for tables with up to 1 million rows and `sqrt(rows)` for larger datasets. +- For `probes` start with `lists / 10` for tables up to 1 million rows and `sqrt(lists)` for larger datasets. diff --git a/doc/development/ai_features.md b/doc/development/ai_features.md index c46117c4afd..52dc37caec3 100644 --- a/doc/development/ai_features.md +++ b/doc/development/ai_features.md @@ -17,6 +17,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w - Abstraction for - OpenAI - Google Vertex AI + - Anthropic - Rate Limiting - Circuit Breaker - Multi-Level feature flags @@ -26,9 +27,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w - Configuration for Moderation check of inputs - Automatic Markdown Rendering of responses - Centralised Group Level settings for experiment and 3rd party -- Experimental API endpoints for exploration of AI API’s by GitLab team members without the need for credentials +- Experimental API endpoints for exploration of AI APIs by GitLab team members without the need for credentials - OpenAI - Google Vertex AI + - Anthropic ## Feature flags @@ -41,7 +43,7 @@ See the [feature flag tracker](https://gitlab.com/gitlab-org/gitlab/-/issues/405 ## Implement a new AI action -To implement a new AI action, connect to the OpenAI API. You can connect to this API using either the: +To implement a new AI action, connect to the preferred AI provider. You can connect to this API using either the: - Experimental REST API. - Abstraction layer. @@ -84,6 +86,43 @@ For features that use the embedding database, additional setup is needed. 1. Run `gdk reconfigure` 1. Run database migrations to create the embedding database +### Set up GitLab Duo Chat + +1. [Enable Anthropic API features](#configure-anthropic-access). +1. [Enable OpenAI support](#configure-openai-access). +1. [Ensure the embedding database is configured](#set-up-the-embedding-database). +1. Enable feature specific feature flag. + + ```ruby + Feature.enable(:gitlab_duo) + Feature.enable(:tanuki_bot) + Feature.enable(:ai_redis_cache) + ``` + +1. Ensure that your current branch is up-to-date with `master`. +1. To access the GitLab Duo Chat interface, in the lower-left corner of any page, select **Help** and **Ask GitLab Duo Chat**. + +#### Tips for local development + +1. When responses are taking too long to appear in the user interface, consider restarting Sidekiq by running `gdk restart rails-background-jobs`. If that doesn't work, try `gdk kill` and then `gdk start`. +1. Alternatively, bypass Sidekiq entirely and run the chat service synchronously. This can help with debugging errors as GraphQL errors are now available in the network inspector instead of the Sidekiq logs. + +```diff +diff --git a/ee/app/services/llm/chat_service.rb b/ee/app/services/llm/chat_service.rb +index 5fa7ae8a2bc1..5fe996ba0345 100644 +--- a/ee/app/services/llm/chat_service.rb ++++ b/ee/app/services/llm/chat_service.rb +@@ -5,7 +5,7 @@ class ChatService < BaseService + private + + def perform +- worker_perform(user, resource, :chat, options) ++ worker_perform(user, resource, :chat, options.merge(sync: true)) + end + + def valid? +``` + ### Setup for GitLab documentation chat (legacy chat) To populate the embedding database for GitLab chat: @@ -133,6 +172,22 @@ Feature.enable(:anthropic_experimentation) Gitlab::CurrentSettings.update!(anthropic_api_key: <insert API key>) ``` +### Testing GitLab Duo Chat with predefined questions + +Because success of answers to user questions in GitLab Duo Chat heavily depends on toolchain and prompts of each tool, it's common that even a minor change in a prompt or a tool impacts processing of some questions. To make sure that a change in the toolchain doesn't break existing functionality, you can use following commands to validate answers to some predefined questions: + +1. Rake task which iterates through questions defined in CSV file and checks tools used for evaluating each question. + +```ruby +rake gitlab:llm:zero_shot:test:questions[<issue_url>] +``` + +1. RSpec which iterates through resource-specific questions on predefined resources: + +```ruby +ANTHROPIC_API_KEY='<key>' REAL_AI_REQUEST=1 rspec ee/spec/lib/gitlab/llm/chain/agents/zero_shot/executor_spec.rb +``` + ## Experimental REST API Use the [experimental REST API endpoints](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/ai/experimentation) to quickly experiment and prototype AI features. @@ -153,7 +208,7 @@ The experimental endpoint is only available to GitLab team members on production ### GraphQL API -To connect to the OpenAI API using the Abstraction Layer, use an extendable GraphQL API called +To connect to the AI provider API using the Abstraction Layer, use an extendable GraphQL API called [`aiAction`](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/graphql/mutations/ai/action.rb). The `input` accepts key/value pairs, where the `key` is the action that needs to be performed. We only allow one AI action per mutation request. @@ -182,6 +237,8 @@ mutation { The GraphQL API then uses the [OpenAI Client](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/gitlab/llm/open_ai/client.rb) to send the response. +Remember that other clients are available and you should not use OpenAI. + #### How to receive a response As the OpenAI API requests are handled in a background job, we do not keep the request alive and @@ -201,6 +258,8 @@ You should only subscribe to the subscription once the mutation is sent. If mult #### Current abstraction layer flow +The following graph uses OpenAI as an example. You can use different providers. + ```mermaid flowchart TD A[GitLab frontend] -->B[AiAction GraphQL mutation] diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index d6492b7814e..9bc62ed7095 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -813,6 +813,12 @@ field :token, GraphQL::Types::String, null: true, description: 'Token for login.' ``` +Similarly, you can also mark an entire mutation as Alpha by updating where the mutation is mounted in `app/graphql/types/mutation_type.rb`: + +```ruby +mount_mutation Mutations::Ci::JobArtifact::BulkDestroy, alpha: { milestone: '15.10' } +``` + Alpha GraphQL items is a custom GitLab feature that leverages GraphQL deprecations. An Alpha item appears as deprecated in the GraphQL schema. Like all deprecated schema items, you can test an Alpha field in [GraphiQL](../api/graphql/index.md#graphiql). However, be aware that the GraphiQL @@ -1668,7 +1674,7 @@ should look like this: ### Mounting the mutation To make the mutation available it must be defined on the mutation -type that is stored in `graphql/types/mutation_types`. The +type that is stored in `graphql/types/mutation_type`. The `mount_mutation` helper method defines a field based on the GraphQL-name of the mutation: diff --git a/doc/development/api_styleguide.md b/doc/development/api_styleguide.md index 566267b97f1..2ed4d934022 100644 --- a/doc/development/api_styleguide.md +++ b/doc/development/api_styleguide.md @@ -179,7 +179,7 @@ request that has an optional parameter: optional :user_ids, type: Array[Integer], coerce_with: ::API::Validations::Types::CommaSeparatedToIntegerArray.coerce, desc: 'The user ids for this rule' ``` -Normally, a request to PUT `/test?user_ids` would cause Grape to pass +Usually, a request to PUT `/test?user_ids` would cause Grape to pass `params` of `{ user_ids: nil }`. This may introduce errors with endpoints that expect a blank array and @@ -386,18 +386,6 @@ expect(response).to match_response_schema('merge_requests') Also see [verifying N+1 performance](#verifying-with-tests) in tests. -## Error handling - -When throwing an error with a message that is meant to be user-facing, you should -use the error message utility function contained in `lib/gitlab/utils/error_message.rb`. -It adds a prefix to the error message, making it distinguishable from non-user-facing error messages. - -```ruby -Gitlab::Utils::ErrorMessage.to_user_facing('Example user-facing error-message') -``` - -Please make sure that the Frontend is aware of the prefix usage and is using the according utils. See [Error handling](fe_guide/style/javascript.md#error-handling) in JavaScript style guide for more information. - ## Include a changelog entry All client-facing changes **must** include a [changelog entry](changelog.md). diff --git a/doc/development/application_limits.md b/doc/development/application_limits.md index b1efc11db62..40d157a4411 100644 --- a/doc/development/application_limits.md +++ b/doc/development/application_limits.md @@ -171,7 +171,7 @@ The process for adding a new throttle is loosely: 1. Extend `Gitlab::RackAttack` and `Gitlab::RackAttack::Request` to configure the new rate limit, and apply it to the desired requests. 1. Add the new settings to the Admin Area form in `app/views/admin/application_settings/_ip_limits.html.haml`. -1. Document the new settings in [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) and [Application settings API](../api/settings.md). +1. Document the new settings in [User and IP rate limits](../administration/settings/user_and_ip_rate_limits.md) and [Application settings API](../api/settings.md). 1. Configure the rate limit for GitLab.com and document it in [GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits). Refer to these past issues for implementation details: diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 133cf8a2998..25f471dc9d2 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -1011,7 +1011,7 @@ Puma. All these components should run as different system users to GitLab (for example, `postgres`, `redis`, and `www-data`, instead of `git`). As the `git` user it starts Sidekiq and Puma (a simple Ruby HTTP server -running on port `8080` by default). Under the GitLab user there are normally 4 +running on port `8080` by default). Under the GitLab user there are usually 4 processes: `puma master` (1 process), `puma cluster worker` (2 processes), `sidekiq` (1 process). @@ -1067,7 +1067,7 @@ Usage: /etc/init.d/postgresql {start|stop|restart|reload|force-reload|status} [v GitLab (includes Puma and Sidekiq logs): -- `/home/git/gitlab/log/` contains `application.log`, `production.log`, `sidekiq.log`, `puma.stdout.log`, `git_json.log` and `puma.stderr.log` normally. +- `/home/git/gitlab/log/` usually contains `application.log`, `production.log`, `sidekiq.log`, `puma.stdout.log`, `git_json.log` and `puma.stderr.log`. GitLab Shell: diff --git a/doc/development/audit_event_guide/index.md b/doc/development/audit_event_guide/index.md index c7c723d1686..c49d3a243c0 100644 --- a/doc/development/audit_event_guide/index.md +++ b/doc/development/audit_event_guide/index.md @@ -32,7 +32,7 @@ To instrument an audit event, the following attributes should be provided: | Attribute | Type | Required? | Description | |:-------------|:---------------------|:----------|:------------------------------------------------------------------| | `name` | String | false | Action name to be audited. Represents the [type of the event](#event-type-definitions). Used for error tracking | -| `author` | User | true | User who authors the change | +| `author` | User | true | User who authors the change. Can be an [internal user](../internal_users.md). For example, [inactive project deletion](../../administration/inactive_project_deletion.md) audit events are authored by `GitLab-Admin-Bot`. | | `scope` | User, Project, Group | true | Scope which the audit event belongs to | | `target` | Object | true | Target object being audited | | `message` | String | true | Message describing the action ([not translated](#i18n-and-the-audit-event-message-attribute)) | diff --git a/doc/development/auto_devops.md b/doc/development/auto_devops.md index 1f98a37ac9d..ccbad7f7314 100644 --- a/doc/development/auto_devops.md +++ b/doc/development/auto_devops.md @@ -41,7 +41,7 @@ Some jobs use images that are built from external projects: [`auto-deploy-app`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) is used to deploy. There are extra variables that get passed to the CI jobs when Auto -DevOps is enabled that are not present in a normal CI job. These can be +DevOps is enabled that are not present in a typical CI job. These can be found in [`ProjectAutoDevops`](https://gitlab.com/gitlab-org/gitlab/-/blob/bf69484afa94e091c3e1383945f60dbe4e8681af/app/models/project_auto_devops.rb). diff --git a/doc/development/avoiding_required_stops.md b/doc/development/avoiding_required_stops.md new file mode 100644 index 00000000000..0308e0c30c0 --- /dev/null +++ b/doc/development/avoiding_required_stops.md @@ -0,0 +1,137 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Avoiding required stops + +Required stops are any changes to GitLab [components](architecture.md) or +dependencies that result in the need to upgrade to and stop at a specific +`major.minor` version when [upgrading GitLab](../update/index.md). + +While Development maintains a [maintainenance policy](../policy/maintenance.md) +that results in a three-release (3 month) backport window - GitLab maintains a +much longer window of [version support](https://about.gitlab.com/support/statement-of-support/#version-support) +that includes the current major version, as well as the two previous major +versions. Based on the schedule of previous major releases, GitLab customers can +lag behind the current release for up to three years and still expect to have +support for upgrades. + +For example, a GitLab user upgrading from GitLab 14.0.12 to GitLab 16.1, +which is a fully supported [upgrade path](../update/index.md#upgrade-paths), may have +the following required stops: `14.3.6`, `14.9.5`, `14.10.5`, `15.0.5`, `15.1.6`, +`15.4.6`, and `15.11.11` before upgrading to the latest `16.1.z` version. + +Past required stops have not been discovered for months after their introduction, +often as the result of extensive troubleshooting and assistance from Support +Engineers, Customer Success Managers, and Development Engineers as users upgrade +across greater than 1-3 minor releases. + +Wherever possible, a required stop should be avoided. If it can't be avoided, +the required stop should be aligned to a _scheduled_ required stop. + +Scheduled required stops are often implemented for the previous `major`.`minor` +release just prior to a `major` version release in order to accomodate multiple +[planned deprecations](../update/terminology.md#deprecation) and known +[breaking changes](../update/terminology.md#breaking-change). + +Additionally, as of GitLab 16, we have introduced +[_scheduled_ `major`.`minor` required stops](../update/index.md#upgrade-paths): + +>>> +During GitLab 16.x, we are scheduling two or three required upgrade stops. + +We will give at least two milestones of notice when we schedule a required +upgrade stop. The first planned required upgrade stop is scheduled for GitLab +16.3. If nothing is introduced requiring an upgrade stop, GitLab 16.3 will be +treated as a regular upgrade. +>>> + +## Causes of required stops + +### Inaccurate assumptions about completed migrations + +The majority of required stops are due to assumptions about the state of the +data model in a given release, usually in the form of interdependent database +migrations, or code changes that assume that schema changes introduced in +prior migrations have completed by the time the code loads. + +Designing changes and migrations for [backwards compatibility between versions](multi_version_compatibility.md) can mitigate stop concerns with continuous or +"zero-downtime" upgrades. However, the **contract** phase will likely introduce +a required stop when a migration/code change is introduced that requires +that background migrations have completed before running or loading. + +WARNING: +If you're considering adding or removing a migration, or introducing code that +assumes that migrations have completed in a given release, first review +the database-related documentation on [required stops](database/required_stops.md). + +#### Examples + +- GitLab `12.1`: Introduced a background migration changing `merge_status` in + MergeRequests depending on the `state` value. The `state` attribute was removed + in `12.10`. It took until `13.6` to document the required stop. +- GitLab `13.8`: Includes a background migration to deal with duplicate service + records. In `13.9`, a unique index was applied in another migration that + depended on the background migration completing. Not discovered/documented until + GitLab `14.3` +- GitLab `14.3`: Includes a potentially long-running background migration against + `merge_request_diff_commits` that was foregrounded in `14.5`. This change resulted in + extensive downtime for users with large GitLab installations. Not documented + until GitLab `15.1` +- GitLab `14.9`: Includes a batched background migration for `namespaces` and `projects` + that needs to finish before another batched background migration added in `14.10` executes, + forcing a required stop. The migration can take hours or days to complete on + large GitLab installations. + +Additional details as well as links to related issues and merge requests can be +found in: [Issue: Put in place measures to avoid addition/proliferation of GitLab upgrade path stops](https://gitlab.com/gitlab-org/gitlab/-/issues/375553) + +### Removal of code workarounds and mitigations + +Similar to assumptions about the data model/schema/migration state, required +`major.minor` stops have been introduced due to the intentional removal of +code implemented to workaround previously discovered issues. + +#### Examples + +- GitLab `13.1`: A security fix in Rails `6.0.3.1` introduced a CSRF token change + (causing a canary environment incident). We introduced code to maintain acceptance + of previously generated tokens, and removed the code in `13.2`, creating a known + required stop in `13.1`. +- GitLab `15.4`: Introduces a migration to fix an inaccurate `expires_at` timestamp + for job artifacts that had been mitigated in code since GitLab `14.9`. The + workaround was removed in GitLab `15.6`, causing `15.4` to be a required stop. + +### Deprecations + +Deprecations, particularly [breaking changes](../update/terminology.md#breaking-change) +can also cause required stops if they introduce long migration delays or require +manual actions on the part of GitLab administrators. + +These are generally accepted as a required stop around a major release, either +stopping at the latest `major.minor` release immediately proceeding +a new `major` release, and potentially the lastest `major.0` patch release, and +to date, discovered required stops related to deprecations have been limited to +these releases. + +#### Examples + +Examples of deprecations are too numerous to be listed here, but can found +in the [deprecations and removals by version](../update/deprecations.md) as well +as the [version-specific upgrading instructions](../update/index.md#version-specific-upgrading-instructions), +[version-specific changes for the GitLab package (Omnibus)](../update/package/index.md#version-specific-changes), +and [GitLab chart upgrade notes](https://docs.gitlab.com/charts/installation/upgrade.html). + +## Further reading + +- [Documentation: Database - Adding required stops](database/required_stops.md) +- [Documentation: Upgrading GitLab](../update/index.md) + - [Package (Omnibus) upgrade](../update/package/index.md) + - [Docker upgrade](../install/docker.md#upgrade) + - [GitLab chart](https://docs.gitlab.com/charts/installation/upgrade.html) +- [Issue: Put in place measures to avoid addition/proliferation of GitLab upgrade path stops](https://gitlab.com/gitlab-org/gitlab/-/issues/375553) +- [Issue: Brainstorm ways for background migrations to be finalized without introducing a required upgrade step](https://gitlab.com/gitlab-org/gitlab/-/issues/357561) +- [Issue: Scheduled required paths for GitLab upgrades to improve UX](https://gitlab.com/gitlab-org/gitlab/-/issues/358417) +- [Epic: GitLab Releases and Maintenance policies](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/988) diff --git a/doc/development/backend/ruby_style_guide.md b/doc/development/backend/ruby_style_guide.md index 1a1c0db49f7..714c8571d20 100644 --- a/doc/development/backend/ruby_style_guide.md +++ b/doc/development/backend/ruby_style_guide.md @@ -175,3 +175,24 @@ Previous discussions include: - <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/44234> - <https://gitlab.com/gitlab-org/gitlab-foss/-/issues/36076> - <https://gitlab.com/gitlab-org/gitlab/-/issues/198046> + +### Type safety + +Now that we've upgraded to Ruby 3, we have more options available +to enforce [type safety](https://en.wikipedia.org/wiki/Type_safety). + +Some of these options are supported as part of the Ruby syntax and do not require the use of specific type safety tools like [Sorbet](https://sorbet.org/) or [RBS](https://github.com/ruby/rbs). However, we might consider these tools in the future as well. + +For more information, see [Type safety](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/remote_development#type-safety) in the `remote_development` domain README. + +### Functional patterns + +Although Ruby and especially Rails are primarily based on [object-oriented programming](https://en.wikipedia.org/wiki/object-oriented_programming) patterns, Ruby is a very flexible language and supports [functional programming](https://en.wikipedia.org/wiki/Functional_programming) patterns as well. + +Functional programming patterns, especially in domain logic, can often result in more readable, maintainable, and bug-resistant code while still using idiomatic and familiar Ruby patterns. +However, functional programming patterns should be used carefully because some patterns would cause confusion and should be avoided even if they're directly supported by Ruby. The [`curry` method](https://www.rubydoc.info/stdlib/core/Method:curry) is a likely example. + +For more information, see: + +- [Functional patterns](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/remote_development#functional-patterns) +- [Railway-oriented programming and the `Result` class](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/remote_development#railway-oriented-programming-and-the-result-class) diff --git a/doc/development/cicd/cicd_reference_documentation_guide.md b/doc/development/cicd/cicd_reference_documentation_guide.md index 530bc62b603..e358b24c60f 100644 --- a/doc/development/cicd/cicd_reference_documentation_guide.md +++ b/doc/development/cicd/cicd_reference_documentation_guide.md @@ -125,7 +125,7 @@ can include changes introduced in different GitLab versions. For example: **Additional details**: - The expiration time period begins when the artifact is uploaded and stored on GitLab. - If the expiry time is not defined, it defaults to the [instance wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration). + If the expiry time is not defined, it defaults to the [instance wide setting](../../administration/settings/continuous_integration.md#default-artifacts-expiration). - To override the expiration date and protect artifacts from being automatically deleted: - Select **Keep** on the job page. - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/22761), set the value of diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md index b186c37f933..dceb2da5951 100644 --- a/doc/development/cicd/index.md +++ b/doc/development/cicd/index.md @@ -187,7 +187,7 @@ See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/16111) for the fu ## Compute quota -> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "units of compute" in GitLab 16.1. +> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "compute minutes" in GitLab 16.1. This diagram shows how the [Compute quota](../../ci/pipelines/cicd_minutes.md) feature and its components work. diff --git a/doc/development/cicd/pipeline_wizard.md b/doc/development/cicd/pipeline_wizard.md index 213a9fe1762..f322ec2e2bf 100644 --- a/doc/development/cicd/pipeline_wizard.md +++ b/doc/development/cicd/pipeline_wizard.md @@ -143,7 +143,7 @@ user to the pipeline, for example. ### Template file location -Template files are normally stored as YAML files in `~/pipeline_wizard/templates/`. +Template files are usually stored as YAML files in `~/pipeline_wizard/templates/`. The `PipelineWizard` component expects the `template` property as an un-parsed `String`, and Webpack is configured to load `.yml` files from the above folder as strings. diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md index 77e529867af..bf392a3f446 100644 --- a/doc/development/cicd/templates.md +++ b/doc/development/cicd/templates.md @@ -424,27 +424,11 @@ To add a metric definition for a new template: 1. Install and start the [GitLab GDK](https://gitlab.com/gitlab-org/gitlab-development-kit#installation). 1. In the `gitlab` directory in your GDK, check out the branch that contains the new template. -1. [Add the template inclusion event](../internal_analytics/service_ping/implement.md#add-new-events) - with this Rake task: - - ```shell - bin/rake gitlab:usage_data:generate_ci_template_events - ``` - - The task adds a section like the following to [`lib/gitlab/usage_data_counters/known_events/ci_templates.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/ci_templates.yml): - - ```yaml - - name: p_ci_templates_my_template_name - category: ci_templates - aggregation: weekly - ``` - -1. Copy the value of `name` from the new YAML section, and add it to the weekly - and monthly CI/CD template total count metrics: +1. Add the new template event name to the weekly and monthly CI/CD template total count metrics: - [`config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_7d/20210216184557_ci_templates_total_unique_counts_weekly.yml) - [`config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216184559_ci_templates_total_unique_counts_monthly.yml) -1. Use the same `name` as above as the last argument in the following command to +1. Use the same event name as above as the last argument in the following command to [add new metric definitions](../internal_analytics/service_ping/metrics_dictionary.md#metrics-added-dynamic-to-service-ping-payload): ```shell @@ -481,7 +465,6 @@ To add a metric definition for a new template: For example, these are the metrics configuration files for the [5 Minute Production App template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/5-Minute-Production-App.gitlab-ci.yml): -- The template inclusion event: [`lib/gitlab/usage_data_counters/known_events/ci_templates.yml#L438-L441`](https://gitlab.com/gitlab-org/gitlab/-/blob/dcddbf83c29d1ad0839d55362c1b43888304f453/lib/gitlab/usage_data_counters/known_events/ci_templates.yml#L438-L441) - The weekly and monthly metrics definitions: - [`config/metrics/counts_7d/20210901223501_p_ci_templates_5_minute_production_app_weekly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/1a6eceff3914f240864b2ca15ae2dc076ea67bf6/config/metrics/counts_7d/20210216184515_p_ci_templates_5_min_production_app_weekly.yml) - [`config/metrics/counts_28d/20210901223505_p_ci_templates_5_minute_production_app_monthly.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/counts_28d/20210216184517_p_ci_templates_5_min_production_app_monthly.yml) diff --git a/doc/development/code_review.md b/doc/development/code_review.md index e9f00b640d9..6aedb18c15d 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -82,7 +82,7 @@ We make the following assumption with regards to automatically being considered - Team members working in a specific stage/group (for example, create: source code) are considered domain experts for that area of the app they work on. - Team members working on a specific feature (for example, search) are considered domain experts for that feature. -We default to assigning reviews to team members with domain expertise for code reviews. For UX reviews we default to the recommended designer from the Reviewer roulette. +We default to assigning reviews to team members with domain expertise for code reviews. UX reviews default to the recommended reviewer from the Review Roulette. Due to designer capacity limits, areas not supported by a Product Designer will no longer require a UX review unless it is a community contribution. When a suitable [domain expert](#domain-experts) isn't available, you can choose any team member to review the MR, or follow the [Reviewer roulette](#reviewer-roulette) recommendation (see above for UX reviews). To find a domain expert: @@ -106,7 +106,9 @@ Reviewer roulette is an internal tool for use on GitLab.com, and not available f The [Danger bot](dangerbot.md) randomly picks a reviewer and a maintainer for each area of the codebase that your merge request seems to touch. It makes **recommendations** for developer reviewers and you should override it if you think someone else is a better -fit. User-facing changes are also required to have a UX review, even if it's behind a feature flag. Default to the recommended UX reviewer suggested. +fit. + +We only do UX reviews for MRs from teams that include a Product Designer. User-facing changes from these teams are required to have a UX review, even if it's behind a feature flag. Default to the recommended UX reviewer suggested. It picks reviewers and maintainers from the list at the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) @@ -398,6 +400,8 @@ Maintainers are the DRI of assuring that the acceptance criteria of a merge requ In general, [quality is everyone's responsibility](https://about.gitlab.com/handbook/engineering/quality/), but maintainers of an MR are held responsible for **ensuring** that an MR meets those general quality standards. +This includes [avoiding the creation of technical debt in follow-up issues](contributing/issue_workflow.md#technical-debt-in-follow-up-issues). + If a maintainer feels that an MR is substantial enough, or requires a [domain expert](#domain-experts), maintainers have the discretion to request a review from another reviewer, or maintainer. Here are some examples of maintainers proactively doing this during review: diff --git a/doc/development/contributing/first_contribution.md b/doc/development/contributing/first_contribution.md index 3cf7f222fe4..4bebabbc125 100644 --- a/doc/development/contributing/first_contribution.md +++ b/doc/development/contributing/first_contribution.md @@ -302,7 +302,7 @@ Now you're ready to push changes from the community fork to the main GitLab repo  Select **Create merge request**. - If you don't see this message, on the left sidebar, select **Merge requests > New merge request**. + If you don't see this message, on the left sidebar, select **Code > Merge requests > New merge request**. 1. Take a look at the branch names. You should be merging from your branch in the community fork to the `master` branch in the GitLab repository. diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index eb26ddbd9e9..b929a5c0f04 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -71,7 +71,7 @@ To write and test your code, you will use the GitLab Development Kit. Now [Open a merge request](../../user/project/merge_requests/creating_merge_requests.md) to merge your code and its documentation. The earlier you open a merge request, the sooner you can get feedback. You can [mark it as a draft](../../user/project/merge_requests/drafts.md) -to signal that you’re not done yet. +to signal that you're not done yet. 1. In the merge request, fill out all the information requested in the template, like why you are introducing these changes and a link to the issue this merge request is attempting to close/fix. @@ -159,4 +159,4 @@ If you need any help while contributing to GitLab: - For any other questions or feedback on contributing: - Ping `@gitlab-org/community-relations/contributor-success` in a comment on your merge request or issue. - Feel free to [make a new issue with the Contributor Success team](https://gitlab.com/gitlab-org/community-relations/contributor-success/team-task/-/issues/) sharing your experience. -- Did you run out of compute credits for your GitLab merge requests? Join the [GitLab community forks](https://gitlab.com/gitlab-community/meta) project. +- Did you run out of compute minutes for your GitLab merge requests? Join the [GitLab community forks](https://gitlab.com/gitlab-community/meta) project. diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index d24875e559a..651c7214275 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -31,7 +31,7 @@ We were using Overcommit prior to Lefthook, so you may want to uninstall it firs ### Install Lefthook 1. You can install lefthook in [different ways](https://github.com/evilmartians/lefthook/blob/master/docs/install.md#install-lefthook). - If you do not choose to install it globally (e.g. via Homebrew or package managers), and only want to use it for the GitLab project, + If you do not choose to install it globally (for example, via Homebrew or package managers), and only want to use it for the GitLab project, you can install the Ruby gem via: ```shell diff --git a/doc/development/database/adding_database_indexes.md b/doc/development/database/adding_database_indexes.md index 23a12413975..4c2dacd29ff 100644 --- a/doc/development/database/adding_database_indexes.md +++ b/doc/development/database/adding_database_indexes.md @@ -252,7 +252,7 @@ Sometimes it is necessary to add an index to support a [batched background migra It is commonly done by creating two [post deployment migrations](post_deployment_migrations.md): 1. Add the new index, often a [temporary index](#temporary-indexes). -1. [Queue the batched background migration](batched_background_migrations.md#queueing). +1. [Queue the batched background migration](batched_background_migrations.md#enqueue-a-batched-background-migration). In most cases, no additional work is needed. The new index is created and is used as expected when queuing and executing the batched background migration. @@ -471,7 +471,8 @@ This migration enters the index name and definition into the `postgres_async_ind table. The process that runs on weekends pulls indexes from this table and attempt to remove them. -You must test the database index changes locally before creating a merge request. +You must [test the database index changes locally](#verify-indexes-removed-asynchronously) before creating a merge request. +Include the output of the test in the merge request description. ### Verify the MR was deployed and the index no longer exists in production diff --git a/doc/development/database/avoiding_downtime_in_migrations.md b/doc/development/database/avoiding_downtime_in_migrations.md index 25310554c24..6a819e9f6cd 100644 --- a/doc/development/database/avoiding_downtime_in_migrations.md +++ b/doc/development/database/avoiding_downtime_in_migrations.md @@ -316,6 +316,13 @@ Example migration: Changing column defaults is difficult because of how Rails handles values that are equal to the default. +NOTE: +Rails ignores sending the default values to PostgreSQL when writing records. It leaves this task to +the database. When migrations change the default values of the columns, the running application is unaware +of this change due to the schema cache. The application is then under the risk of accidentally writing +wrong data to the database, especially when deploying the new version of the code +long after we run database migrations. + If running code ever explicitly writes the old default value of a column, you must follow a multi-step process to prevent Rails replacing the old default with the new default in INSERT queries that explicitly specify the old default. @@ -381,7 +388,7 @@ when migrating a column in a large table (for example, `issues`). Background migrations spread the work / load over a longer time period, without slowing down deployments. -For more information, see [the documentation on cleaning up batched background migrations](batched_background_migrations.md#cleaning-up). +For more information, see [the documentation on cleaning up batched background migrations](batched_background_migrations.md#cleaning-up-a-batched-background-migration). ## Adding indexes diff --git a/doc/development/database/background_migrations.md b/doc/development/database/background_migrations.md index 457694c7abd..be121cc0bfe 100644 --- a/doc/development/database/background_migrations.md +++ b/doc/development/database/background_migrations.md @@ -1,508 +1,11 @@ --- -stage: Data Stores -group: Database -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2023-10-11' --- -# Background migrations +This document was moved to [another location](index.md). -WARNING: -Background migrations are strongly discouraged in favor of the new [batched background migrations framework](batched_background_migrations.md). -Check that documentation and determine if that framework suits your needs and fall back -to these only if required. - -Background migrations should be used to perform data migrations whenever a -migration exceeds [the time limits in our guidelines](../migration_style_guide.md#how-long-a-migration-should-take). For example, you can use background -migrations to migrate data that's stored in a single JSON column -to a separate table instead. - -If the database cluster is considered to be in an unhealthy state, background -migrations automatically reschedule themselves for a later point in time. - -## When To Use Background Migrations - -You should use a background migration when you migrate _data_ in tables that have -so many rows that the process would exceed [the time limits in our guidelines](../migration_style_guide.md#how-long-a-migration-should-take) if performed using a regular Rails migration. - -- Background migrations should be used when migrating data in [high-traffic tables](../migration_style_guide.md#high-traffic-tables). -- Background migrations may also be used when executing numerous single-row queries -for every item on a large dataset. Typically, for single-record patterns, runtime is -largely dependent on the size of the dataset, hence it should be split accordingly -and put into background migrations. -- Background migrations should not be used to perform schema migrations. - -Some examples where background migrations can be useful: - -- Migrating events from one table to multiple separate tables. -- Populating one column based on JSON stored in another column. -- Migrating data that depends on the output of external services (for example, an API). - -NOTE: -If the background migration is part of an important upgrade, make sure it's announced -in the release post. Discuss with your Project Manager if you're not sure the migration falls -into this category. - -## Isolation - -Background migrations must be isolated and cannot use application code (for example, -models defined in `app/models` except the `ApplicationRecord` classes). Since these migrations -can take a long time to run it's possible for new versions to be deployed while they are still running. - -It's also possible for different migrations to be executed at the same time. -This means that different background migrations should not migrate data in a -way that would cause conflicts. - -## Accessing data for multiple databases - -See [Accessing data for multiple databases of Batched Background Migrations](batched_background_migrations.md#accessing-data-for-multiple-databases) for more details. - -## Idempotence - -Background migrations are executed in a context of a Sidekiq process. -Usual Sidekiq rules apply, especially the rule that jobs should be small -and idempotent. - -See [Sidekiq best practices guidelines](https://github.com/mperham/sidekiq/wiki/Best-Practices) -for more details. - -Make sure that in case that your migration job is retried, data -integrity is guaranteed. - -## Background migrations for EE-only features - -All the background migration classes for EE-only features should be present in GitLab CE. -For this purpose, an empty class can be created for GitLab CE, and it can be extended for GitLab EE -as explained in the [guidelines for implementing Enterprise Edition features](../ee_features.md#code-in-libgitlabbackground_migration). - -## How It Works - -Background migrations are simple classes that define a `perform` method. A -Sidekiq worker then executes such a class, passing any arguments to it. All -migration classes must be defined in the namespace -`Gitlab::BackgroundMigration`, the files should be placed in the directory -`lib/gitlab/background_migration/`. - -## Scheduling - -Scheduling a background migration should be done in a post-deployment -migration that includes `Gitlab::Database::MigrationHelpers` -To do so, use the following code while -replacing the class name and arguments with whatever values are necessary for -your migration: - -```ruby -migrate_in('BackgroundMigrationClassName', [arg1, arg2, ...]) -``` - -You can use the function `queue_background_migration_jobs_by_range_at_intervals` -to automatically split the job into batches: - -```ruby -queue_background_migration_jobs_by_range_at_intervals( - ClassName, - 'BackgroundMigrationClassName', - 2.minutes, - batch_size: 10_000 - ) -``` - -You also need to make sure that newly created data is either migrated, or -saved in both the old and new version upon creation. For complex and time -consuming migrations it's best to schedule a background job using an -`after_create` hook so this doesn't affect response timings. The same applies to -updates. Removals in turn can be handled by defining foreign keys with -cascading deletes. - -### Rescheduling background migrations - -If one of the background migrations contains a bug that is fixed in a patch -release, the background migration needs to be rescheduled so the migration would -be repeated on systems that already performed the initial migration. - -When you reschedule the background migration, make sure to turn the original -scheduling into a no-op by clearing up the `#up` and `#down` methods of the -migration performing the scheduling. Otherwise the background migration would be -scheduled multiple times on systems that are upgrading multiple patch releases at -once. - -When you start the second post-deployment migration, you should delete any -previously queued jobs from the initial migration with the provided -helper: - -```ruby -delete_queued_jobs('BackgroundMigrationClassName') -``` - -## Cleaning Up - -NOTE: -Cleaning up any remaining background migrations _must_ be done in either a major -or minor release, you _must not_ do this in a patch release. - -Because background migrations can take a long time you can't immediately clean -things up after scheduling them. For example, you can't drop a column that's -used in the migration process as this would cause jobs to fail. This means that -you need to add a separate _post deployment_ migration in a future release -that finishes any remaining jobs before cleaning things up (for example, removing a -column). - -As an example, say you want to migrate the data from column `foo` (containing a -big JSON blob) to column `bar` (containing a string). The process for this would -roughly be as follows: - -1. Release A: - 1. Create a migration class that performs the migration for a row with a given ID. - You can use [background jobs tracking](#background-jobs-tracking) to simplify cleaning up. - 1. Deploy the code for this release, this should include some code that will - schedule jobs for newly created data (for example, using an `after_create` hook). - 1. Schedule jobs for all existing rows in a post-deployment migration. It's - possible some newly created rows may be scheduled twice so your migration - should take care of this. -1. Release B: - 1. Deploy code so that the application starts using the new column and stops - scheduling jobs for newly created data. - 1. In a post-deployment migration, finalize all jobs that have not succeeded by now. - If you used [background jobs tracking](#background-jobs-tracking) in release A, - you can use `finalize_background_migration` from `BackgroundMigrationHelpers` to ensure no jobs remain. - This helper will: - 1. Use `Gitlab::BackgroundMigration.steal` to process any remaining - jobs in Sidekiq. - 1. Reschedule the migration to be run directly (that is, not through Sidekiq) - on any rows that weren't migrated by Sidekiq. This can happen if, for - instance, Sidekiq received a SIGKILL, or if a particular batch failed - enough times to be marked as dead. - 1. Remove `Gitlab::Database::BackgroundMigrationJob` rows where - `status = succeeded`. To retain diagnostic information that may - help with future bug tracking you can skip this step by specifying - the `delete_tracking_jobs: false` parameter. - 1. Remove the old column. - -This may also require a bump to the [import/export version](../../user/project/settings/import_export.md), if -importing a project from a prior version of GitLab requires the data to be in -the new format. - -## Example - -To explain all this, let's use the following example: the table `integrations` has a -field called `properties` which is stored in JSON. For all rows you want to -extract the `url` key from this JSON object and store it in the `integrations.url` -column. There are millions of integrations and parsing JSON is slow, thus you can't -do this in a regular migration. - -To do this using a background migration we start with defining our migration -class: - -```ruby -class Gitlab::BackgroundMigration::ExtractIntegrationsUrl - class Integration < ::ApplicationRecord - self.table_name = 'integrations' - end - - def perform(start_id, end_id) - Integration.where(id: start_id..end_id).each do |integration| - json = JSON.load(integration.properties) - - integration.update(url: json['url']) if json['url'] - rescue JSON::ParserError - # If the JSON is invalid we don't want to keep the job around forever, - # instead we'll just leave the "url" field to whatever the default value - # is. - next - end - end -end -``` - -Next we need to adjust our code so we schedule the above migration for newly -created and updated integrations. We can do this using something along the lines of -the following: - -```ruby -class Integration < ::ApplicationRecord - after_commit :schedule_integration_migration, on: :update - after_commit :schedule_integration_migration, on: :create - - def schedule_integration_migration - BackgroundMigrationWorker.perform_async('ExtractIntegrationsUrl', [id, id]) - end -end -``` - -We're using `after_commit` here to ensure the Sidekiq job is not scheduled -before the transaction completes as doing so can lead to race conditions where -the changes are not yet visible to the worker. - -Next we need a post-deployment migration that schedules the migration for -existing data. - -```ruby -class ScheduleExtractIntegrationsUrl < Gitlab::Database::Migration[2.1] - disable_ddl_transaction! - - MIGRATION = 'ExtractIntegrationsUrl' - DELAY_INTERVAL = 2.minutes - - def up - queue_background_migration_jobs_by_range_at_intervals( - define_batchable_model('integrations'), - MIGRATION, - DELAY_INTERVAL) - end - - def down - end -end -``` - -After deployed our application continues using the data as before, but at the -same time ensures that both existing and new data is migrated. - -In the next release we can remove the `after_commit` hooks and related code. We -also need to add a post-deployment migration that consumes any remaining -jobs and manually run on any un-migrated rows. Such a migration would look like -this: - -```ruby -class ConsumeRemainingExtractIntegrationsUrlJobs < Gitlab::Database::Migration[2.1] - disable_ddl_transaction! - - def up - # This must be included - Gitlab::BackgroundMigration.steal('ExtractIntegrationsUrl') - - # This should be included, but can be skipped - see below - define_batchable_model('integrations').where(url: nil).each_batch(of: 50) do |batch| - range = batch.pluck('MIN(id)', 'MAX(id)').first - - Gitlab::BackgroundMigration::ExtractIntegrationsUrl.new.perform(*range) - end - end - - def down - end -end -``` - -The final step runs for any un-migrated rows after all of the jobs have been -processed. This is in case a Sidekiq process running the background migrations -received SIGKILL, leading to the jobs being lost. (See -[more reliable Sidekiq queue](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/36791) for more information.) - -If the application does not depend on the data being 100% migrated (for -instance, the data is advisory, and not mission-critical), then this final step -can be skipped. - -This migration then processes any jobs for the `ExtractIntegrationsUrl` migration -and continue once all jobs have been processed. Once done you can safely remove -the `integrations.properties` column. - -## Testing - -It is required to write tests for: - -- The background migrations' scheduling migration. -- The background migration itself. -- A cleanup migration. - -The `:migration` and `schema: :latest` RSpec tags are automatically set for -background migration specs. -See the -[Testing Rails migrations](../testing_guide/testing_migrations_guide.md#testing-a-non-activerecordmigration-class) -style guide. - -Keep in mind that `before` and `after` RSpec hooks are going -to migrate you database down and up, which can result in other background -migrations being called. That means that using `spy` test doubles with -`have_received` is encouraged, instead of using regular test doubles, because -your expectations defined in a `it` block can conflict with what is being -called in RSpec hooks. See [issue #35351](https://gitlab.com/gitlab-org/gitlab/-/issues/18839) -for more details. - -## Best practices - -1. Make sure to know how much data you're dealing with. -1. Make sure that background migration jobs are idempotent. -1. Make sure that tests you write are not false positives. -1. Make sure that if the data being migrated is critical and cannot be lost, the - clean-up migration also checks the final state of the data before completing. -1. When migrating many columns, make sure it does not generate too many - dead tuples in the process (you may need to directly query the number of dead tuples - and adjust the scheduling according to this piece of data). -1. Make sure to discuss the numbers with a database specialist, the migration may add - more pressure on DB than you expect (measure on staging, - or ask someone to measure on production). -1. Make sure to know how much time it takes to run all scheduled migrations. -1. Provide an estimation section in the description, estimating both the total migration - run time and the query times for each background migration job. Explain plans for each query - should also be provided. - - For example, assuming a migration that deletes data, include information similar to - the following section: - - ```plaintext - Background Migration Details: - - 47600 items to delete - batch size = 1000 - 47600 / 1000 = 48 batches - - Estimated times per batch: - - 820ms for select statement with 1000 items (see linked explain plan) - - 900ms for delete statement with 1000 items (see linked explain plan) - Total: ~2 sec per batch - - 2 mins delay per batch (safe for the given total time per batch) - - 48 batches * 2 min per batch = 96 mins to run all the scheduled jobs - ``` - - The execution time per batch (2 sec in this example) is not included in the calculation - for total migration time. The jobs are scheduled 2 minutes apart without knowledge of - the execution time. - -## Additional tips and strategies - -### Nested batching - -A strategy to make the migration run faster is to schedule larger batches, and then use `EachBatch` -within the background migration to perform multiple statements. - -The background migration helpers that queue multiple jobs such as -`queue_background_migration_jobs_by_range_at_intervals` use [`EachBatch`](iterating_tables_in_batches.md). -The example above has batches of 1000, where each queued job takes two seconds. If the query has been optimized -to make the time for the delete statement within the [query performance guidelines](query_performance.md), -1000 may be the largest number of records that can be deleted in a reasonable amount of time. - -The minimum and most common interval for delaying jobs is two minutes. This results in two seconds -of work for each two minute job. There's nothing that prevents you from executing multiple delete -statements in each background migration job. - -Looking at the example above, you could alternatively do: - -```plaintext -Background Migration Details: - -47600 items to delete -batch size = 10_000 -47600 / 10_000 = 5 batches - -Estimated times per batch: -- Records are updated in sub-batches of 1000 => 10_000 / 1000 = 10 total updates -- 820ms for select statement with 1000 items (see linked explain plan) -- 900ms for delete statement with 1000 items (see linked explain plan) -Sub-batch total: ~2 sec per sub-batch, -Total batch time: 2 * 10 = 20 sec per batch - -2 mins delay per batch - -5 batches * 2 min per batch = 10 mins to run all the scheduled jobs -``` - -The batch time of 20 seconds still fits comfortably within the two minute delay, yet the total run -time is cut by a tenth from around 100 minutes to 10 minutes! When dealing with large background -migrations, this can cut the total migration time by days. - -When batching in this way, it is important to look at query times on the higher end -of the table or relation being updated. `EachBatch` may generate some queries that become much -slower when dealing with higher ID ranges. - -### Delay time - -When looking at the batch execution time versus the delay time, the execution time -should fit comfortably within the delay time for a few reasons: - -- To allow for a variance in query times. -- To allow `autovacuum` to catch up after periods of high churn. - -Never try to optimize by fully filling the delay window even if you are confident -the queries themselves have no timing variance. - -### Background jobs tracking - -NOTE: -Background migrations with job tracking enabled must call `mark_all_as_succeeded` for its batch, even if no work is needed to be done. - -`queue_background_migration_jobs_by_range_at_intervals` can create records for each job that is scheduled to run. -You can enable this behavior by passing `track_jobs: true`. Each record starts with a `pending` status. Make sure that your worker updates the job status to `succeeded` by calling `Gitlab::Database::BackgroundMigrationJob.mark_all_as_succeeded` in the `perform` method of your background migration. - -```ruby -# Background migration code - -def perform(start_id, end_id) - # do work here - - mark_job_as_succeeded(start_id, end_id) -end - -private - -# Make sure that the arguments passed here match those passed to the background -# migration -def mark_job_as_succeeded(*arguments) - Gitlab::Database::BackgroundMigrationJob.mark_all_as_succeeded( - self.class.name.demodulize, - arguments - ) -end -``` - -```ruby -# Post deployment migration -MIGRATION = 'YourBackgroundMigrationName' -DELAY_INTERVAL = 2.minutes.to_i # can be different -BATCH_SIZE = 10_000 # can be different - -disable_ddl_transaction! - -def up - queue_background_migration_jobs_by_range_at_intervals( - define_batchable_model('name_of_the_table_backing_the_model'), - MIGRATION, - DELAY_INTERVAL, - batch_size: BATCH_SIZE, - track_jobs: true - ) -end - -def down - # no-op -end -``` - -See [`lib/gitlab/background_migration/drop_invalid_vulnerabilities.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/background_migration/drop_invalid_vulnerabilities.rb) for a full example. - -#### Rescheduling pending jobs - -You can reschedule pending migrations from the `background_migration_jobs` table by creating a post-deployment migration and calling `requeue_background_migration_jobs_by_range_at_intervals` with the migration name and delay interval. - -```ruby -# Post deployment migration -MIGRATION = 'YourBackgroundMigrationName' -DELAY_INTERVAL = 2.minutes - -disable_ddl_transaction! - -def up - requeue_background_migration_jobs_by_range_at_intervals(MIGRATION, DELAY_INTERVAL) -end - -def down - # no-op -end -``` - -See [`db/post_migrate/20210604070207_retry_backfill_traversal_ids.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/db/post_migrate/20210604070207_retry_backfill_traversal_ids.rb) for a full example. - -### Viewing failure error logs - -After running a background migration, if any jobs have failed, you can view the logs in [Kibana](https://log.gprd.gitlab.net/goto/5f06a57f768c6025e1c65aefb4075694). -View the production Sidekiq log and filter for: - -- `json.class: BackgroundMigrationWorker` -- `json.job_status: fail` -- `json.meta.caller_id: <MyBackgroundMigrationSchedulingMigrationClassName>` -- `json.args: <MyBackgroundMigrationClassName>` - -Looking at the `json.exception.class`, `json.exception.message`, `json.exception.backtrace`, and `json.exception.sql` values may be helpful in understanding why the jobs failed. - -Depending on when and how the failure occurred, you may find other helpful information by filtering with `json.class: <MyBackgroundMigrationClassName>`. +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/database/batched_background_migrations.md b/doc/development/database/batched_background_migrations.md index 3e54a78757a..1bdd24afcad 100644 --- a/doc/development/database/batched_background_migrations.md +++ b/doc/development/database/batched_background_migrations.md @@ -39,10 +39,10 @@ Background migrations can help when: - If the batched background migration is part of an important upgrade, it must be announced in the release post. Discuss with your Project Manager if you're unsure if the migration falls into this category. -- You should use the [generator](#generator) to create batched background migrations, +- You should use the [generator](#generate-a-batched-background-migration) to create batched background migrations, so that required files are created by default. -## How it works +## How batched background migrations work Batched background migrations (BBM) are subclasses of `Gitlab::BackgroundMigration::BatchedMigrationJob` that define a `perform` method. @@ -201,66 +201,25 @@ models defined in `app/models` except the `ApplicationRecord` classes). Because these migrations can take a long time to run, it's possible for new versions to deploy while the migrations are still running. -## Accessing data for multiple databases +## How to -Background Migration contrary to regular migrations does have access to multiple databases -and can be used to efficiently access and update data across them. To properly indicate -a database to be used it is desired to create ActiveRecord model inline the migration code. -Such model should use a correct [`ApplicationRecord`](multiple_databases.md#gitlab-schema) -depending on which database the table is located. As such usage of `ActiveRecord::Base` -is disallowed as it does not describe a explicitly database to be used to access given table. - -```ruby -# good -class Gitlab::BackgroundMigration::ExtractIntegrationsUrl - class Project < ::ApplicationRecord - self.table_name = 'projects' - end - - class Build < ::Ci::ApplicationRecord - self.table_name = 'ci_builds' - end -end +### Generate a batched background migration -# bad -class Gitlab::BackgroundMigration::ExtractIntegrationsUrl - class Project < ActiveRecord::Base - self.table_name = 'projects' - end - - class Build < ActiveRecord::Base - self.table_name = 'ci_builds' - end -end -``` - -Similarly the usage of `ActiveRecord::Base.connection` is disallowed and needs to be -replaced preferably with the usage of model connection. - -```ruby -# good -Project.connection.execute("SELECT * FROM projects") - -# acceptable -ApplicationRecord.connection.execute("SELECT * FROM projects") +The custom generator `batched_background_migration` scaffolds necessary files and +accepts `table_name`, `column_name`, and `feature_category` as arguments. Usage: -# bad -ActiveRecord::Base.connection.execute("SELECT * FROM projects") +```shell +bundle exec rails g batched_background_migration my_batched_migration --table_name=<table-name> --column_name=<column-name> --feature_category=<feature-category> ``` -## Batched background migrations for EE-only features - -All the background migration classes for EE-only features should be present in GitLab FOSS. -For this purpose, create an empty class for GitLab FOSS, and extend it for GitLab EE -as explained in the guidelines for -[implementing Enterprise Edition features](../ee_features.md#code-in-libgitlabbackground_migration). +This command creates the following files: -NOTE: -Background migration classes for EE-only features that use job arguments should define them -in the GitLab FOSS class. This is required to prevent job arguments validation from failing when -migration is scheduled in GitLab FOSS context. +- `db/post_migrate/20230214231008_queue_my_batched_migration.rb` +- `spec/migrations/20230214231008_queue_my_batched_migration_spec.rb` +- `lib/gitlab/background_migration/my_batched_migration.rb` +- `spec/lib/gitlab/background_migration/my_batched_migration_spec.rb` -## Queueing +### Enqueue a batched background migration Queueing a batched background migration should be done in a post-deployment migration. Use this `queue_batched_background_migration` example, queueing the @@ -278,63 +237,13 @@ queue_batched_background_migration( NOTE: This helper raises an error if the number of provided job arguments does not match -the number of [job arguments](#job-arguments) defined in `JOB_CLASS_NAME`. +the number of [job arguments](#use-job-arguments) defined in `JOB_CLASS_NAME`. Make sure the newly-created data is either migrated, or saved in both the old and new version upon creation. Removals in turn can be handled by defining foreign keys with cascading deletes. -### Requeuing batched background migrations - -If one of the batched background migrations contains a bug that is fixed in a patch -release, you must requeue the batched background migration so the migration -repeats on systems that already performed the initial migration. - -When you requeue the batched background migration, turn the original -queuing into a no-op by clearing up the `#up` and `#down` methods of the -migration performing the requeuing. Otherwise, the batched background migration is -queued multiple times on systems that are upgrading multiple patch releases at -once. - -When you start the second post-deployment migration, delete the -previously batched migration with the provided code: - -```ruby -delete_batched_background_migration(MIGRATION_NAME, TABLE_NAME, COLUMN, JOB_ARGUMENTS) -``` - -## Cleaning up - -NOTE: -Cleaning up any remaining background migrations must be done in either a major -or minor release. You must not do this in a patch release. - -Because background migrations can take a long time, you can't immediately clean -things up after queueing them. For example, you can't drop a column used in the -migration process, as jobs would fail. You must add a separate _post-deployment_ -migration in a future release that finishes any remaining -jobs before cleaning things up. (For example, removing a column.) - -To migrate the data from column `foo` (containing a big JSON blob) to column `bar` -(containing a string), you would: - -1. Release A: - 1. Create a migration class that performs the migration for a row with a given ID. - 1. Update new rows using one of these techniques: - - Create a new trigger for simple copy operations that don't need application logic. - - Handle this operation in the model/service as the records are created or updated. - - Create a new custom background job that updates the records. - 1. Queue the batched background migration for all existing rows in a post-deployment migration. -1. Release B: - 1. Add a post-deployment migration that checks if the batched background migration is completed. - 1. Deploy code so that the application starts using the new column and stops to update new records. - 1. Remove the old column. - -Bump to the [import/export version](../../user/project/settings/import_export.md) may -be required, if importing a project from a prior version of GitLab requires the -data to be in the new format. - -## Job arguments +### Use job arguments `BatchedMigrationJob` provides the `job_arguments` helper method for job classes to define the job arguments they need. @@ -373,7 +282,7 @@ class CopyColumnUsingBackgroundMigrationJob < BatchedMigrationJob end ``` -## Additional filters +### Use filters By default, when creating background jobs to perform the migration, batched background migrations iterate over the full specified table. This iteration is done using the @@ -450,21 +359,344 @@ NOTE: When applying additional filters, it is important to ensure they are properly covered by an index to optimize `EachBatch` performance. In the example above we need an index on `(type, id)` to support the filters. See [the `EachBatch` documentation for more information](iterating_tables_in_batches.md). -## Generator +### Access data for multiple databases -The custom generator `batched_background_migration` scaffolds necessary files and -accepts `table_name`, `column_name`, and `feature_category` as arguments. Usage: +Background Migration contrary to regular migrations does have access to multiple databases +and can be used to efficiently access and update data across them. To properly indicate +a database to be used it is desired to create ActiveRecord model inline the migration code. +Such model should use a correct [`ApplicationRecord`](multiple_databases.md#gitlab-schema) +depending on which database the table is located. As such usage of `ActiveRecord::Base` +is disallowed as it does not describe a explicitly database to be used to access given table. -```shell -bundle exec rails g batched_background_migration my_batched_migration --table_name=<table-name> --column_name=<column-name> --feature_category=<feature-category> +```ruby +# good +class Gitlab::BackgroundMigration::ExtractIntegrationsUrl + class Project < ::ApplicationRecord + self.table_name = 'projects' + end + + class Build < ::Ci::ApplicationRecord + self.table_name = 'ci_builds' + end +end + +# bad +class Gitlab::BackgroundMigration::ExtractIntegrationsUrl + class Project < ActiveRecord::Base + self.table_name = 'projects' + end + + class Build < ActiveRecord::Base + self.table_name = 'ci_builds' + end +end ``` -This command creates these files: +Similarly the usage of `ActiveRecord::Base.connection` is disallowed and needs to be +replaced preferably with the usage of model connection. -- `db/post_migrate/20230214231008_queue_my_batched_migration.rb` -- `spec/migrations/20230214231008_queue_my_batched_migration_spec.rb` -- `lib/gitlab/background_migration/my_batched_migration.rb` -- `spec/lib/gitlab/background_migration/my_batched_migration_spec.rb` +```ruby +# good +Project.connection.execute("SELECT * FROM projects") + +# acceptable +ApplicationRecord.connection.execute("SELECT * FROM projects") + +# bad +ActiveRecord::Base.connection.execute("SELECT * FROM projects") +``` + +### Re-queue batched background migrations + +A batched background migration might need to be re-run for one of several +reasons: + +- The migration contains a bug ([example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93546)). +- The migration cleaned up data but the data became de-normalized again due to a + bypass in application logic ([example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123002)). +- The batch size of the original migration causes the migration to fail ([example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121404)). + +To requeue a batched background migration, you must: + +- No-op the contents of the `#up` and `#down` methods of the + original migration file. Otherwise, the batched background migration is created, + deleted, then created again on systems that are upgrading multiple patch + releases at once. +- Add a new post-deployment migration that re-runs the batched background + migration. +- In the new post-deployment migration, delete the existing batched background + migration using the `delete_batched_background_migration` method at the start + of the `#up` method to ensure that any existing runs are cleaned up. +- Update the `db/docs/batched_background_migration/*.yml` file from the original + migration to include information about the requeue. + +### Batch over non-distinct columns + +The default batching strategy provides an efficient way to iterate over primary key columns. +However, if you need to iterate over columns where values are not unique, you must use a +different batching strategy. + +The `LooseIndexScanBatchingStrategy` batching strategy uses a special version of [`EachBatch`](iterating_tables_in_batches.md#loose-index-scan-with-distinct_each_batch) +to provide efficient and stable iteration over the distinct column values. + +This example shows a batched background migration where the `issues.project_id` column is used as +the batching column. + +Database post-migration: + +```ruby +class ProjectsWithIssuesMigration < Gitlab::Database::Migration[2.1] + MIGRATION = 'BatchProjectsWithIssues' + INTERVAL = 2.minutes + BATCH_SIZE = 5000 + SUB_BATCH_SIZE = 500 + restrict_gitlab_migration gitlab_schema: :gitlab_main + + disable_ddl_transaction! + def up + queue_batched_background_migration( + MIGRATION, + :issues, + :project_id, + job_interval: INTERVAL, + batch_size: BATCH_SIZE, + batch_class_name: 'LooseIndexScanBatchingStrategy', # Override the default batching strategy + sub_batch_size: SUB_BATCH_SIZE + ) + end + + def down + delete_batched_background_migration(MIGRATION, :issues, :project_id, []) + end +end +``` + +Implementing the background migration class: + +```ruby +module Gitlab + module BackgroundMigration + class BatchProjectsWithIssues < Gitlab::BackgroundMigration::BatchedMigrationJob + include Gitlab::Database::DynamicModelHelpers + + operation_name :backfill_issues + + def perform + distinct_each_batch do |batch| + project_ids = batch.pluck(batch_column) + # do something with the distinct project_ids + end + end + end + end +end +``` + +NOTE: +[Additional filters](#use-filters) defined with `scope_to` are ignored by `LooseIndexScanBatchingStrategy` and `distinct_each_batch`. + +### Calculate overall time estimation of a batched background migration + +It's possible to estimate how long a BBM takes to complete. GitLab already provides an estimation through the `db:gitlabcom-database-testing` pipeline. +This estimation is built based on sampling production data in a test environment and represents the max time that the migration could take and, not necessarily, +the actual time that the migration takes. In certain scenarios, estimations provided by the `db:gitlabcom-database-testing` pipeline may not be enough to +calculate all the singularities around the records being migrated, making further calculations necessary. As it made necessary, the formula +`interval * number of records / max batch size` can be used to determine an approximate estimation of how long the migration takes. +Where `interval` and `max batch size` refer to options defined for the job, and the `total tuple count` is the number of records to be migrated. + +NOTE: +Estimations may be affected by the [migration optimization mechanism](#migration-optimization). + +### Cleaning up a batched background migration + +NOTE: +Cleaning up any remaining background migrations must be done in either a major +or minor release. You must not do this in a patch release. + +Because background migrations can take a long time, you can't immediately clean +things up after queueing them. For example, you can't drop a column used in the +migration process, as jobs would fail. You must add a separate _post-deployment_ +migration in a future release that finishes any remaining +jobs before cleaning things up. (For example, removing a column.) + +To migrate the data from column `foo` (containing a big JSON blob) to column `bar` +(containing a string), you would: + +1. Release A: + 1. Create a migration class that performs the migration for a row with a given ID. + 1. Update new rows using one of these techniques: + - Create a new trigger for copy operations that don't need application logic. + - Handle this operation in the model/service as the records are created or updated. + - Create a new custom background job that updates the records. + 1. Queue the batched background migration for all existing rows in a post-deployment migration. +1. Release B: + 1. Add a post-deployment migration that checks if the batched background migration is completed. + 1. Deploy code so that the application starts using the new column and stops to update new records. + 1. Remove the old column. + +Bumping the [import/export version](../../user/project/settings/import_export.md) may +be required, if importing a project from a prior version of GitLab requires the +data to be in the new format. + +## Managing + +NOTE: +BBM management takes place through `chatops` integration, which is limited to GitLab team members only. + +### List batched background migrations + +To list the batched background migrations in the system, run this command: + +`/chatops run batched_background_migrations list` + +This command supports the following options: + +- Database selection: + - `--database DATABASE_NAME`: Connects to the given database: + - `main`: Uses the main database (default). + - `ci`: Uses the CI database. +- Environment selection: + - `--dev`: Uses the `dev` environment. + - `--staging`: Uses the `staging` environment. + - `--staging_ref`: Uses the `staging_ref` environment. + - `--production` : Uses the `production` environment (default). + +Output example: + + + +NOTE: +ChatOps returns 20 batched background migrations order by `created_at` (DESC). + +### Monitor the progress and status of a batched background migration + +To see the status and progress of a specific batched background migration, run this command: + +`/chatops run batched_background_migrations status MIGRATION_ID` + +This command supports the following options: + +- Database selection: + - `--database DATABASE_NAME`: Connects to the given database: + - `main`: Uses the main database (default) + - `ci`: Uses the CI database +- Environment selection: + - `--dev`: Uses the `dev` environment. + - `--staging`: Uses the `staging` environment. + - `--staging_ref`: Uses the `staging_ref` environment. + - `--production` : Uses the `production` environment (default). + +Output example: + + + +`Progress` represents the percentage of the background migration that has been completed. + +Definitions of the batched background migration states: + +- **Active:** Either: + - Ready to be picked by the runner. + - Running batched jobs. +- **Finalizing:** Running batched jobs. +- **Failed:** Failed batched background migration. +- **Finished:** Completed batched background migration. +- **Paused:** Not visible to the runner. + +### Pause a batched background migration + +If you want to pause a batched background migration, you need to run the following command: + +`/chatops run batched_background_migrations pause MIGRATION_ID` + +This command supports the following options: + +- Database selection: + - `--database DATABASE_NAME`: Connects to the given database: + - `main`: Uses the main database (default). + - `ci`: Uses the CI database. +- Environment selection: + - `--dev`: Uses the `dev` environment. + - `--staging`: Uses the `staging` environment. + - `--staging_ref`: Uses the `staging_ref` environment. + - `--production` : Uses the `production` environment (default). + +Output example: + + + +NOTE: +You can pause only `active` batched background migrations. + +### Resume a batched background migration + +If you want to resume a batched background migration, you need to run the following command: + +`/chatops run batched_background_migrations resume MIGRATION_ID` + +This command supports the following options: + +- Database selection: + - `--database DATABASE_NAME`: Connects to the given database: + - `main`: Uses the main database (default). + - `ci`: Uses the CI database. +- Environment selection: + - `--dev`: Uses the `dev` environment. + - `--staging`: Uses the `staging` environment. + - `--staging_ref`: Uses the `staging_ref` environment. + - `--production` : Uses the `production` environment (default). + +Output example: + + + +NOTE: +You can resume only `active` batched background migrations + +### Enable or disable background migrations + +In extremely limited circumstances, a GitLab administrator can disable either or +both of these [feature flags](../../administration/feature_flags.md): + +- `execute_background_migrations` +- `execute_batched_migrations_on_schedule` + +These flags are enabled by default. Disable them only as a last resort +to limit database operations in special circumstances, like database host maintenance. + +WARNING: +Do not disable either of these flags unless you fully understand the ramifications. If you disable +the `execute_background_migrations` or `execute_batched_migrations_on_schedule` feature flag, +GitLab upgrades might fail and data loss might occur. + +## Batched background migrations for EE-only features + +All the background migration classes for EE-only features should be present in GitLab FOSS. +For this purpose, create an empty class for GitLab FOSS, and extend it for GitLab EE +as explained in the guidelines for +[implementing Enterprise Edition features](../ee_features.md#code-in-libgitlabbackground_migration). + +NOTE: +Background migration classes for EE-only features that use job arguments should define them +in the GitLab FOSS class. Definitions are required to prevent job arguments validation from failing when +migration is scheduled in the GitLab FOSS context. + +You can use the [generator](#generate-a-batched-background-migration) to generate an EE-only migration scaffold by passing +`--ee-only` flag when generating a new batched background migration. + +## Throttling batched migrations + +Because batched migrations are update-heavy and there were few incidents in the past because of the heavy load from migrations while the database was underperforming, a throttling mechanism exists to mitigate them. + +These database indicators are checked to throttle a migration. On getting a +stop signal, the migration is paused for a set time (10 minutes): + +- WAL queue pending archival crossing a threshold. +- Active autovacuum on the tables on which the migration works on. +- Patroni apdex SLI dropping below the SLO. + +It's an ongoing effort to add more indicators to further enhance the +database health check framework. For more details, see +[epic 7594](https://gitlab.com/groups/gitlab-org/-/epics/7594). ## Example @@ -636,71 +868,6 @@ background migration. After the batched migration is completed, you can safely depend on the data in `routes.namespace_id` being populated. -### Batching over non-distinct columns - -The default batching strategy provides an efficient way to iterate over primary key columns. -However, if you need to iterate over columns where values are not unique, you must use a -different batching strategy. - -The `LooseIndexScanBatchingStrategy` batching strategy uses a special version of [`EachBatch`](iterating_tables_in_batches.md#loose-index-scan-with-distinct_each_batch) -to provide efficient and stable iteration over the distinct column values. - -This example shows a batched background migration where the `issues.project_id` column is used as -the batching column. - -Database post-migration: - -```ruby -class ProjectsWithIssuesMigration < Gitlab::Database::Migration[2.1] - MIGRATION = 'BatchProjectsWithIssues' - INTERVAL = 2.minutes - BATCH_SIZE = 5000 - SUB_BATCH_SIZE = 500 - restrict_gitlab_migration gitlab_schema: :gitlab_main - - disable_ddl_transaction! - def up - queue_batched_background_migration( - MIGRATION, - :issues, - :project_id, - job_interval: INTERVAL, - batch_size: BATCH_SIZE, - batch_class_name: 'LooseIndexScanBatchingStrategy', # Override the default batching strategy - sub_batch_size: SUB_BATCH_SIZE - ) - end - - def down - delete_batched_background_migration(MIGRATION, :issues, :project_id, []) - end -end -``` - -Implementing the background migration class: - -```ruby -module Gitlab - module BackgroundMigration - class BatchProjectsWithIssues < Gitlab::BackgroundMigration::BatchedMigrationJob - include Gitlab::Database::DynamicModelHelpers - - operation_name :backfill_issues - - def perform - distinct_each_batch do |batch| - project_ids = batch.pluck(batch_column) - # do something with the distinct project_ids - end - end - end - end -end -``` - -NOTE: -[Additional filters](#additional-filters) defined with `scope_to` are ignored by `LooseIndexScanBatchingStrategy` and `distinct_each_batch`. - ## Testing Writing tests is required for: @@ -771,115 +938,6 @@ for more details. The batched background migrations framework has ChatOps support. Using ChatOps, GitLab engineers can interact with the batched background migrations present in the system. -#### List batched background migrations - -To list the batched background migrations in the system, run this command: - -`/chatops run batched_background_migrations list` - -This command supports the following options: - -- Database selection: - - `--database DATABASE_NAME`: Connects to the given database: - - `main`: Uses the main database (default). - - `ci`: Uses the CI database. -- Environment selection: - - `--dev`: Uses the `dev` environment. - - `--staging`: Uses the `staging` environment. - - `--staging_ref`: Uses the `staging_ref` environment. - - `--production` : Uses the `production` environment (default). - -Output example: - - - -NOTE: -ChatOps returns 20 batched background migrations order by `created_at` (DESC). - -#### Monitor the progress and status of a batched background migration - -To see the status and progress of a specific batched background migration, run this command: - -`/chatops run batched_background_migrations status MIGRATION_ID` - -This command supports the following options: - -- Database selection: - - `--database DATABASE_NAME`: Connects to the given database: - - `main`: Uses the main database (default) - - `ci`: Uses the CI database -- Environment selection: - - `--dev`: Uses the `dev` environment. - - `--staging`: Uses the `staging` environment. - - `--staging_ref`: Uses the `staging_ref` environment. - - `--production` : Uses the `production` environment (default). - -Output example: - - - -`Progress` represents the percentage of the background migration that has been completed. - -Definitions of the batched background migration states: - -- **Active:** Either: - - Ready to be picked by the runner. - - Running batched jobs. -- **Finalizing:** Running batched jobs. -- **Failed:** Failed batched background migration. -- **Finished:** Completed batched background migration. -- **Paused:** Not visible to the runner. - -#### Pause a batched background migration - -If you want to pause a batched background migration, you need to run the following command: - -`/chatops run batched_background_migrations pause MIGRATION_ID` - -This command supports the following options: - -- Database selection: - - `--database DATABASE_NAME`: Connects to the given database: - - `main`: Uses the main database (default). - - `ci`: Uses the CI database. -- Environment selection: - - `--dev`: Uses the `dev` environment. - - `--staging`: Uses the `staging` environment. - - `--staging_ref`: Uses the `staging_ref` environment. - - `--production` : Uses the `production` environment (default). - -Output example: - - - -NOTE: -You can pause only `active` batched background migrations. - -#### Resume a batched background migration - -If you want to resume a batched background migration, you need to run the following command: - -`/chatops run batched_background_migrations resume MIGRATION_ID` - -This command supports the following options: - -- Database selection: - - `--database DATABASE_NAME`: Connects to the given database: - - `main`: Uses the main database (default). - - `ci`: Uses the CI database. -- Environment selection: - - `--dev`: Uses the `dev` environment. - - `--staging`: Uses the `staging` environment. - - `--staging_ref`: Uses the `staging_ref` environment. - - `--production` : Uses the `production` environment (default). - -Output example: - - - -NOTE: -You can resume only `active` batched background migrations - ### Viewing failure error logs You can view failures in two ways: @@ -1020,5 +1078,5 @@ creation. ## Legacy background migrations -Batched background migrations replaced the [legacy background migrations framework](background_migrations.md). +Batched background migrations replaced the legacy background migrations framework. Check that documentation in reference to any changes involving that framework. diff --git a/doc/development/database/database_reviewer_guidelines.md b/doc/development/database/database_reviewer_guidelines.md index 933bbe9c060..c297c90918f 100644 --- a/doc/development/database/database_reviewer_guidelines.md +++ b/doc/development/database/database_reviewer_guidelines.md @@ -26,17 +26,23 @@ For more information on the database review process, check the [database review ## How to apply for becoming a database reviewer -Team members are encouraged to self-identify as database domain experts, and add it to their profile YAML file: +Team members are encouraged to self-identify as database domain experts, by adding it +to your profile YAML file: -```yaml -projects: - gitlab: - - reviewer database -``` +1. Make a merge request using the + [`Database reviewer` template](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/.gitlab/merge_request_templates/Database%20reviewer.md). +1. Add your database expertise to your YAML file: -Create the merge request [using the "Database reviewer" template](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/.gitlab/merge_request_templates/Database%20reviewer.md), -adding your expertise your profile YAML file. Assign to a database maintainer or the -[Database Team's Engineering Manager](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/). + ```yaml + projects: + gitlab: + - reviewer database + ``` + +1. Create the merge request + [using the "Database reviewer" template](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/.gitlab/merge_request_templates/Database%20reviewer.md). +1. Assign to a database maintainer or the + [Database Team's Engineering Manager](https://about.gitlab.com/handbook/engineering/development/enablement/data_stores/database/). After the `team.yml` update is merged, the [Reviewer roulette](../code_review.md#reviewer-roulette) may recommend you as a database reviewer. @@ -75,21 +81,10 @@ topics and use cases. The most frequently required during database reviewing are ## How to apply to become a database maintainer -Once a database reviewer feels confident on switching to a database maintainer, -they can update their [team profile](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/team.yml) -to a `trainee_maintainer database`: - -```yaml -projects: - gitlab: - - trainee_maintainer database -``` - -The first step is to a create a [Trainee Database Maintainer Issue](https://gitlab.com/gitlab-com/www-gitlab-com/-/issues/new?issuable_template=trainee-database-maintainer). -Use and follow the process described in the 'Trainee database maintainer' template. +Database maintainership uses the same process as other projects for identifying maintainers. +[Follow the general process documented here](https://about.gitlab.com/handbook/engineering/workflow/code-review/#how-to-become-a-project-maintainer). -Note that [trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) -are three times as likely to be picked by the [Danger bot](../dangerbot.md) as other reviewers. +For database specific requirements, please see [`Project maintainer process for gitlab-database`](https://about.gitlab.com/handbook/engineering/workflow/code-review/#project-maintainer-process-for-gitlab-database) ## What to do if you feel overwhelmed diff --git a/doc/development/database/efficient_in_operator_queries.md b/doc/development/database/efficient_in_operator_queries.md index a770dfe6531..a0c71f49e2d 100644 --- a/doc/development/database/efficient_in_operator_queries.md +++ b/doc/development/database/efficient_in_operator_queries.md @@ -117,7 +117,7 @@ On average, we can say the following: From the list, it's apparent that the number of `issues` records has the largest impact on the performance. -As per normal usage, we can say that the number of issue records grows +As per typical usage, we can say that the number of issue records grows at a faster rate than the `namespaces` and the `projects` records. This problem affects most of our group-level features where records are listed @@ -672,7 +672,7 @@ end #### Ordering by `JOIN` columns Ordering records by mixed columns where one or more columns are coming from `JOIN` tables -works with limitations. It requires extra configuration (CTE). The trick is to use a +works with limitations. It requires extra configuration via Common Table Expression (CTE). The trick is to use a non-materialized CTE to act as a "fake" table which exposes all required columns. NOTE: diff --git a/doc/development/database/index.md b/doc/development/database/index.md index f532e054849..1ee6aeaa213 100644 --- a/doc/development/database/index.md +++ b/doc/development/database/index.md @@ -8,6 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Database Reviews +- During the design phase of the feature you're working on, be mindful if you are adding any database-related changes. If you're adding or modifying a query, start looking at the `explain` plan early to avoid surprises late in the review phase. +- If, at any time, you need help optimizing a query or understanding an `explain` plan, ask for assistance in `#database`. - If you're creating a database MR for review, check out our [Database review guidelines](../database_review.md). It provides an introduction on database-related changes, migrations, and complex SQL queries. @@ -30,7 +32,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w - [Create a regular migration](../migration_style_guide.md#create-a-regular-schema-migration), including creating new models - [Deleting migrations](deleting_migrations.md) - [Different types of migrations](../migration_style_guide.md#choose-an-appropriate-migration-type) -- [Legacy background migrations guidelines](background_migrations.md) - [Migrations for multiple databases](migrations_for_multiple_databases.md) - [Migrations style guide](../migration_style_guide.md) for creating safe SQL migrations - [Partitioning tables](table_partitioning.md) diff --git a/doc/development/database/insert_into_tables_in_batches.md b/doc/development/database/insert_into_tables_in_batches.md index 78aa793f28b..547ca25b264 100644 --- a/doc/development/database/insert_into_tables_in_batches.md +++ b/doc/development/database/insert_into_tables_in_batches.md @@ -141,7 +141,7 @@ the key differences between these classes are listed in the table below. | `insert_all!` | Attribute hashes | No | No | Yes | Yes | To summarize, `BulkInsertSafe` moves bulk inserts closer to how ActiveRecord objects -and inserts would normally behave. However, if all you need is to insert raw data in bulk, then +and inserts would usually behave. However, if all you need is to insert raw data in bulk, then `insert_all` is more efficient. ## Insert `has_many` associations in bulk diff --git a/doc/development/database/keyset_pagination.md b/doc/development/database/keyset_pagination.md index 42d7458b45a..ff8038ee24c 100644 --- a/doc/development/database/keyset_pagination.md +++ b/doc/development/database/keyset_pagination.md @@ -159,7 +159,7 @@ configuration is necessary: - Function-based ordering. - Ordering with a custom tie-breaker column, like `iid`. -These order objects can be defined in the model classes as normal ActiveRecord scopes, there is no special behavior that prevents using these scopes elsewhere (Kaminari, background jobs). +These order objects can be defined in the model classes as standard ActiveRecord scopes, there is no special behavior that prevents using these scopes elsewhere (Kaminari, background jobs). ### `NULLS LAST` ordering diff --git a/doc/development/database/load_balancing.md b/doc/development/database/load_balancing.md index f623ad1eab0..f6e145ac7f5 100644 --- a/doc/development/database/load_balancing.md +++ b/doc/development/database/load_balancing.md @@ -21,7 +21,7 @@ is implemented in GitLab Rails and Sidekiq. ## Components -F few Ruby classes are involved in the load balancing process. All of them are +A few Ruby classes are involved in the load balancing process. All of them are in the namespace `Gitlab::Database::LoadBalancing`: 1. `Host` diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index 91a22d8c26b..fd380bee385 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -370,7 +370,7 @@ end ``` This endpoint still works when the parent `Project` model is deleted. This can be considered a -a data leak which should not happen under normal circumstances: +a data leak which should not happen under typical circumstances: ```ruby def show @@ -723,7 +723,7 @@ timeout or a worker crash, the next job continues the processing. ### Accumulation of deleted records There can be cases where the workers need to process an unusually large amount of data. This can -happen under normal usage, for example when a large project or group is deleted. In this scenario, +happen under typical usage, for example when a large project or group is deleted. In this scenario, there can be several million rows to be deleted or nullified. Due to the limits enforced by the worker, processing this data takes some time. diff --git a/doc/development/database/migrations_for_multiple_databases.md b/doc/development/database/migrations_for_multiple_databases.md index b903c56651d..ae9348a2090 100644 --- a/doc/development/database/migrations_for_multiple_databases.md +++ b/doc/development/database/migrations_for_multiple_databases.md @@ -32,8 +32,8 @@ Migrations cannot mix **DDL** and **DML** changes as the application requires th The DDL migrations are all migrations that: 1. Create or drop a table (for example, `create_table`). -1. Add or remove an index (for example, `add_index`, `add_index_concurrently`). -1. Add or remove a foreign key (for example `add_foreign_key`, `add_foreign_key_concurrently`). +1. Add or remove an index (for example, `add_index`, `add_concurrent_index`). +1. Add or remove a foreign key (for example `add_foreign_key`, `add_concurrent_foreign_key`). 1. Add or remove a column with or without a default value (for example, `add_column`). 1. Create or drop trigger functions (for example, `create_trigger_function`). 1. Attach or detach triggers from tables (for example, `track_record_deletions`, `untrack_record_deletions`). diff --git a/doc/development/database/multiple_databases.md b/doc/development/database/multiple_databases.md index 6adfdc90cf2..7c6b94144b4 100644 --- a/doc/development/database/multiple_databases.md +++ b/doc/development/database/multiple_databases.md @@ -26,8 +26,9 @@ Each table of GitLab needs to have a `gitlab_schema` assigned: - `gitlab_main`: describes all tables that are being stored in the `main:` database (for example, like `projects`, `users`). - `gitlab_ci`: describes all CI tables that are being stored in the `ci:` database (for example, `ci_pipelines`, `ci_builds`). - `gitlab_geo`: describes all Geo tables that are being stored in the `geo:` database (for example, like `project_registry`, `secondary_usage_data`). -- `gitlab_shared`: describe all application tables that contain data across all decomposed databases (for example, `loose_foreign_keys_deleted_records`) for models that inherit from `Gitlab::Database::SharedModel`. -- `gitlab_internal`: describe all internal tables of Rails and PostgreSQL (for example, `ar_internal_metadata`, `schema_migrations`, `pg_*`). +- `gitlab_shared`: describes all application tables that contain data across all decomposed databases (for example, `loose_foreign_keys_deleted_records`) for models that inherit from `Gitlab::Database::SharedModel`. +- `gitlab_internal`: describes all internal tables of Rails and PostgreSQL (for example, `ar_internal_metadata`, `schema_migrations`, `pg_*`). +- `gitlab_pm`: describes all tables that store `package_metadata` (it is an alias for `gitlab_main`). - `...`: more schemas to be introduced with additional decomposed databases The usage of schema enforces the base class to be used: @@ -36,6 +37,7 @@ The usage of schema enforces the base class to be used: - `Ci::ApplicationRecord` for `gitlab_ci` - `Geo::TrackingBase` for `gitlab_geo` - `Gitlab::Database::SharedModel` for `gitlab_shared` +- `PackageMetadata::ApplicationRecord` for `gitlab_pm` ### The impact of `gitlab_schema` @@ -70,6 +72,10 @@ all decomposed databases. `gitlab_internal` describes Rails-defined tables (like `schema_migrations` or `ar_internal_metadata`), as well as internal PostgreSQL tables (for example, `pg_attribute`). Its primary purpose is to [support other databases](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85842#note_943453682), like Geo, that might be missing some of those application-defined `gitlab_shared` tables (like `loose_foreign_keys_deleted_records`), but are valid Rails databases. +### The special purpose of `gitlab_pm` + +`gitlab_pm` stores package metadata describing public repositories. This data is used for the License Compliance and Dependency Scanning product categories and is maintained by the [Composition Analysis Group](https://about.gitlab.com/handbook/engineering/development/sec/secure/composition-analysis). It is an alias for `gitlab_main` intended to make it easier to route to a different database in the future. + ## Migrations Read [Migrations for Multiple Databases](migrations_for_multiple_databases.md). @@ -598,6 +604,13 @@ to limit the modes where tests can run, and skip them on any other modes. | `skip_if_multiple_databases_are_setup(:ci)` | Only on **single-db** | | `skip_if_multiple_databases_not_setup(:ci)` | On **single-db-ci-connection** and **multiple databases** | +## Testing for multiple databases, including main_clusterwide + +By default, we do not setup the `main_clusterwide` connection in CI pipelines. However, if you add the label `~"pipeline:run-clusterwide-db"`, the pipelines will run with 3 connections, `main`, `ci` and `main_clusterwide`. + +NOTE: +This setup is not completely ready yet, and running pipelines in the setup may fail some jobs. As of July 2023, this is only used by **group::tenant scale** to test out changes while building [Cells](../../architecture/blueprints/cells/index.md). + ## Locking writes on the tables that don't belong to the database schemas When the CI database is promoted and the two databases are fully split, diff --git a/doc/development/database/not_null_constraints.md b/doc/development/database/not_null_constraints.md index 77fa23bbb19..2b10e440799 100644 --- a/doc/development/database/not_null_constraints.md +++ b/doc/development/database/not_null_constraints.md @@ -202,7 +202,7 @@ end If you have to clean up a nullable column for a [high-traffic table](../migration_style_guide.md#high-traffic-tables) (for example, the `artifacts` in `ci_builds`), your background migration goes on for a while and -it needs an additional [batched background migration cleaning up](batched_background_migrations.md#cleaning-up) +it needs an additional [batched background migration cleaning up](batched_background_migrations.md#cleaning-up-a-batched-background-migration) in the release after adding the data migration. In that rare case you need 3 releases end-to-end: diff --git a/doc/development/database/pagination_guidelines.md b/doc/development/database/pagination_guidelines.md index 04a2a8cdf9c..d6550d0a515 100644 --- a/doc/development/database/pagination_guidelines.md +++ b/doc/development/database/pagination_guidelines.md @@ -214,7 +214,7 @@ Limit (cost=137878.89..137881.65 rows=20 width=1309) (actual time=5523.588..552 (8 rows) ``` -We can argue that a normal user does not visit these pages, however, API users could easily navigate to very high page numbers (scraping, collecting data). +We can argue that a typical user does not visit these pages, however, API users could easily navigate to very high page numbers (scraping, collecting data). ### Keyset pagination diff --git a/doc/development/database/post_deployment_migrations.md b/doc/development/database/post_deployment_migrations.md index 0d0047531f9..8ee9a4ae099 100644 --- a/doc/development/database/post_deployment_migrations.md +++ b/doc/development/database/post_deployment_migrations.md @@ -33,7 +33,7 @@ release managers through the Say you're using Chef for deploying new versions of GitLab and you'd like to run post deployment migrations after deploying a new version. Let's assume you -normally use the command `chef-client` to do so. To make use of this feature +usually use the command `chef-client` to do so. To make use of this feature you'd have to run this command as follows: ```shell @@ -63,7 +63,7 @@ behave exactly like regular Rails migrations. Post deployment migrations can be used to perform migrations that mutate state that an existing version of GitLab depends on. For example, say you want to remove a column from a table. This requires downtime as a GitLab instance -depends on this column being present while it's running. Normally you'd follow +depends on this column being present while it's running. Usually you'd follow these steps in such a case: 1. Stop the GitLab instance diff --git a/doc/development/database/required_stops.md b/doc/development/database/required_stops.md index e4f66f4424f..361a0495545 100644 --- a/doc/development/database/required_stops.md +++ b/doc/development/database/required_stops.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Adding required stops -Required stops should only be added when it is deemed absolutely necessary, due to their +[Required stops](../../update/index.md#upgrade-paths) should only be added when it is deemed absolutely necessary, because of their disruptive effect on customers. Before adding a required stop, consider if any alternative approaches exist to avoid a required stop. Sometimes a required stop is unavoidable. In those cases, follow the instructions below. @@ -95,3 +95,9 @@ is identified after release, the following steps must still be completed: 1. Update `Gitlab::Database::MIN_SCHEMA_GITLAB_VERSION` in `lib/gitlab/database.rb` to the new required stop versions. Do not change `Gitlab::Database::MIN_SCHEMA_VERSION`. +1. In the `charts` project, update the + [upgrade check hook](https://gitlab.com/gitlab-org/charts/gitlab/-/blame/master/templates/_runcheck.tpl#L32) + to the required stop version. +1. In the `omnibus-gitlab` project, update the + [pre-install version check](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/templates/package-scripts/preinst.erb#L43) + to the required stop version. diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index 5cd325bfa56..54bdeaa7b28 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -308,7 +308,7 @@ end If you have to clean up a text column for a really [large table](https://gitlab.com/gitlab-org/gitlab/-/blob/master/rubocop/rubocop-migrations.yml#L3) (for example, the `artifacts` in `ci_builds`), your background migration goes on for a while and -it needs an additional [batched background migration cleaning up](batched_background_migrations.md#cleaning-up) +it needs an additional [batched background migration cleaning up](batched_background_migrations.md#cleaning-up-a-batched-background-migration) in the release after adding the data migration. In that rare case you need 3 releases end-to-end: diff --git a/doc/development/database/table_partitioning.md b/doc/development/database/table_partitioning.md index 88b2ccbc6a2..8477dd180a6 100644 --- a/doc/development/database/table_partitioning.md +++ b/doc/development/database/table_partitioning.md @@ -256,7 +256,7 @@ class BackfillPartitionAuditEvents < Gitlab::Database::Migration[2.1] end ``` -This step [queues a batched background migration](batched_background_migrations.md#queueing) internally with BATCH_SIZE and SUB_BATCH_SIZE as `50,000` and `2,500`. Refer [Batched Background migrations guide](batched_background_migrations.md) for more details. +This step [queues a batched background migration](batched_background_migrations.md#enqueue-a-batched-background-migration) internally with BATCH_SIZE and SUB_BATCH_SIZE as `50,000` and `2,500`. Refer [Batched Background migrations guide](batched_background_migrations.md) for more details. ### Step 3: Post-backfill cleanup (Release N+1) diff --git a/doc/development/database_review.md b/doc/development/database_review.md index 76e9add215b..42021a5ae95 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -151,9 +151,7 @@ Include in the MR description: - Write the raw SQL in the MR description. Preferably formatted nicely with [pgFormatter](https://sqlformat.darold.net) or <https://paste.depesz.com> and using regular quotes -<!-- vale gitlab.NonStandardQuotes = NO --> - (for example, `"projects"."id"`) and avoiding smart quotes (for example, `“projects”.“id”`). -<!-- vale gitlab.NonStandardQuotes = YES --> + (for example, `"projects"."id"`) and avoiding smart quotes (for example, `"projects"."id"`). - In case of queries generated dynamically by using parameters, there should be one raw SQL query for each variation. For example, a finder for issues that may take as a parameter an optional filter on projects, @@ -244,7 +242,7 @@ Include in the MR description: - Manually trigger the [database testing](database/database_migration_pipeline.md) job (`db:gitlabcom-database-testing`) in the `test` stage. - If a single `update` is below than `1s` the query can be placed directly in a regular migration (inside `db/migrate`). - - Background migrations are normally used, but not limited to: + - Background migrations are usually used, but not limited to: - Migrating data in larger tables. - Making numerous SQL queries per record in a dataset. - Review queries (for example, make sure batch sizes are fine) diff --git a/doc/development/deprecation_guidelines/index.md b/doc/development/deprecation_guidelines/index.md index e81bed07da9..af89da5ec65 100644 --- a/doc/development/deprecation_guidelines/index.md +++ b/doc/development/deprecation_guidelines/index.md @@ -31,8 +31,9 @@ However, at GitLab, we [give agency](https://about.gitlab.com/handbook/values/#g ## When can a feature be removed/changed? -Generally, feature or configuration can be removed/changed only on major release. -It also should be [deprecated in advance](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations). +Features or configuration can only be removed/changed in a major release. + +They must be [deprecated in advance](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations). For API removals, see the [GraphQL](../../api/graphql/index.md#deprecation-and-removal-process) and [GitLab API](../documentation/restful_api_styleguide.md#deprecations) guidelines. @@ -40,54 +41,64 @@ For configuration removals, see the [Omnibus deprecation policy](../../administr For versioning and upgrade details, see our [Release and Maintenance policy](../../policy/maintenance.md). -## Update the deprecations and removals documentation pages +## Requesting a breaking change in a minor release -The [deprecations](../../update/deprecations.md) and [removals](../../update/removals.md) -documentation is generated from the YAML files located in -[`gitlab/data/`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/data). +GitLab self-managed packages are semantically versioned and follow our [maintenance policy](../../policy/maintenance.md). This process applies to features and APIs that are generally available, not beta or experimental. -To update the deprecations and removals pages when an entry is added, -edited, or removed: +This maintenance policy is in place to allow our customers to prepare for disruptive changes by establishing a clear and predictable pattern that is widely used in the software industry. For many of our customers, GitLab is a business-critical application and surprising changes can cause damages and erode trust. -1. From the command line, navigate to your local clone of the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) project. -1. Create, edit, or remove the YAML file under [deprecations](https://gitlab.com/gitlab-org/gitlab/-/tree/master/data/deprecations) - or [removals](https://gitlab.com/gitlab-org/gitlab/-/tree/master/data/removals). -1. Compile the deprecation or removals documentation with the appropriate command: +Introducing breaking changes in minor releases is against policy because it can disrupt our customers and introduces an element of randomness that requires customers to check for breaking changes every minor release to ensure that their business is not impacted. This does not align with our goal [to make it as easy as possible for customers to do business with GitLab](https://about.gitlab.com/company/yearlies/#fy24-yearlies) and is strongly discouraged. - - For deprecations: +Breaking changes are deployed to GitLab.com after they are merged into the codebase and do not respect the minor release cadence. Special care must be taken to inform the [Customer Support](https://about.gitlab.com/handbook/support/) and [Customer Success](https://about.gitlab.com/handbook/customer-success/) teams so that we can offer fast resolution to any customers that may be impacted by unexpected breaking changes. - ```shell - bin/rake gitlab:docs:compile_deprecations - ``` +Breaking our own policies, in particular shipping breaking changes in minor releases, is only reserved for situations in which GitLab establishes that delaying a breaking change would overall have a significantly more negative impact to customers than shipping it in a minor release. The most important lens for evaluating if an exception is granted is customer results. - - For removals: +Introducing a breaking change in a minor release requires a PM and EM to follow the process below to request an exception: - ```shell - bin/rake gitlab:docs:compile_removals - ``` +1. Open a new issue in the [product issue tracker using the Breaking Change Exception template](https://gitlab.com/gitlab-com/Product/-/issues/new?issuable_template=Breaking-Change-Exception) +1. Title should follow the format `Breaking change exception: Description` +1. Provide an impact assessment for the breaking change + 1. How many customers are impacted? + 1. Can we get the same outcome without a breaking-change? (i.e. no removal) + 1. Can the breaking-change wait till the next major release, or the next scheduled upgrade stop, for example [Database scenarios](../database/required_stops.md))? + 1. What is the alternative for customers to do the same job the change will break? + 1. How difficult is it for customers to migrate to the alternative? Is there a migration plan? +1. Provide a communication plan and establish a clear timeline, including the targeted minor release. +1. Notify Support and Customer Success so they can share information with relevant customers. +1. Obtain approval from the VP of Development, VP of Product Management, and VP of Customer Support for this area +1. Obtain approval from the CPO and CTO -1. If needed, you can verify the docs are up to date with: +## Update the deprecations and removals documentation - - For deprecations: +The [deprecations and removals](../../update/deprecations.md) +documentation is generated from the YAML files located in +[`gitlab/data/deprecations`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/data/deprecations). - ```shell - bin/rake gitlab:docs:check_deprecations - ``` +To update the deprecations and removals page when a YAML file is added, +edited, or removed: + +1. From the command line, navigate to your local clone of the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) project. +1. Create, edit, or remove the YAML file under [`data/deprecations`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/data/deprecations). +1. Compile the deprecations and removals documentation: - - For removals: + ```shell + bin/rake gitlab:docs:compile_deprecations + ``` + +1. If needed, you can verify the docs are up to date with: - ```shell - bin/rake gitlab:docs:check_removals - ``` + ```shell + bin/rake gitlab:docs:check_deprecations + ``` 1. Commit the updated documentation and push the changes. -1. Create a merge request using the [Deprecations](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/merge_request_templates/Deprecations.md) - or [Removals](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/merge_request_templates/Removals.md) templates. +1. Create a merge request using the [Deprecations and Removals](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/merge_request_templates/Deprecations.md) + template. Related Handbook pages: - <https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes> -- <https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-deprecations-and-removals-docs> +- <https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-deprecations-and-removals-page> ## Update the related documentation diff --git a/doc/development/development_processes.md b/doc/development/development_processes.md index c5d60d18419..4ce9df8d7da 100644 --- a/doc/development/development_processes.md +++ b/doc/development/development_processes.md @@ -22,6 +22,7 @@ Must-reads: Complementary reads: +- [Avoiding required stops](avoiding_required_stops.md) - [Contribute to GitLab](contributing/index.md) - [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) - [Patch release process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/patch/process.md#process-for-developers) diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index 823a71eb130..da6af8b95ef 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -40,7 +40,7 @@ better understand the end-to-end path of a request through the system. When a re process boundaries, the correlation ID is injected into the outgoing request. This enables the propagation of the correlation ID to each downstream subsystem. -Correlation IDs are normally generated in the Rails application in response to +Correlation IDs are usually generated in the Rails application in response to certain web requests. Some user facing systems don't generate correlation IDs in response to user requests (for example, Git pushes over SSH). @@ -139,8 +139,8 @@ This can be shown by typing `p` `b` in the browser window. Once the performance bar is enabled, select **Trace** in the performance bar to go to the Jaeger UI. -The Jaeger search UI returns a query for the `Correlation-ID` of the current request. Normally, -this search should return a single trace result. Selecting this result shows the detail of the +The Jaeger search UI returns a query for the `Correlation-ID` of the current request. +This search should return a single trace result. Selecting this result shows the detail of the trace in a hierarchical time-line.  diff --git a/doc/development/documentation/alpha_beta.md b/doc/development/documentation/alpha_beta.md index a6f756f93ae..61f07e79e12 100644 --- a/doc/development/documentation/alpha_beta.md +++ b/doc/development/documentation/alpha_beta.md @@ -29,7 +29,7 @@ For example: FLAG: On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to enable the feature flag named `example_flag`. +To make it available, an administrator can enable the feature flag named `example_flag`. On GitLab.com, this feature is not available. This feature is not ready for production use. Use this great new feature when you need to do this new thing. diff --git a/doc/development/documentation/contribute.md b/doc/development/documentation/contribute.md index d3b7d9a4d93..b6573665f8d 100644 --- a/doc/development/documentation/contribute.md +++ b/doc/development/documentation/contribute.md @@ -54,7 +54,7 @@ When you are ready to update the documentation: 1. In the **Commit message** text box, enter a commit message. Use 3-5 words, start with a capital letter, and do not end with a period. 1. Select **Commit changes**. - 1. On the left sidebar, select **Merge requests**. + 1. On the left sidebar, select **Code > Merge requests**. 1. Select **New merge request**. 1. For the source branch, select your fork and branch. If you did not create a branch, select `master`. For the target branch, select the [GitLab repository](https://gitlab.com/gitlab-org/gitlab) `master` branch. diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md index 5cee300481b..a08052bf0e4 100644 --- a/doc/development/documentation/feature_flags.md +++ b/doc/development/documentation/feature_flags.md @@ -72,7 +72,7 @@ FLAG: A `FLAG` note renders on the GitLab documentation site as: FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `example_flag`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `example_flag`. On GitLab.com, this feature is not available. This feature is not ready for production use. @@ -80,15 +80,15 @@ This feature is not ready for production use. | If the feature is... | Use this text | |--------------------------|---------------| -| Available | ``On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Unavailable | ``On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Available to some users | ``On self-managed GitLab, by default this feature is available to a subset of users. To show or hide the feature for all, ask an administrator to [change the status of the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Available, per-group | ``On self-managed GitLab, by default this feature is available. To hide the feature per group, ask an administrator to [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Unavailable, per-group | ``On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Available, per-project | ``On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Unavailable, per-project | ``On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Available, per-user | ``On self-managed GitLab, by default this feature is available. To hide the feature per user, ask an administrator to [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | -| Unavailable, per-user | ``On self-managed GitLab, by default this feature is not available. To make it available per user, ask an administrator to [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Available | ``On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Unavailable | ``On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Available to some users | ``On self-managed GitLab, by default this feature is available to a subset of users. To show or hide the feature for all, an administrator can [change the status of the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Available, per-group | ``On self-managed GitLab, by default this feature is available. To hide the feature per group, an administrator can [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Unavailable, per-group | ``On self-managed GitLab, by default this feature is not available. To make it available per group, an administrator can [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Available, per-project | ``On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, an administrator can [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Unavailable, per-project | ``On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Available, per-user | ``On self-managed GitLab, by default this feature is available. To hide the feature per user, an administrator can [disable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | +| Unavailable, per-user | ``On self-managed GitLab, by default this feature is not available. To make it available per user, an administrator can [enable the feature flag](<path to>/administration/feature_flags.md) named `flag_name`.`` | ### GitLab.com availability information @@ -113,7 +113,7 @@ The following examples show the progression of a feature flag. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `forti_token_cloud`. +an administrator can [enable the feature flag](../administration/feature_flags.md) named `forti_token_cloud`. The feature is not ready for production use. ``` @@ -125,7 +125,7 @@ When the feature is enabled in production, you can update the version history: FLAG: On self-managed GitLab, by default this feature is available. To hide the feature per user, -ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `forti_token_cloud`. +an administrator can [disable the feature flag](../administration/feature_flags.md) named `forti_token_cloud`. ``` And, when the feature is done and fully available to all users: diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 761dde839c1..3f26a5a7f4e 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -223,7 +223,7 @@ merge it as early as possible. ### Linking to `/help` When you're building a new feature, you may need to link to the documentation -from the GitLab application. This is normally done in files inside the +from the GitLab application. This is usually done in files inside the `app/views/` directory, with the help of the `help_page_path` helper method. The `help_page_path` contains the path to the document you want to link to, diff --git a/doc/development/documentation/restful_api_styleguide.md b/doc/development/documentation/restful_api_styleguide.md index c39c7c02108..a5d565ffa79 100644 --- a/doc/development/documentation/restful_api_styleguide.md +++ b/doc/development/documentation/restful_api_styleguide.md @@ -63,7 +63,7 @@ METHOD /endpoint Supported attributes: | Attribute | Type | Required | Description | -|:-------------------------|:---------|:---------|:----------------------| +|--------------------------|----------|----------|-----------------------| | `attribute` | datatype | Yes | Detailed description. | | `attribute` **(<tier>)** | datatype | No | Detailed description. | | `attribute` | datatype | No | Detailed description. | @@ -73,7 +73,7 @@ If successful, returns [`<status_code>`](rest/index.md#status-codes) and the fol response attributes: | Attribute | Type | Description | -|:-------------------------|:---------|:----------------------| +|--------------------------|----------|-----------------------| | `attribute` | datatype | Detailed description. | | `attribute` **(<tier>)** | datatype | Detailed description. | @@ -127,8 +127,8 @@ To deprecate an attribute: 1. Add inline deprecation text to the description. ```markdown - | Attribute | Type | Required | Description | - |:--------------|:-------|:---------|:----------------------------------------------------------------------------------------------------------------------------------------------------| + | Attribute | Type | Required | Description | + |---------------|--------|----------|-------------| | `widget_name` | string | No | [Deprecated](<link-to-issue>) in GitLab 14.7 and is planned for removal in 15.4. Use `widget_id` instead. The name of the widget. | ``` @@ -144,7 +144,7 @@ Sort the table by required attributes first, then alphabetically. ```markdown | Attribute | Type | Required | Description | -|:-----------------------------|:--------------|:---------|:----------------------------------------------------| +|------------------------------|---------------|----------|-----------------------------------------------------| | `title` | string | Yes | Title of the issue. | | `assignee_ids` **(PREMIUM)** | integer array | No | IDs of the users to assign the issue to. | | `confidential` | boolean | No | Sets the issue to confidential. Default is `false`. | @@ -153,7 +153,7 @@ Sort the table by required attributes first, then alphabetically. Rendered example: | Attribute | Type | Required | Description | -|:-----------------------------|:--------------|:---------|:----------------------------------------------------| +|------------------------------|---------------|----------|-----------------------------------------------------| | `title` | string | Yes | Title of the issue. | | `assignee_ids` **(PREMIUM)** | integer array | No | IDs of the users to assign the issue to. | | `confidential` | boolean | No | Sets the issue to confidential. Default is `false`. | @@ -180,7 +180,7 @@ Sort the table alphabetically. ```markdown | Attribute | Type | Description | -|:-----------------------------|:--------------|:------------------------------------------| +|------------------------------|---------------|-------------------------------------------| | `assignee_ids` **(PREMIUM)** | integer array | IDs of the users to assign the issue to. | | `confidential` | boolean | Whether the issue is confidential or not. | | `title` | string | Title of the issue. | @@ -189,7 +189,7 @@ Sort the table alphabetically. Rendered example: | Attribute | Type | Description | -|:-----------------------------|:--------------|:------------------------------------------| +|------------------------------|---------------|-------------------------------------------| | `assignee_ids` **(PREMIUM)** | integer array | IDs of the users to assign the issue to. | | `confidential` | boolean | Whether the issue is confidential or not. | | `title` | string | Title of the issue. | @@ -211,7 +211,7 @@ For information about writing attribute descriptions, see the [GraphQL API descr commands apart into multiple lines. | Methods | Description | -|:------------------------------------------------|:-------------------------------------------------------| +|-------------------------------------------------|--------------------------------------------------------| | `--header "PRIVATE-TOKEN: <your_access_token>"` | Use this method as is, whenever authentication needed. | | `--request POST` | Use this method when creating new objects. | | `--request PUT` | Use this method when updating existing objects. | diff --git a/doc/development/documentation/site_architecture/folder_structure.md b/doc/development/documentation/site_architecture/folder_structure.md index 8a1d9b9fee3..1c9fc1441c4 100644 --- a/doc/development/documentation/site_architecture/folder_structure.md +++ b/doc/development/documentation/site_architecture/folder_structure.md @@ -17,7 +17,7 @@ API. Our goal is to have a clear hierarchical structure with meaningful URLs like `docs.gitlab.com/user/project/merge_requests/`. With this pattern, you can immediately tell that you are navigating to user-related documentation about -Project features; specifically about Merge Requests. Our site's paths match +project features; specifically about merge requests. Our site's paths match those of our repository, so the clear structure also makes documentation easier to update. @@ -26,13 +26,20 @@ Put files for a specific product area into the related folder: | Directory | Contents | |:----------------------|:------------------| | `doc/user/` | Documentation for users. Anything that can be done in the GitLab user interface goes here, including usage of the `/admin` interface. | -| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. Administrator settings in the GitLab user interface are under `doc/user/admin_area/`. | +| `doc/administration/` | Documentation that requires the user to have access to the server where GitLab is installed. Administrator settings in the GitLab user interface are under `doc/administration/`. | | `doc/api/` | Documentation for the API. | | `doc/development/` | Documentation related to the development of GitLab, whether contributing code or documentation. Related process and style guides should go here. | | `doc/legal/` | Legal documents about contributing to GitLab. | | `doc/install/` | Instructions for installing GitLab. | | `doc/update/` | Instructions for updating GitLab. | -| `doc/topics/` | Indexes per topic (`doc/topics/topic_name/index.md`): all resources for that topic. | +| `doc/tutorials/` | Tutorials for how to use GitLab. | + +The following are legacy or deprecated folders. +Do not add new content to these folders: + +- `/gitlab-basics/` +- `/topics/` +- `/university/` ## Work with directories and files @@ -45,33 +52,18 @@ When working with directories and files: 1. When creating or renaming a file or directory and it has more than one word in its name, use underscores (`_`) instead of spaces or dashes. For example, proper naming would be `import_project/import_from_github.md`. This applies - to both image files and Markdown files. -1. For image files, do not exceed 100KB. + to both [image files](../styleguide/index.md#images) and Markdown files. 1. Do not upload video files to the product repositories. [Link or embed videos](../styleguide/index.md#videos) instead. -1. There are four main directories: `user`, `administration`, `api`, and - `development`. -1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, - `profile/`, `dashboard/` and `admin_area/`. +1. In the `doc/user/` directory: - `doc/user/project/` should contain all project related documentation. - `doc/user/group/` should contain all group related documentation. - `doc/user/profile/` should contain all profile related documentation. Every page you would navigate under `/profile` should have its own document, for example, `account.md`, `applications.md`, or `emails.md`. - - `doc/user/dashboard/` should contain all dashboard related documentation. - - `doc/user/admin_area/` should contain all administrator-related - documentation describing what can be achieved by accessing the GitLab - administrator interface (not to be confused with `doc/administration` where - server access is required). - - Every category under `/admin/application_settings/` should have its - own document located at `doc/user/admin_area/settings/`. For example, - the **Visibility and Access Controls** category should have a document - located at `doc/user/admin_area/settings/visibility_and_access_controls.md`. -1. The `doc/topics/` directory holds topic-related technical content. Create - `doc/topics/topic_name/subtopic_name/index.md` when subtopics become necessary. - General user and administrator documentation should be placed accordingly. -1. The `/university/` directory is *deprecated* and the majority of its documentation - has been moved. +1. In the `doc/administration/` directory: all administrator-related + documentation for administrators, including admin tasks done in both + the UI and on the backend servers. If you're unsure where to place a document or a content addition, this shouldn't stop you from authoring and contributing. Use your best judgment, and then ask diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 6a56d824e35..bd4ded8ca11 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -133,7 +133,7 @@ the team is happy to review and improve upon your content. Review the [Documentation guidelines](index.md) before you begin your first documentation MR. Maintaining a knowledge base separate from the documentation would -be against the documentation-first methodology, because the content would overlap with +be against the documentation-first methodology because the content would overlap with the documentation. ## Writing for localization @@ -190,6 +190,8 @@ Use backticks for: - [Code blocks](#code-blocks). - Error messages. +- Commands, parameters, and filenames. +- Values. For example: "In the **Name** text box, type `test`." ### Markdown Rules @@ -379,7 +381,7 @@ If you use an acronym, spell it out on first use on a page. You do not need to s ### Numbers -When using numbers in text, spell out zero through nine, and use numbers for 10 and greater. For details, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/numbers). +For numbers in text, spell out zero through nine and use numbers for 10 and greater. For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/numbers). ## Text @@ -698,6 +700,9 @@ page), use these phrases: | No | `**{dotted-circle}** No` | **{dotted-circle}** No | | Yes | `**{check-circle}** Yes` | **{check-circle}** Yes | +Don't use `**{dotted-circle}**` and `**{check-circle}**` in API documentation. +Instead, follow the [API topic template](../restful_api_styleguide.md#api-topic-template). + ### Footnotes To indicate a footnote, use the HTML tag `<sup>` with a number. @@ -821,7 +826,7 @@ For example: You can expand on this text by using phrases like `For more information about this feature, see...` -Do not to use the following constructions: +Do not use the following constructions: - `Learn more about...` - `To read more...`. @@ -878,7 +883,7 @@ If you must use one of these links: - If the link is to a confidential issue, mention that the issue is visible only to GitLab team members, as in the first example. - If the link requires a specific role or permissions, mention that information, as in the second example. -- Put the link in backticks, so that it does not cause link checkers to fail. +- Put the link in backticks so that it does not cause link checkers to fail. Examples: @@ -909,7 +914,7 @@ document to ensure it links to the most recent version of the file. ## Navigation -When documenting how to navigate through the GitLab UI: +When documenting how to navigate the GitLab UI: - Always use location, then action. - From the **Visibility** dropdown list (location), select **Public** (action). @@ -968,13 +973,13 @@ To open either project or group settings: To create a project: ```markdown -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. ``` To create a group: ```markdown -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. ``` To open the Admin Area: @@ -1196,7 +1201,7 @@ include a visual representation to help readers understand it, you can: an area of the screen. - Create a short video of the interaction and link to it. -## Emojis +## Emoji Don't use the Markdown emoji format, for example `:smile:`, for any purpose. Use [GitLab SVG icons](#gitlab-svg-icons) instead. @@ -1546,11 +1551,20 @@ When names change, it is more complicated to search or grep text that has line b Tier badges are displayed as orange text next to a topic title. These badges link to the GitLab pricing page. -You must assign a tier badge: +You should assign a tier badge: - To all H1 topic titles, except the pages under `doc/development/*`. - To topic titles that don't apply to the same tier as the H1. +The H1 tier badge should be the badge that applies to the lowest tier for the features on the page. + +Some pages won't have a tier badge, because no obvious tier badge applies. For example: + +- Tutorials. +- Pages that compare features from different tiers. + +#### Add a tier badge + To add a tier badge to a topic title, add the relevant tier badge after the title text. For example: diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index 0bbd679abe5..84ab2ecc4e9 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -152,7 +152,7 @@ Instead of: - This feature enables users to add files to their repository. This phrasing is more active and is from the user perspective, rather than the person who implemented the feature. -[View details in the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/a/allow-allows). +For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/a/allow-allows). ## analytics @@ -169,8 +169,8 @@ Instead of **and/or**, use **or** or rewrite the sentence to spell out both opti ## and so on -Do not use **and so on**. Instead, be more specific. For details, see -[the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/a/and-so-on). +Do not use **and so on**. Instead, be more specific. For more information, see the +[Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/a/and-so-on). ## area @@ -284,7 +284,7 @@ CI/CD is always uppercase. No need to spell it out on first use. ## CI/CD minutes -Do not use **CI/CD minutes**. This term was renamed to [**units of compute**](#units-of-compute). +Do not use **CI/CD minutes**. This term was renamed to [**compute minutes**](#compute-minutes). ## click @@ -318,16 +318,30 @@ Use **compute** for the resources used by runners to run CI/CD jobs. Related terms: -- [**units of compute**](#units-of-compute): How compute usage is calculated. For example, `400 units of compute`. -- [**compute quota**](../../../ci/pipelines/cicd_minutes.md): The limit of units of compute that a namespace can use each month. -- **compute usage**: The number of units of compute that the namespace has used from the monthly quota. +- [**compute minutes**](#compute-minutes): How compute usage is calculated. For example, `400 compute minutes`. +- [**compute quota**](../../../ci/pipelines/cicd_minutes.md): The limit of compute minutes that a namespace can use each month. +- **compute usage**: The number of compute minutes that the namespace has used from the monthly quota. + +## compute minutes + +Use **compute minutes** instead of these (or similar) terms: + +- **CI/CD minutes** +- **CI minutes** +- **pipeline minutes** +- **CI pipeline minutes** +- **pipeline minutes** + +For more information, see [epic 2150](https://gitlab.com/groups/gitlab-com/-/epics/2150). ## confirmation dialog -Use **confirmation dialog** to describe the dialog box that asks you to confirm your action. For example: +Use **confirmation dialog** to describe the dialog that asks you to confirm an action. For example: - On the confirmation dialog, select **OK**. +Do not use **confirmation box** or **confirmation dialog box**. See also [**dialog**](#dialog). + ## Container Registry Use title case for the GitLab Container Registry. @@ -387,9 +401,29 @@ When writing about the Developer role: Do not use **Developer permissions**. A user who is assigned the Developer role has a set of associated permissions. +## dialog + +Use **dialog** rather than any of these alternatives: + +- **dialog box** +- **modal** +- **modal dialog** +- **modal window** +- **pop-up** +- **pop-up window** +- **window** + +See also [**confirmation dialog**](#confirmation-dialog). For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/d/dialog-box-dialog-dialogue). + +When the dialog is the location of an action, use **on** as a preposition. For example: + +- On the **Grant permission** dialog, select **Group**. + +See also [**on**](#on). + ## disable -See [the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/d/disable-disabled) for guidance on **disable**. +See the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/d/disable-disabled) for guidance on **disable**. Use **inactive** or **off** instead. ([Vale](../testing.md#vale) rule: [`InclusionAbleism.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionAbleism.yml)) ## disallow @@ -439,17 +473,30 @@ Do not use **easily**. If the user doesn't find the process to be easy, we lose Do not use Latin abbreviations. Use **for example**, **such as**, **for instance**, or **like** instead. ([Vale](../testing.md#vale) rule: [`LatinTerms.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/LatinTerms.yml)) +## ellipsis + +When documenting UI text, if the UI includes an ellipsis, do not include the ellipsis in the documentation. +For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/punctuation/ellipses). + +Use: + +- **Create new** + +Instead of: + +- **Create new...** + ## email Do not use **e-mail** with a hyphen. When plural, use **emails** or **email messages**. ([Vale](../testing.md#vale) rule: [`SubstitutionSuggestions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionSuggestions.yml)) -## emojis +## emoji -Use **emojis** to refer to the plural form of **emoji**. +Use **emoji** to refer to the plural form of **emoji**. ## enable -See [the Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/e/enable-enables) for guidance on **enable**. +See the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/e/enable-enables) for guidance on **enable**. Use **active** or **on** instead. ([Vale](../testing.md#vale) rule: [`InclusionAbleism.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionAbleism.yml)) ## enter @@ -680,7 +727,7 @@ Instead of: ## I -Do not use first-person singular. Use **you**, **we**, or **us** instead. ([Vale](../testing.md#vale) rule: [`FirstPerson.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FirstPerson.yml)) +Do not use first-person singular. Use **you** or rewrite the phrase instead. ## i.e. @@ -862,15 +909,27 @@ Do not use `master`. Use `main` when you need a sample [default branch name](#de ## may, might -**Might** means something has the probability of occurring. **May** gives permission to do something. Consider **can** instead of **may**. +**Might** means something has the probability of occurring. Might is often used in troubleshooting documentation. -## MB, megabytes +**May** gives permission to do something. Consider **can** instead of **may**. -For **MB** and **GB**, follow the [Microsoft guidance](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms). +Consider rewording phrases that use these terms. These terms often indicate possibility and doubt, and technical writing strives to be precise. -## me, myself, mine +See also [you can](#you-can). + +Use: + +- The `committed_date` and `authored_date` fields are generated from different sources, and might not be identical. +- A typical pipeline consists of four stages, executed in the following order: + +Instead of: -Do not use first-person singular. Use **you**, **we**, or **us** instead. ([Vale](../testing.md#vale) rule: [`FirstPerson.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/FirstPerson.yml)) +- The `committed_date` and `authored_date` fields are generated from different sources, and may not be identical. +- A typical pipeline might consist of four stages, executed in the following order: + +## MB, megabytes + +For **MB** and **GB**, follow the [Microsoft guidance](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms). ## member @@ -1009,15 +1068,12 @@ For more information, see the ## on -When documenting how to select high-level UI elements, use the word **on**. +When documenting high-level UI elements, use **on** as a preposition. For example: -Use: +- On the left sidebar, select **Settings > CI/CD**. +- On the **Grant permission** dialog, select **Group**. -- `On the left sidebar...` - -Instead of: - -- Do not: `From the left sidebar...` or `In the left sidebar...` +Do not use **from** or **in**. For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/f/from-vs-on). ## once @@ -1098,7 +1154,7 @@ Use lowercase for **personal access token**. ## please -Do not use **please**. For details, see the [Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/p/please). +Do not use **please**. For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/p/please). ## Premium @@ -1453,9 +1509,9 @@ Use [**2FA** and **two-factor authentication**](#2fa-two-factor-authentication) ## type -Use **type** when the cursor remains in the field you're typing in. For example, -in a search dialog, you begin typing and the field populates results. You do not -click out of the field. +Use **type** when the cursor remains where you're typing. For example, +in a search box, you begin typing and search results appear. You do not +click out of the search box. For example: @@ -1470,25 +1526,13 @@ See also [**enter**](#enter). Use **Ultimate**, in uppercase, for the subscription tier. When you refer to **Ultimate** in the context of other subscription tiers, follow [the subscription tier](#subscription-tier) guidance. -## units of compute - -Use **units of compute** instead of these (or similar) terms: - -- **CI/CD minutes** -- **CI minutes** -- **pipeline minutes** -- **CI pipeline minutes** -- **pipeline minutes** - -For more information, see [issue 5218](https://gitlab.com/gitlab-com/Product/-/issues/5218). - ## units of measurement Use a space between the number and the unit of measurement. For example, **128 GB**. ([Vale](../testing.md#vale) rule: [`Units.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Units.yml)) -For other guidance, follow -[the Microsoft style guidelines](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms). +For more information, see the +[Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/term-collections/bits-bytes-terms). ## update @@ -1524,7 +1568,7 @@ If the UI element is not in a corner, use **upper left** and **upper right**. Do not use **top left** and **top right**. -For details, see the [Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/u/upper-left-upper-right). +For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/u/upper-left-upper-right). ## useful @@ -1571,7 +1615,7 @@ Instead of: - While job 1 can run quickly, job 2 is more precise. -For details, see the [Microsoft style guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/w/while). +For more information, see the [Microsoft Style Guide](https://learn.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/w/while). ## whilst @@ -1616,6 +1660,7 @@ For example: - Use code review analytics to view merge request data. - Create a board to organize your team tasks. - Configure variables to restrict pushes to a repository. +- Add links to external accounts you have, like Skype and Twitter. Use **you can** for optional actions. For example: diff --git a/doc/development/documentation/workflow.md b/doc/development/documentation/workflow.md index 1a4194aebd9..eb1ea28d3b8 100644 --- a/doc/development/documentation/workflow.md +++ b/doc/development/documentation/workflow.md @@ -69,7 +69,7 @@ Remember: GitLab documentation is distributed under the [CC BY-SA 4.0 license](https://creativecommons.org/licenses/by-sa/4.0/), which presupposes that GitLab owns the documentation. -Under current law in the US and the EU, it’s possible that AI-generated works might either: +Under current law in the US and the EU, it's possible that AI-generated works might either: - not be owned by anyone because they weren't created by a human, or - belong to the AI training data's creator, if the AI verbatim reproduces content that it trained on diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index aeb9739ecb3..9219fcd6710 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -222,7 +222,7 @@ This also applies to views. #### Testing EE-only backend features -To test an EE class that doesn't exist in CE, create the spec file as you normally +To test an EE class that doesn't exist in CE, create the spec file as you usually would in the `ee/spec` directory, but without the second `ee/` subdirectory. For example, a class `ee/app/models/vulnerability.rb` would have its tests in `ee/spec/models/vulnerability_spec.rb`. @@ -303,7 +303,7 @@ This is also not just applied to models. Here's a list of other examples: #### Testing EE features based on CE features To test an `EE` namespaced module that extends a CE class with EE features, -create the spec file as you normally would in the `ee/spec` directory, including the second `ee/` subdirectory. +create the spec file as you usually would in the `ee/spec` directory, including the second `ee/` subdirectory. For example, an extension `ee/app/models/ee/user.rb` would have its tests in `ee/spec/models/ee/user_spec.rb`. In the `RSpec.describe` call, use the CE class name where the EE module would be used. @@ -713,7 +713,7 @@ end ### Code in `lib/` Place EE-specific logic in the top-level `EE` module namespace. Namespace the -class beneath the `EE` module just as you would normally. +class beneath the `EE` module as you usually would. For example, if CE has LDAP classes in `lib/gitlab/ldap/` then you would place EE-specific LDAP classes in `ee/lib/ee/gitlab/ldap`. @@ -870,7 +870,7 @@ end #### EE-specific behavior -Sometimes we need EE-specific behavior in some of the APIs. Normally we could +Sometimes we need EE-specific behavior in some of the APIs. Usually we could use EE methods to override CE methods, however API routes are not methods and therefore cannot be overridden. We need to extract them into a standalone method, or introduce some "hooks" where we could inject behavior in the CE diff --git a/doc/development/event_store.md b/doc/development/event_store.md index 10dc0b1a7a9..c54e6ae2d07 100644 --- a/doc/development/event_store.md +++ b/doc/development/event_store.md @@ -365,3 +365,19 @@ RSpec.describe MergeRequests::UpdateHeadPipelineWorker do end end ``` + +## Best practices + +- Maintain [CE & EE separation and compatibility](ee_features.md#separation-of-ee-code-in-the-backend): + - Define the event class and publish the event in the same code where the event always occurs (CE or EE). + - If the event occurs as a result of a CE feature, the event class must both be defined and published in CE. + Likewise if the event occurs as a result of an EE feature, the event class must both be defined and published in EE. + - Define subscribers that depends on the event in the same code where the dependent feature exists (CE or EE). + - You can have an event published in CE (for example `Projects::ProjectCreatedEvent`) and a subscriber that depends + on this event defined in EE (for example `Security::SyncSecurityPolicyWorker`). +- Define the event class and publish the event within the same bounded context (top-level Ruby namespace). + - A given bounded context should only publish events related to its own context. +- Evaluate signal/noise ratio when subscribing to an event. How many events do you process vs ignore + within the subscriber? Consider using [conditional dispatch](#conditional-dispatch-of-events) + if you are interested in only a small subset of events. Balance between executing synchronous checks with + conditional dispatch or schedule potentially redundant workers. diff --git a/doc/development/experiment_guide/implementing_experiments.md b/doc/development/experiment_guide/implementing_experiments.md index 5bce9f1fab5..6fe58a1da54 100644 --- a/doc/development/experiment_guide/implementing_experiments.md +++ b/doc/development/experiment_guide/implementing_experiments.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w [Examples](https://gitlab.com/gitlab-org/growth/growth/-/wikis/GLEX-Framework-code-examples) Start by generating a feature flag using the `bin/feature-flag` command as you -normally would for a development feature flag, making sure to use `experiment` for +usually would for a development feature flag, making sure to use `experiment` for the type. For the sake of documentation let's name our feature flag (and experiment) `pill_color`. @@ -280,7 +280,7 @@ about contexts now. We can assume we run the experiment in one or a few places, but track events potentially in many places. The tracking call remains the same, with -the arguments you would normally use when +the arguments you would usually use when [tracking events using snowplow](../snowplow/index.md). The easiest example of tracking an event in Ruby would be: diff --git a/doc/development/fe_guide/content_editor.md b/doc/development/fe_guide/content_editor.md index 25140a067ca..fdeabc99ea4 100644 --- a/doc/development/fe_guide/content_editor.md +++ b/doc/development/fe_guide/content_editor.md @@ -4,27 +4,27 @@ group: Knowledge info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Content Editor development guidelines +# Rich text editor development guidelines -The Content Editor is a UI component that provides a WYSIWYG editing +The rich text editor is a UI component that provides a WYSIWYG editing experience for [GitLab Flavored Markdown](../../user/markdown.md) in the GitLab application. It also serves as the foundation for implementing Markdown-focused editors that target other engines, like static site generators. We use [Tiptap 2.0](https://tiptap.dev/) and [ProseMirror](https://prosemirror.net/) -to build the Content Editor. These frameworks provide a level of abstraction on top of +to build the rich text editor. These frameworks provide a level of abstraction on top of the native [`contenteditable`](https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/Editable_content) web technology. ## Usage guide -Follow these instructions to include the Content Editor in a feature. +Follow these instructions to include the rich text editor in a feature. -1. [Include the Content Editor component](#include-the-content-editor-component). +1. [Include the rich text editor component](#include-the-rich-text-editor-component). 1. [Set and get Markdown](#set-and-get-markdown). 1. [Listen for changes](#listen-for-changes). -### Include the Content Editor component +### Include the rich text editor component Import the `ContentEditor` Vue component. We recommend using asynchronous named imports to take advantage of caching, as the ContentEditor is a big dependency. @@ -43,7 +43,7 @@ export default { </script> ``` -The Content Editor requires two properties: +The rich text editor requires two properties: - `renderMarkdown` is an asynchronous function that returns the response (String) of invoking the [Markdown API](../../api/markdown.md). @@ -95,7 +95,7 @@ export default { ### Listen for changes -You can still react to changes in the Content Editor. Reacting to changes helps +You can still react to changes in the rich text editor. Reacting to changes helps you know if the document is empty or dirty. Use the `@change` event handler for this purpose. @@ -131,7 +131,7 @@ export default { ## Implementation guide -The Content Editor is composed of three main layers: +The rich text editor is composed of three main layers: - **The editing tools UI**, like the toolbar and the table structure editor. They display the editor's state and mutate it by dispatching commands. @@ -163,7 +163,7 @@ We implement [node views](https://tiptap.dev/guide/node-views/vue/#node-views-wi to provide inline editing tools for some content types, like tables and images. Node views allow separating the presentation of a content type from its [model](https://prosemirror.net/docs/guide/#doc.data_structures). Using a Vue component in -the presentation layer enables sophisticated editing experiences in the Content Editor. +the presentation layer enables sophisticated editing experiences in the rich text editor. Node views are located in `~/content_editor/components/wrappers`. #### Dispatch commands @@ -248,19 +248,19 @@ export default { The Tiptap [Editor](https://tiptap.dev/api/editor) class manages the editor's state and encapsulates all the business logic that powers -the Content Editor. The Content Editor constructs a new instance of this class and +the rich text editor. The rich text editor constructs a new instance of this class and provides all the necessary extensions to support [GitLab Flavored Markdown](../../user/markdown.md). #### Implement new extensions -Extensions are the building blocks of the Content Editor. You can learn how to implement +Extensions are the building blocks of the rich text editor. You can learn how to implement new ones by reading [the Tiptap guide](https://tiptap.dev/guide/custom-extensions). We recommend checking the list of built-in [nodes](https://tiptap.dev/api/nodes) and [marks](https://tiptap.dev/api/marks) before implementing a new extension from scratch. -Store the Content Editor extensions in the `~/content_editor/extensions` directory. +Store the rich text editor extensions in the `~/content_editor/extensions` directory. When using a Tiptap built-in extension, wrap it in a ES6 module inside this directory: ```javascript @@ -313,11 +313,11 @@ by first rendering the Markdown as HTML using the [Markdown API endpoint](../../ ```mermaid sequenceDiagram - participant A as Content Editor - participant E as Tiptap Object - participant B as Markdown Serializer + participant A as rich text editor + participant E as Tiptap object + participant B as Markdown serializer participant C as Markdown API - participant D as ProseMirror Parser + participant D as ProseMirror parser A->>B: deserialize(markdown) B->>C: render(markdown) C-->>B: html @@ -343,13 +343,13 @@ classes documentation before implementing a serializer: ```mermaid sequenceDiagram - participant A as Content Editor - participant B as Markdown Serializer + participant A as rich text editor + participant B as Markdown serializer participant C as ProseMirror Markdown A->>B: serialize(document) B->>C: serialize(document, serializers) - C-->>A: markdown string + C-->>A: Markdown string ``` `prosemirror-markdown` requires implementing a serializer function for each content type supported -by the Content Editor. We implement serializers in `~/content_editor/services/markdown_serializer.js`. +by the rich text editor. We implement serializers in `~/content_editor/services/markdown_serializer.js`. diff --git a/doc/development/fe_guide/design_tokens.md b/doc/development/fe_guide/design_tokens.md new file mode 100644 index 00000000000..9a1cc48c68f --- /dev/null +++ b/doc/development/fe_guide/design_tokens.md @@ -0,0 +1,320 @@ +--- +type: reference, dev +stage: none +group: Development +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Design tokens + +Design tokens provide a single source-of-truth for values (such as color, spacing, and duration), in different modes (such as default and dark), for different tools (such as Figma and code). + +## Usage + +We manage design tokens in the [`gitlab-ui`](https://gitlab.com/gitlab-org/gitlab-ui) repository. This repository is published on [npm](https://www.npmjs.com/package/@gitlab/ui), and is managed as a dependency with yarn. To upgrade to a new version run `yarn upgrade @gitlab/ui`. + +Design tokens are provided in different modes (default/dark) and file formats for use in CSS (custom properties), JavaScript (ES6 Constants/JSON), and SCSS (variables), for example: + +**JavaScript** + +```javascript +import { BLUE_500 } from '@gitlab/ui/dist/tokens/js/tokens'; + +const color = BLUE_500; // #1f75cb +``` + +**CSS** + +```css +@import '@gitlab/ui/dist/tokens/css/tokens'; + +.elem { + color: var(--blue-500); /* #1f75cb */ +} +``` + +**SCSS** + +```scss +@import '@gitlab/ui/dist/tokens/scss/tokens'; + +.elem { + color: $blue-500; /* #1f75cb */ +} +``` + +### Dark mode + +Where color design tokens are updated for dark mode, their values are provided with the same name in files appended with `.dark`, for example: + +**JavaScript** + +```javascript +import { BLUE_500 } from '@gitlab/ui/dist/tokens/js/tokens.dark'; + +const color = BLUE_500; // #428fdc +``` + +**CSS** + +```css +@import '@gitlab/ui/dist/tokens/css/tokens.dark'; + +.elem { + color: var(--blue-500); /* #428fdc */ +} +``` + +**SCSS** + +```scss +@import '@gitlab/ui/dist/tokens/scss/tokens.dark'; + +.elem { + color: $blue-500; /* #428fdc */ +} +``` + +## Creating or updating design tokens + +### Format + +Our design tokens use the [Design Tokens Format Module](https://tr.designtokens.org/format/) for defining design tokens that integrate with different tools and are converted to required file formats. It is a [community group draft report](https://www.w3.org/standards/types#reports), published by the [Design Tokens Community Group](https://www.w3.org/community/design-tokens/). + +The Design Tokens Format Module promotes a `*.token.json` extension standard for design token files, with a format that includes [a name and `$value`](https://tr.designtokens.org/format/#name-and-value) and an explicit [`$type`](https://tr.designtokens.org/format/#type-0): + +```json +// color.tokens.json +{ + "token name": { + "$value": "#000", + "$type": "color" + } +} +``` + +### Transformations + +Our design tokens use [style-dictionary](https://amzn.github.io/style-dictionary) to convert design tokens into consumable file formats (CSS/SCSS/JavaScript/JSON). + +A parser makes [design tokens format properties](https://tr.designtokens.org/format/#design-token-properties) compatible with [style-dictionary design token attributes](https://amzn.github.io/style-dictionary/#/tokens?id=design-token-attributes). + +| Design Tokens Format Module | style-dictionary | +| -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| [`$value`](https://tr.designtokens.org/format/#name-and-value) property | [`value`](https://amzn.github.io/style-dictionary/#/tokens?id=design-token-attributes) property | +| [`$type`](https://tr.designtokens.org/format/#type-0) property | implicit nested [`category → type → item` (CTI) convention](https://amzn.github.io/style-dictionary/#/tokens?id=category-type-item) | +| [`$description`](https://tr.designtokens.org/format/#description) property | [`comment`](https://amzn.github.io/style-dictionary/#/tokens?id=design-token-attributes) property | + +### Names + +A design token name is a unique and case-sensitive identifier of a value. A name can be used across different [modes](#modes) to generate style overrides. + +### Groups + +Groups are arbitrary ways to cluster tokens together in a category. They should not be used to infer the type or purpose of design tokens. For that purpose, use the [`$type`](#type) property. + +```json +{ + "group name": { + "token name": { + "$value": "#000" + } + } +} +``` + +Group names prepend design token names in generated output, for example: + +**JavaScript** + +```javascript +const GROUP_NAME_TOKEN_NAME = "#000"; +``` + +**CSS** + +```css +:root { + --group-name-token-name: #000; +} +``` + +**SCSS** + +```scss +$group-name-token-name: #000; +``` + +### Values + +Name and `$value` are the minimum required properties of a design token, `$value` is a reserved word. + +```json +{ + "token name": { + "$value": "16" + } +} +``` + +A design token value can be a string or [alias](#aliases), for example: + +| Example | Value | +| ------------- | ------------------- | +| color | `"#1f75cb"` | +| font weight | `"bold"` | +| spacing scale | `"16"` | +| easing | `"ease-out"` | +| duration | `"200"` | +| alias | `"{color.default}"` | + +### Aliases + +A design token's value can be a reference to another token, for example the alias token `text-color` has the value `{color.default}`: + +```json +{ + "text-color": { + "$value": "{color.default}" + } +} +``` + +This allows generated CSS and SCSS that are output by using [Output References](https://amzn.github.io/style-dictionary/#/formats?id=references-in-output-files) to use references as variables: + +**CSS** + +```css +:root { + --text-color: var(--color-default); +} +``` + +**SCSS** + +```scss +$text-color: $color-default; +``` + +### Type + +An optional [$type](https://tr.designtokens.org/format/#type-0) property is used for value transformations and grouping tokens together, for example: + +```json +{ + "token name": { + "$value": "#000", + "$type": "color" + } +} +``` + +Results in the output `tokens.grouped.json` that can be used for documentation or tooling configuration: + +```json +{ + "color": { + "token name": "#000" + } +} +``` + +### Modes + +Modes are processed on top of default tokens and can be combined with other modes, and inherited separately from stylesheets. Modes are denoted with a `.{mode}.token.json` filename which is used to filter tokens by file, for example: for dark mode token files, end with `.dark.token.json`. + +#### Default design tokens + +**Input** + +`color.tokens.json` + +```json +{ + "text-color": { + "$value": "#000", + "$type": "color" + } +} +``` + +**Output** + +`tokens.grouped.json` + +```json +{ + "color": { + "text-color": "#000" + } +} +``` + +`tokens.js` + +```javascript +export const TEXT_COLOR = "#000"; +``` + +`tokens.scss` + +```scss +$text-color: #000; +``` + +`tokens.css` + +```css +:root { + --text-color: #000; +} +``` + +#### Dark mode design tokens + +Design tokens for different modes are generated separately from default tokens. Using the same name for tokens ensures they will override default values when imported, for example: + +**Input** + +`color.dark.tokens.json` + +```json +{ + "text-color": { + "$value": "#fff", + "$type": "color" + } +} +``` + +**Output** + +`tokens.dark.grouped.json` + +```json +{ + "color": { + "text-color": "#fff" + } +} +``` + +`tokens.dark.js` + +```javascript +export const TEXT_COLOR = "#fff"; +``` + +`tokens.dark.scss` + +```scss +$text-color: #fff; +``` + +`tokens.dark.css` + +```css +:root { + --text: #fff; +} +``` diff --git a/doc/development/fe_guide/frontend_faq.md b/doc/development/fe_guide/frontend_faq.md index 995730796b4..ab75cc27b6a 100644 --- a/doc/development/fe_guide/frontend_faq.md +++ b/doc/development/fe_guide/frontend_faq.md @@ -163,7 +163,7 @@ Sometimes it's necessary to test locally what the frontend production build woul The production build takes a few minutes to be completed. Any code changes at this point are displayed only after executing the item 3 above again. -To return to the normal development mode: +To return to the standard development mode: 1. Open `gitlab.yaml` located in your `gitlab` installation folder, scroll down to the `webpack` section and change back `dev_server` to `enabled: true`. 1. Run `yarn clean` to remove the production assets and free some space (optional). diff --git a/doc/development/fe_guide/graphql.md b/doc/development/fe_guide/graphql.md index da3a6eff79d..00c7bb5d6c8 100644 --- a/doc/development/fe_guide/graphql.md +++ b/doc/development/fe_guide/graphql.md @@ -591,8 +591,7 @@ pageInfo { Here: -- `startCursor` and `endCursor` display the cursor of the first and last items - respectively. +- `startCursor` displays the cursor of the first items and `endCursor` displays the cursor of the last items. - `hasPreviousPage` and `hasNextPage` allow us to check if there are more pages available before or after the current page. @@ -1418,6 +1417,48 @@ wrapper = mount(SomeComponent, { }); ``` +#### Testing subscriptions + +When testing subscriptions, be aware that default behavior for subscription in `vue-apollo@4` is to re-subscribe and immediatelly issue new request on error (unless value of `skip` restricts us from doing that) + +```javascript +import waitForPromises from 'helpers/wait_for_promises'; + +// subscriptionMock is registered as handler function for subscription +// in our helper +const subcriptionMock = jest.fn().mockResolvedValue(okResponse); + +// ... + +it('testing error state', () => { + // Avoid: will stuck below! + subscriptionMock = jest.fn().mockRejectedValue({ errors: [] }); + + // component calls subscription mock as part of + createComponent(); + // will be stuck forever: + // * rejected promise will trigger resubscription + // * re-subscription will call subscriptionMock again, resulting in rejected promise + // * rejected promise will trigger next re-subscription, + await waitForPromises(); + // ... +}) +``` + +To avoid such infinite loops when using `vue@3` and `vue-apollo@4` consider using one-time rejections + +```javascript +it('testing failure', () => { + // OK: subscription will fail once + subscriptionMock.mockRejectedValueOnce({ errors: [] }); + // component calls subscription mock as part of + createComponent(); + await waitForPromises(); + + // code below now will be executred +}) +``` + #### Testing `@client` queries ##### Using mock resolvers diff --git a/doc/development/fe_guide/index.md b/doc/development/fe_guide/index.md index 3d05d395ef1..8675866fce6 100644 --- a/doc/development/fe_guide/index.md +++ b/doc/development/fe_guide/index.md @@ -19,13 +19,68 @@ Be wary of [the limitations that come with using Hamlit](https://github.com/k0ku <!-- vale gitlab.Spelling = YES --> +When it comes to CSS, we use a utils-based CSS approach. GitLab has its own CSS utils which are packaged inside the `gitlab-ui` project and can be seen [in the repository](https://gitlab.com/gitlab-org/gitlab-ui/-/tree/main/src/scss/utility-mixins) or on [UNPKG](https://unpkg.com/browse/@gitlab/ui@latest/src/scss/utility-mixins/). Please favor using these before adding or using any SCSS classes. + We also use [SCSS](https://sass-lang.com) and plain JavaScript with modern ECMAScript standards supported through [Babel](https://babeljs.io/) and ES module support through [webpack](https://webpack.js.org/). +When making API calls, we use [GraphQL](graphql.md) as [the first choice](../../api/graphql/index.md#vision). There are still instances where GitLab REST API is used such as when creating new simple HAML pages or in legacy part of the codebase, but we should always default to GraphQL when possible. + Working with our frontend assets requires Node (v12.22.1 or greater) and Yarn (v1.10.0 or greater). You can find information on how to install these on our [installation guide](../../install/installation.md#5-node). +## Vision + +As Frontend engineers, we strive to give users **delightful experiences**. We should always think of how this applies at GitLab specifically: a great GitLab experience means helping our userbase ship **their own projects faster and with more confidence** when shipping their own software. This means that whenever confronted with a choice for the future of our department, we should remember to try to put this first. + +### Values + +We define three core values, Stability, Speed and Maintainability (SSM) + +#### Stability + +Although velocity is extremely important, we believe that GitLab is now an enterprise-grade platform that requires even the smallest MVC to be **stable, tested and with a good architecture**. We should not merge code, even as an MVC, that could introduce degradation, poor performance, confusion or generally lower our users expectations. + +This is an extension of the core value that want our users to have confidence in their own software and to do so, they need to have **confidence in GitLab first**. This means that our own confidence in our software should be at the absolute maximum. + +#### Speed + +Users should be able to navigate through the GitLab application with ease. This implies fast load times, easy to find pages, clear UX and an overall sense that they can accomplish their goal without friction. + +Additionally, we want our speed to be felt and appreciated by our developers. This means that we should put a lot of effort and thoughts into processes, tools and documentation that help us achieve success faster across our department. This benefits us as engineers, but also our users that end up receiving quality features at a faster rate. + +#### Maintainability + +GitLab is now a large, enterprise-grade software and it often requires complex code to give the best possible experience. Although complexity is a necessity, we must remain vigilent to not let it grow more than it should. To minimize this, we want to focus on making our codebase maintainable by **encapsulating complexity**. This is done by: + +- Building tools that solve commonly-faced problems and making them easily discoverable. +- Writing better documentation on how we solve our problems. +- Writing loosly coupled components that can be easily added or removed from our codebase. +- Remove older technologies or pattern that we deem are no longer acceptable. + +By focusing on these aspects, we aim to allow engineers to contain complexity in well defined boundaries and quickly share them with their peers. + +### Goals + +Now that our values have been defined, we can base our goals on these values and determine what we would like to achieve at GitLab with this in mind. + +- Lowest possible FID, LCP and cross-page navigation times +- Minimal page reloads when interacting with the UI +- Have as little Vue applications per page as possible +- Leverage Ruby ViewComponents for simple pages and avoid Vue overhead when possible +- Migrate existing VueX stores to Apollo, but more urgently **stop using both together** +- Remove jQuery from our codebase +- Add a visual testing framework +- Reduce CSS bundle size to a minimum +- Reduce cognitive overhead and improve maintainability of our CSS +- Improve our pipelines speed +- Build a better set of shared components with documentation + +### Frontend onboarding course + +The [Frontend onboarding course](onboarding_course/index.md) provides a 6-week structured curriculum to learn how to contribute to the GitLab frontend. + ### Browser Support For supported browsers, see our [requirements](../../install/requirements.md#supported-web-browsers). @@ -108,8 +163,8 @@ How we implement [keyboard shortcuts](keyboard_shortcuts.md) that can be customi ## Editors -GitLab text editing experiences are provided by the [Source Editor](source_editor.md) and -the [Content Editor](content_editor.md). +GitLab text editing experiences are provided by the [source editor](source_editor.md) and +the [rich text editor](content_editor.md). ## Frontend FAQ diff --git a/doc/development/fe_guide/logging.md b/doc/development/fe_guide/logging.md index 30cf290fbe8..750bf95e8b2 100644 --- a/doc/development/fe_guide/logging.md +++ b/doc/development/fe_guide/logging.md @@ -22,7 +22,7 @@ Whenever a `catch(e)` exists, and `e` is something unexpected, log the details. ### What makes an error unexpected? -Sometimes a caught exception can be part of normal operations. For instance, third-party +Sometimes a caught exception can be part of standard operations. For instance, third-party libraries might throw an exception based on certain inputs. If we can gracefully handle these exceptions, then they are expected. Don't log them noisily. For example: @@ -36,7 +36,7 @@ try { } catch (e) { if (e instanceof FooSyntaxError) { // To handle a `FooSyntaxError`, we just need to instruct the user to change their input. - // This isn't unexpected, and is part of normal operations. + // This isn't unexpected, and is part of standard operations. setUserMessage(`Try writing better code. ${e.message}`); } else { // We're not sure what `e` is, so something unexpected and bad happened... @@ -50,7 +50,7 @@ try { We have a helpful `~/lib/logger` module which encapsulates how we can consistently log runtime errors in GitLab. Import `logError` from this -module, and use it as you normally would `console.error`. Pass the actual `Error` +module, and use it as you typically would `console.error`. Pass the actual `Error` object, so the stack trace and other details can be captured in the log: ```javascript diff --git a/doc/development/fe_guide/merge_request_widget_extensions.md b/doc/development/fe_guide/merge_request_widget_extensions.md index 07029aec015..f5bf9049db1 100644 --- a/doc/development/fe_guide/merge_request_widget_extensions.md +++ b/doc/development/fe_guide/merge_request_widget_extensions.md @@ -261,8 +261,10 @@ important not to alter the status code and headers. If `fetchCollapsedData()` or `fetchFullData()` methods throw an error: -- The loading state of the extension is updated to `LOADING_STATES.collapsedError` - and `LOADING_STATES.expandedError` respectively. +- The loading state of the extension is updated to `LOADING_STATES.collapsedError` if + `fetchCollapsedData()` method throws an error. +- The loading state of the extension is updated to `LOADING_STATES.expandedError` if + `fetchFullData()` method throws an error. - The extensions header displays an error icon and updates the text to be either: - The text defined in `$options.i18n.error`. - "Failed to load" if `$options.i18n.error` is not defined. diff --git a/doc/development/fe_guide/onboarding_course/index.md b/doc/development/fe_guide/onboarding_course/index.md new file mode 100644 index 00000000000..0b0ffc69f1b --- /dev/null +++ b/doc/development/fe_guide/onboarding_course/index.md @@ -0,0 +1,64 @@ +--- +stage: Manage +group: Foundations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Frontend onboarding course + +Welcome to the Frontend Onboarding Course at GitLab! +In this course, we walk you through your first professional frontend development experience, helping you gain real world skills and learn how to contribute to a large-scale codebase effectively. + +Throughout the course, we'll follow a structured approach. +Each lesson focuses on solving a specific problem on GitLab, giving you hands-on experience. +You'll learn theory that you can immediately put into practice. +By working on real-world GitLab issues, you'll encounter challenges and learn how to navigate the codebase effectively while at the same time improving GitLab the product. + +We believe in an interactive learning experience. +You'll have the opportunity to ask questions and seek help from the GitLab community. +We appreciate your contributions and are here to support your learning while at the same time making GitLab better. + +Our teaching style prioritizes practical learning. +Lessons include an introduction to the problem, theory, live coding walkthroughs, and similar issues for you to tackle. +As you progress, the complexity of the tasks increase, helping you grow your skills. + +Join us on this journey of front-end development at GitLab. Say hello in [the Discord community](https://discord.gg/gitlab) and let's learn and improve together. + +## Lessons + +- [Lesson 1](lesson_1.md) + +## Structure and timings + +The course is run over 6 weeks, with a required time commitment of 5-10 hours per week. + +The course is free of charge, but we do ask for a commitment to complete the curriculum (including 10 merged merge requests). + +After completing the course, you receive a certificate and GitLab achievement. + +Each week consists of the following sessions: + +- 1-hour relaxed discussion-style lesson with explanation of how GitLab frontend works. Each week features a different guest and includes an AMA portion. +- 2-hour live coding lesson with a practical task for participants to complete. +- 2 x 2-hour dedicated “Office Hours” sessions where participants can work on the task assigned in the lesson with GitLab frontend engineers. (2 sessions in different timezones as this will require participants to join synchronously) + +A fortnightly 1-on-1 mentoring sessions are also available to each participant. + +There are 10 places available on the course. +The date will be set after the course material has been prepared. +Please complete the [Frontend Onboarding Course Application Form](https://forms.gle/39Rs4w4ZxQuByhE4A) to apply. + +You may also participate in the course informally at your own pace, without the benefit of the synchronous office hours or mentoring session. +GitLab team members are happy to support you regardless. + +## Curriculum summary + +### Lesson 1 + +- What is a development environment? + - What is the GDK? + - Installing the GDK. + - GDK tips and tricks. + - Using GitPod to run the GDK. +- Navigating the GitLab codebase. +- Writing a good merge request. diff --git a/doc/development/fe_guide/onboarding_course/lesson_1.md b/doc/development/fe_guide/onboarding_course/lesson_1.md new file mode 100644 index 00000000000..e82d350f854 --- /dev/null +++ b/doc/development/fe_guide/onboarding_course/lesson_1.md @@ -0,0 +1,183 @@ +--- +stage: manage +group: foundations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Lesson 1 + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=k4C3-FKvZyI">Lesson 1 intro</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/k4C3-FKvZyI" frameborder="0" allowfullscreen> </iframe> +</figure> + +In this lesson you tackle the smallest of problems - a one-character text change. To do so, we have to learn: + +- How to set up a GitLab Development Environment. +- How to navigate the GitLab code base. +- How to create a merge request in the GitLab project. + +After we have learned these 3 things, a GitLab team member will do a live coding demo. +In the demo, they'll use each of the things learned by completing one of these small issues, so that you can complete an issue by yourself. + +There is a list of issues that are very similar to the one we'll be live coding [here in the "Linked items" section](https://gitlab.com/gitlab-org/gitlab/-/issues/389920), it would be worth commenting on one of these now to get yourself assigned to one so that you can follow along. + +## What is the GDK? + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=qXGXshfo934">What is the GDK</a>? +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/qXGXshfo934" frameborder="0" allowfullscreen> </iframe> +</figure> + +The GDK (GitLab Development Kit) is a local instance of GitLab that allows developers to run and test GitLab on their own computers. +Unlike frontend only applications, the GDK runs the entire GitLab application, including the back-end services, APIs, and a local database. +This allows developers to make changes, test them in real-time, and validate their modifications. + +Tips for using the GDK: + +- Troubleshooting documentation: When encountering issues with the GDK, refer to the troubleshooting documentation in the [GDK repository](https://gitlab.com/gitlab-org/gitlab-development-kit/-/tree/main/doc/troubleshooting). + These resources provide useful commands and tips to help resolve common problems. +- Using the Rails console: The Rails console is an essential tool for interacting with your local instance of GitLab. + You can access it by running `gdk rails c` and use it to enable or disable feature flags, perform backend operations, and more. +- Stay updated: Regularly update your GDK by running `gdk update`. + This command fetches the latest branch of the GitLab project, as well as the latest branch of the GDK and its dependencies. + Keeping your GDK up to date helps ensure you will be working with the latest version of GitLab and make sure you have the latest bug fixes. + +Remember, if you need further assistance or have specific questions, you can reach out to the GitLab community through our [Discord](https://discord.gg/gitlab) or [other available support channels](https://about.gitlab.com/community/contribute/). + +## Installing and using the GDK locally + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=fcOyjuCizmY">Installing the GDK</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/fcOyjuCizmY" frameborder="0" allowfullscreen> </iframe> +</figure> + +For the latest installation instructions, refer to the [GitLab Development Kit documentation](https://gitlab.com/gitlab-org/gitlab-development-kit#installation). + +Here's a step-by-step summary: + +1. Prerequisites: + - 16 GB RAM. If you have less, consider [using Gitpod](#using-gitpod-instead-of-running-the-gdk-locally) + - Ensure that Git is installed on your machine. + - Install a code editor, such as Visual Studio Code. + - [Create an account](https://gitlab.com/users/sign_up) or [sign in](https://gitlab.com/users/sign_in) on GitLab.com and join the [community members group](https://gitlab.com/gitlab-community/meta#request-access-to-community-forks). +1. Installation: + - Choose a directory to install the GitLab Development Kit (GDK). + - Open your terminal and navigate to the chosen directory. + - Download and run the installation script from the terminal: + + ```shell + curl "https://gitlab.com/gitlab-org/gitlab-development-kit/-/raw/main/support/install" | bash + ``` + + - Only run scripts from trusted sources to ensure your safety. + - The installation process may take around 20 minutes or more. +1. Choosing the repository: + - Instead of cloning the main GitLab repository, use the community fork recommended for wider community members. + - Follow the instructions provided to install the community fork. +1. GDK structure: + - After the installation, the GDK directory is created. + - Inside the GDK directory, you'll find the GitLab project folder. +1. Working with the GDK: + - GDK offers lots of commands you can use to interact with your installation. To run those commands you must be inside the GDK or GitLab folder. + - To start the GDK, run the command `gdk start` in your terminal. + - You can explore available commands and options by running `gdk help` in the terminal. + +Remember to consult the documentation or seek community support if you have any further questions or issues. + +## Using Gitpod instead of running the GDK locally + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=RI2kM5_oii4">Using Gitpod with GitLab</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/RI2kM5_oii4" frameborder="0" allowfullscreen> </iframe> +</figure> + +Gitpod is a service that allows you to run a virtual machine, specifically the GitLab Development Kit (GDK), on the Gitpod server instead of running it on your own machine. +It provides a web-based Integrated Development Environment (IDE) where you can edit code and see the GDK in action. +Gitpod is useful for quickly getting a GDK environment up and running, for making small merge requests without installing the GDK locally, or for running GDK on a machine that may not have enough resources. + +To use Gitpod: + +1. Go to the [GitLab community fork website](https://gitlab.com/gitlab-community/gitlab), select **Edit**, then select **Gitpod**. +1. Configure your settings, such as the editor (VS Code desktop or browser) and the context (usually the `main` or `master` branch). +1. Select **Open** to create your Gitpod workspace. This process may take up to 20 minutes. The GitLab Development Kit (GDK) will be installed in the Gitpod workspace. This installation is faster than downloading and installing the full GDK locally. + +After the workspace is created, you'll find your chosen IDE running in your browser. You can also connect it to your desktop IDE if preferred. +Treat Gitpod just like you would use VS Code locally. Create branches, make code changes, commit them, and push them back to the community fork. + +Other tips: + +- Remember to push your code regularly to avoid the workspace timing out. Idle workspaces are eventually destroyed. +- Customize your Gitpod workspace settings if needed, such as making your instance of GitLab frontend publicly available. +- If you run out of minutes, contact the support team on the Discord server. +- Troubleshoot issues by using commands like `gdk start` and `gdk status` in the Gitpod workspace as you would if it was running locally. + +By following these steps, you can leverage Gitpod to efficiently develop with the GitLab Development Kit without the need for local installation. + +## Navigating the GitLab codebase + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=Wc5u879_0Aw">How to navigate the GitLab codebase</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/Wc5u879_0Aw" frameborder="0" allowfullscreen> </iframe> +</figure> + +Understanding how to navigate the GitLab codebase is essential for contributors. +Navigating the codebase and locating specific files can be challenging but crucial for making changes and addressing issues effectively. +Here we'll explore a step-by-step process for finding files and finding where they are rendered in GitLab. + +If you already know the file you are going to work on and now you want to find where it is rendered: + +1. Start by gathering clues to understand the file’s purpose. Look for relevant information within the file itself, such as keywords or specific content that might indicate its context. +1. You can also examine the file path (or folder structure) to gain insights into where the file might be rendered. + A lot of routing in GitLab is very similar to the folder structure. +1. If you can work out which feature (or one of the features) that this component is used in, you can then leverage the GitLab user documentation to find out how to navigate to the feature page. +1. Follow the component hierarchy, do a global search for the file name to identify the parent component that renders the component. + Continue to follow the hierarchy of components to trace back to a feature you recognize or can search for in the GitLab user docs. +1. You can use `git blame` with an extension like GitLens to find a recent MR where this file was changed. + Most MR’s have a "How to validate" section that you can follow, if the MR doesn't have one, look for the previous change and until you find one that have validation steps. + +If you know which page you need to fix and you want to find the file path, here are some things you can try: + +- Look for content that is unique and doesn’t contain variables so that you can search for the translation variable. +- Try using Vue Dev Tools to find the component name. +- Look for unique identifiers like a `data-testid`,`id` or a unique looking CSS class in the HTML of the component and then search globally the codebase for those identifying strings. + +## Writing a good merge request + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=H5zozDNIn98">How to write a good MR</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/H5zozDNIn98" frameborder="0" allowfullscreen> </iframe> +</figure> + +When writing a merge request there are some important things to be aware of: + +- Your MR will become a permanent part of the documentation of the GitLab project. + It may be used in the future to help people understand why some code works the way it does and why it doesn't use an alternative solution. +- At least 2 other engineers are going to review your code. For the sake of efficiency (much like the code itself you have written) it is best to take a little while longer to get your MR right so that it is quicker and easier for others to read. +- The MRs that you create on GitLab are available to the public. This means you can add a link to MRs you are particularly proud of to your portfolio page when looking for a job. +- Since an MR is a technical document, you should try to implement a technical writing style. + If you don’t know what that is, here is a highly recommended short course from [Google on Technical writing](https://developers.google.com/tech-writing/one). + If you are also contributing to the documentation at GitLab, there is a [Technical Writing Fundamentals course available here from GitLab](https://about.gitlab.com/handbook/product/ux/technical-writing/fundamentals/). + +## Live coding + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=BJCCwc1Czt4">Lesson 1 code walkthrough</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/BJCCwc1Czt4" frameborder="0" allowfullscreen> </iframe> +</figure> + +Now it is your turn to complete your first MR, there is a list of issues that are very similar to the one we just finished that need completing [here in the "Linked items" section](https://gitlab.com/gitlab-org/gitlab/-/issues/389920). Thanks for contributing! (if there are none left, let us know on [Discord](https://discord.gg/gitlab) or [other available support channels](https://about.gitlab.com/community/contribute/) and we'll find more for you) diff --git a/doc/development/fe_guide/storybook.md b/doc/development/fe_guide/storybook.md index eaa8f8b4068..6049dd7c7d3 100644 --- a/doc/development/fe_guide/storybook.md +++ b/doc/development/fe_guide/storybook.md @@ -55,7 +55,7 @@ To add a story: ## Using GitLab REST and GraphQL APIs -You can write stories for components that use either GitLab’s [REST](../../api/rest/index.md) or +You can write stories for components that use either the GitLab [REST](../../api/rest/index.md) or [GraphQL](../../api/graphql/index.md) APIs. ### Set up API access token and GitLab instance URL @@ -72,7 +72,7 @@ a starting point. 1. Set the `API_ACCESS_TOKEN` variable to the access token that you created. -1. Set the `GITLAB_URL` variable to the GitLab instance’s domain URL, for example: `http://gdk.test:3000`. +1. Set the `GITLAB_URL` variable to the GitLab instance's domain URL, for example: `http://gdk.test:3000`. 1. Start or restart your storybook. @@ -80,7 +80,7 @@ You can also use the GitLab API Access panel in the Storybook UI to set the GitL ### Set up API access in your stories -You should apply the `withGitLabAPIAccess` decorator to the stories that will consume GitLab’s APIs. This decorator +You should apply the `withGitLabAPIAccess` decorator to the stories that will consume GitLab APIs. This decorator will display a badge indicating that the story won't work without providing the API access parameters: ```javascript diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md index 987543642f0..cccaefe8a19 100644 --- a/doc/development/fe_guide/style/javascript.md +++ b/doc/development/fe_guide/style/javascript.md @@ -331,8 +331,23 @@ Only export the constants as a collection (array, or object) when there is a nee ## Error handling -When catching a server-side error, you should use the error message -utility function contained in `app/assets/javascripts/lib/utils/error_message.js`. +For internal server errors when the server returns `500`, you should return a +generic error message. + +When the backend returns errors, the errors should be +suitable to display back to the user. + +If for some reason, it is difficult to do so, as a last resort, you can +select particular error messages with prefixing: + +1. Ensure that the backend prefixes the error messages to be displayed with: + + ```ruby + Gitlab::Utils::ErrorMessage.to_user_facing('Example user-facing error-message') + ``` + +1. Use the error message utility function contained in `app/assets/javascripts/lib/utils/error_message.js`. + This utility accepts two parameters: the error object received from the server response and a default error message. The utility examines the message in the error object for a prefix that indicates whether the message is meant to be user-facing or not. If the message is intended @@ -347,7 +362,6 @@ onError(error) { } ``` -To benefit from this parsing mechanism, the utility user should ensure that the server-side -code is aware of this utility's usage and prefixes the error messages where appropriate -before sending them back to the user. See -[Error handling for API](../../api_styleguide.md#error-handling) for more information. +Note that this prefixing must not be used for API responses. Instead follow the +[REST API](../../../api/rest/index.md#data-validation-and-error-reporting), +or [GraphQL guides](../../api_graphql_styleguide.md#error-handling) on how to consume error objects. diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 1a43084245e..8230f38ad8e 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -353,10 +353,11 @@ return new Vue({ #### Accessing feature flags -Use the [`provide` and `inject`](https://v2.vuejs.org/v2/api/#provide-inject) mechanisms -in Vue to make feature flags available to any descendant components in a Vue -application. The `glFeatures` object is already provided in `commons/vue.js`, so -only the mixin is required to use the flags: +After pushing a feature flag to the [frontend](../feature_flags/index.md#frontend), +use the [`provide` and `inject`](https://v2.vuejs.org/v2/api/#provide-inject) +mechanisms in Vue to make feature flags available to any descendant components +in a Vue application. The `glFeatures` object is already provided in +`commons/vue.js`, so only the mixin is required to use the flags: ```javascript // An arbitrary descendant component diff --git a/doc/development/fe_guide/vuex.md b/doc/development/fe_guide/vuex.md index 52278c94e9f..e096d25f2d9 100644 --- a/doc/development/fe_guide/vuex.md +++ b/doc/development/fe_guide/vuex.md @@ -284,7 +284,7 @@ To set this initial state, pass it as a parameter to your store's creation function when mounting your Vue component: ```javascript -// in the Vue app's initialization script (e.g. mount_show.js) +// in the Vue app's initialization script (for example, mount_show.js) import Vue from 'vue'; import Vuex from 'vuex'; diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index 30837ac8f3f..d9eb29a7b7f 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -149,7 +149,7 @@ created using the [Experiment Tracking template](https://gitlab.com/gitlab-org/g `worker` feature flags are used for controlling Sidekiq workers behavior, such as deferring Sidekiq jobs. `worker` feature flags likely do not have any YAML definition as the name could be dynamically generated using -the worker name itself, e.g. `defer_sidekiq_jobs_AuthorizedProjectsWorker`. Some examples for using `worker` type feature +the worker name itself, for example, `run_sidekiq_jobs_AuthorizedProjectsWorker`. Some examples for using `worker` type feature flags can be found in [deferring Sidekiq jobs](#deferring-sidekiq-jobs). ## Feature flag definition and validation @@ -348,7 +348,7 @@ Use the `push_frontend_feature_flag` method which is available to all controller ```ruby before_action do - # Prefer to scope it per project or user e.g. + # Prefer to scope it per project or user, for example push_frontend_feature_flag(:vim_bindings, project) end @@ -713,33 +713,47 @@ Feature flags with [`worker` type](#worker-type) can be used to control the beha ### Deferring Sidekiq jobs -Feature flags with the format of `defer_sidekiq_jobs_{WorkerName}` delay the execution of the worker -by scheduling the job at a later time. +When disabled, feature flags with the format of `run_sidekiq_jobs_{WorkerName}` delay the execution of the worker +by scheduling the job at a later time. This feature flag is enabled by default for all workers. Deferring jobs can be useful during an incident where contentious behavior from worker instances are saturating infrastructure resources (such as database and database connection pool). -The implementation can be found at [DeferJobs Sidekiq server middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/sidekiq_middleware/defer_jobs.rb). +The implementation can be found at [SkipJobs Sidekiq server middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/sidekiq_middleware/skip_jobs.rb). NOTE: -Jobs are deferred indefinitely as long as the feature flag is enabled. It is important to disable the +Jobs are deferred indefinitely as long as the feature flag is disabled. It is important to remove the feature flag after the worker is deemed safe to continue processing. -When set to true, 100% of the jobs are deferred. When you want processing to resume, you can +When set to false, 100% of the jobs are deferred. When you want processing to resume, you can use a **percentage of time** rollout. For example: ```shell -# defer 100% of the jobs -/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker true +# not running any jobs, deferring all 100% of the jobs +/chatops run feature set run_sidekiq_jobs_SlowRunningWorker false -# defer 99% of the jobs, only letting 1% processed -/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker 99 +# only running 10% of the jobs, deferring 90% of the jobs +/chatops run feature set run_sidekiq_jobs_SlowRunningWorker 10 -# defer 50% of the jobs -/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker 50 +# running 50% of the jobs, deferring 50% of the jobs +/chatops run feature set run_sidekiq_jobs_SlowRunningWorker 50 -# stop deferring the jobs, jobs are being processed normally -/chatops run feature set defer_sidekiq_jobs_SlowRunningWorker false +# back to running all jobs normally +/chatops run feature delete run_sidekiq_jobs_SlowRunningWorker +``` + +### Dropping Sidekiq jobs + +Instead of [deferring jobs](#deferring-sidekiq-jobs), jobs can be entirely dropped by enabling the feature flag +`drop_sidekiq_jobs_{WorkerName}`. Use this feature flag when you are certain the jobs are safe to be dropped, i.e. +the jobs do not need to be processed in the future. + +```shell +# drop all the jobs +/chatops run feature set drop_sidekiq_jobs_SlowRunningWorker true + +# process jobs normally +/chatops run feature delete drop_sidekiq_jobs_SlowRunningWorker ``` NOTE: -The percentage of time value denotes the percentage of time the jobs are being deferred (instead of being processed). -For example, setting to `99` means only 1% of the jobs are being processed at random. +Dropping feature flag (`drop_sidekiq_jobs_{WorkerName}`) takes precedence over deferring feature flag (`run_sidekiq_jobs_{WorkerName}`), +i.e. when `drop_sidekiq_jobs` is enabled and `run_sidekiq_jobs` is disabled, jobs are entirely dropped. diff --git a/doc/development/features_inside_dot_gitlab.md b/doc/development/features_inside_dot_gitlab.md index 3c988ec6b21..f55d7e52fbc 100644 --- a/doc/development/features_inside_dot_gitlab.md +++ b/doc/development/features_inside_dot_gitlab.md @@ -16,4 +16,4 @@ When implementing new features, please refer to these existing features to avoid - [Route Maps](../ci/review_apps/index.md#route-maps): `.gitlab/route-map.yml`. - [Customize Auto DevOps Helm Values](../topics/autodevops/customize.md#customize-helm-chart-values): `.gitlab/auto-deploy-values.yaml`. - [Insights](../user/project/insights/index.md#configure-project-insights): `.gitlab/insights.yml`. -- [Service Desk Templates](../user/project/service_desk.md#create-customized-email-templates): `.gitlab/service_desk_templates/`. +- [Service Desk Templates](../user/project/service_desk.md#customize-emails-sent-to-the-requester): `.gitlab/service_desk_templates/`. diff --git a/doc/development/fips_compliance.md b/doc/development/fips_compliance.md index 830a8e3cd2a..bab4d7705f9 100644 --- a/doc/development/fips_compliance.md +++ b/doc/development/fips_compliance.md @@ -59,7 +59,7 @@ listed here that also do not work properly in FIPS mode: - [Container Scanning](../user/application_security/container_scanning/index.md) support for scanning images in repositories that require authentication. - [Code Quality](../ci/testing/code_quality.md) does not support operating in FIPS-compliant mode. - [Dependency scanning](../user/application_security/dependency_scanning/index.md) support for Gradle. -- [Dynamic Application Security Testing (DAST)](../user/application_security/dast/index.md) supports a reduced set of analyzers. Browser-based and proxy-based analyzers are not available in FIPS mode today, however DAST API and DAST API Fuzzing images are available. +- [Dynamic Application Security Testing (DAST)](../user/application_security/dast/proxy-based.md) supports a reduced set of analyzers. The proxy-based analyzer is not available in FIPS mode today, however browser-based DAST, DAST API, and DAST API Fuzzing images are available. - [License compliance](../user/compliance/license_compliance/index.md). - [Solutions for vulnerabilities](../user/application_security/vulnerabilities/index.md#resolve-a-vulnerability) for yarn projects. diff --git a/doc/development/gemfile.md b/doc/development/gemfile.md index e6275068ea8..ed38f6481e7 100644 --- a/doc/development/gemfile.md +++ b/doc/development/gemfile.md @@ -118,9 +118,8 @@ Read more about [Gems development guidelines](gems.md). When upgrading the Rails gem and its dependencies, you also should update the following: +- The [`activerecord_version` in the vendored `attr_encrypted` gemspec](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/gems/attr_encrypted/attr_encrypted.gemspec). - The [`Gemfile` in the `qa` directory](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/Gemfile). -- The [`Gemfile` in Gitaly Ruby](https://gitlab.com/gitlab-org/gitaly/-/blob/master/ruby/Gemfile), - to ensure that we ship only one version of these gems. You should also update npm packages that follow the current version of Rails: diff --git a/doc/development/gems.md b/doc/development/gems.md index 709dfa105bf..57acac7287b 100644 --- a/doc/development/gems.md +++ b/doc/development/gems.md @@ -9,38 +9,37 @@ info: To determine the technical writer assigned to the Stage/Group associated w GitLab uses Gems as a tool to improve code reusability and modularity in a monolithic codebase. -Sometimes we create libraries within our codebase that we want to -extract, either because their functionality is highly isolated, -we want to use them in other applications -ourselves, or we think it would benefit the wider community. -Extracting code to a gem also means that we can be sure that the gem -does not contain any hidden dependencies on our application code. +We extract libraries from our codebase when their functionality +is highly isolated and we want to use them in other applications +ourselves or we think it would benefit the wider community. -## When to use Gems +Extracting code to a gem also ensures that the gem does not contain any hidden +dependencies on our application code. -Gems should be used always when implementing functions that can be considered isolated, -that are decoupled from the business logic of GitLab and can be developed separately. Consider the -following examples where Gem logic could be placed: +Gems should always be used when implementing functionality that can be considered isolated, +that are decoupled from the business logic of GitLab and can be developed separately. -The best example where we can look for opportunities to introduce new gems +The best place in a Rails codebase with opportunities to extract new gems is the [lib/](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/) folder. -The **lib/** folder is a mix of code that is generic/universal, GitLab-specific, and tightly integrated with the rest of the codebase. +Our **lib/** folder is a mix of code that is generic/universal, GitLab-specific, and tightly integrated with the rest of the codebase. -If you cannot find a good place for your code in **lib/** you should strongly -consider creating the new Gem [In the same repo](#in-the-same-repo). +In order to decide whether to extract part of the codebase as a Gem, ask yourself the following questions: -## In the same repo +1. Is this code generic or universal that can be done as a separate and small project? +1. Do I expect it to be used internally outside of the Monolith? +1. Is this useful for the wider community that we should consider releasing as a separate component? -**Our GitLab Gems should be always put in `gems/` of GitLab monorepo.** +If the answer is **Yes** for any of the questions above, you should strongly consider creating a new Gem. -That gives us the advantages of gems (modular code, quicker to run tests in development). -and prevents complexity (coordinating changes across repos, new permissions, multiple projects, etc.). +You can always start by creating a new Gem [in the same repo](#in-the-same-repo) and later evaluate whether to migrate it to a separate repository, when it is intended +to be used by a wider community. -Gems stored in the same repo should be referenced in `Gemfile` with the `path:` syntax. -They should not be published to RubyGems. +WARNING: +To prevent malicious actors from name-squatting the extracted Gems, follow the instructions +to [reserve a gem name](#reserve-a-gem-name). -### Advantages +## Advantages of using Gems Using Gems can provide several benefits for code maintenance: @@ -58,11 +57,102 @@ Using Gems can provide several benefits for code maintenance: Since the gem is packaged, not changed too often, it also allows us to run those tests less frequently improving CI testing time. -### To Do +## Gem naming + +Gems can fall under three different case: + +- `unique_gem`: Don't include `gitlab` in the gem name if the gem doesn't include anything specific to GitLab +- `existing_gem-gitlab`: When you fork and modify/extend a publicly available gem, add the `-gitlab` suffix, according to [Rubygems' convention](https://guides.rubygems.org/name-your-gem/) +- `gitlab-unique_gem`: Include a `gitlab-` prefix to gems that are only useful in the context of GitLab projects. + +Examples of existing gems: + +- `y-rb`: Ruby bindings for yrs. Yrs "wires" is a Rust port of the Yjs framework. +- `activerecord-gitlab`: Adds GitLab-specific patches to the `activerecord` public gem. +- `gitlab-rspec` and `gitlab-utils`: GitLab-specific set of classes to help in a particular context, or re-use code. + +## In the same repo + +**When extracting Gems from existing codebase, put them in `gems/` of the GitLab monorepo** + +That gives us the advantages of gems (modular code, quicker to run tests in development). +and prevents complexity (coordinating changes across repos, new permissions, multiple projects, etc.). + +Gems stored in the same repo should be referenced in `Gemfile` with the `path:` syntax. + +WARNING: +To prevent malicious actors from name-squatting the extracted Gems, follow the instructions +to [reserve a gem name](#reserve-a-gem-name). + +### Create and use a new Gem + +You can see example adding a new gem: [!121676](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121676). + +1. Pick a good name for the gem, by following the [Gem naming](#gem-naming) convention. +1. Create the new gem in `gems/<name-of-gem>` with `bundle gem gems/<name-of-gem> --no-exe --no-coc --no-ext --no-mit`. +1. Remove the `.git` folder in `gems/<name-of-gem>` with `rm -rf gems/<name-of-gem>/.git`. +1. Edit `gems/<name-of-gem>/README.md` to provide a simple description of the Gem. +1. Edit `gems/<name-of-gem>/<name-of-gem>.gemspec` and fill the details about the Gem as in the following example: + + ```ruby + # frozen_string_literal: true + + require_relative "lib/name/of/gem/version" + + Gem::Specification.new do |spec| + spec.name = "<name-of-gem>" + spec.version = Name::Of::Gem::Version::VERSION + spec.authors = ["group::tenant-scale"] + spec.email = ["engineering@gitlab.com"] + + spec.summary = "Gem summary" + spec.description = "A more descriptive text about what the gem is doing." + spec.homepage = "https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/<name-of-gem>" + spec.license = "MIT" + spec.required_ruby_version = ">= 3.0" + spec.metadata["rubygems_mfa_required"] = "true" + + spec.files = Dir['lib/**/*.rb'] + spec.require_paths = ["lib"] + end + ``` + +1. Update `gems/<name-of-gem>/.rubocop.yml` with: + + ```yaml + inherit_from: + - ../config/rubocop.yml + ``` + +1. Configure CI for a newly added Gem: -#### Desired use cases + - Add `gems/<name-of-gem>/.gitlab-ci.yml`: -The `gitlab-utils` is a Gem containing as of set of class that implement common intrisic functions + ```yaml + include: + - local: gems/gem.gitlab-ci.yml + inputs: + gem_name: "<name-of-gem>" + ``` + + - To `.gitlab/ci/gitlab-gems.gitlab-ci.yml` add: + + ```yaml + include: + - local: .gitlab/ci/templates/gem.gitlab-ci.yml + inputs: + gem_name: "<name-of-gem>" + ``` + +1. Reference Gem in `Gemfile` with: + + ```ruby + gem '<name-of-gem>', path: 'gems/<name-of-gem>' + ``` + +### Examples of Gem extractions + +The `gitlab-utils` is a Gem containing as of set of class that implement common intrinsic functions used by GitLab developers, like `strong_memoize` or `Gitlab::Utils.to_boolean`. The `gitlab-database-schema-migrations` is a potential Gem containing our extensions to Rails @@ -72,7 +162,7 @@ or potentially be upstreamed. The `gitlab-database-load-balancing` similar to previous is a potential Gem to implement GitLab specific load balancing to Rails database handling. Since this is rather complex and highly specific code -maintaing it's complexity in a isolated and well tested Gem would help with removing this complexity +maintaining its complexity in a isolated and well tested Gem would help with removing this complexity from a big monolithic codebase. The `gitlab-flipper` is another potential Gem implementing all our custom extensions to support feature @@ -81,18 +171,10 @@ usage, adding consistency checks and various helpers to track owners of feature not really part of GitLab business logic and could be used to better track our implementation of Flipper and possibly much easier change it to dogfood [GitLab Feature Flags](../operations/feature_flags.md). -The `gitlab-ci-reports-parsers` is a potential Gem that could implement all various parsers for various formats. -The parsed output would be transformed into objects that could then be used by GitLab the application -to store it in the database. This functionality could be an additional Gem since it is isolated, -rarely changed, and GitLab Rails only consumes the data. - -The same pattern could be applied to all other type of parsers, like security vulnerabilities, or any -other complex structures that need to be transformed into a form that is consumed by GitLab Rails. - -The `gitlab-active_record` is a gem adding GitLab specific Active Record patches. +The `activerecord-gitlab` is a gem adding GitLab specific Active Record patches. It is very well desired for such to be managed separately to isolate complexity. -#### Other potential use cases +### Other potential use cases The `gitlab-ci-config` is a potential Gem containing all our CI code used to parse `.gitlab-ci.yml`. This code is today lightly interlocked with GitLab the application due to lack of proper abstractions. @@ -101,103 +183,6 @@ with GitLab the application. The interface would for example define an adapter t Once we would have a `gitlab-ci-config` Gem it could be used within GitLab and outside of GitLab Rails and [GitLab CLI](https://gitlab.com/gitlab-org/cli). -### Create and use a new Gem - -You can see example adding new Gem: [!121676](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121676). - -1. Create a new Ruby Gem in `gems/gitlab-<name-of-gem>` with `bundle gem gems/gitlab-<name-of-gem> --no-exe --no-coc --no-ext --no-mit`. -1. Edit or remove `gitlab-<name-of-gem>/README.md` to provide a simple one paragraph description of the Gem. -1. Edit `gitlab-<name-of-gem>/gitlab-<name-of-gem>.gemspec` and fill the details about the Gem as in the following example: - - ```ruby - Gem::Specification.new do |spec| - spec.name = "gitlab-<name-of-gem>" - spec.version = Gitlab::NameOfGem::VERSION - spec.authors = ["group::tenant-scale"] - spec.email = ["engineering@gitlab.com"] - - spec.summary = "GitLab's RSpec extensions" - spec.description = "A set of useful helpers to configure RSpec with various stubs and CI configs." - spec.homepage = "https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-<name-of-gem>" - spec.required_ruby_version = ">= 2.6.0" - end - ``` - -1. Update `gems/gitlab-<name-of-gem>/.rubocop` with: - - ```yaml - inherit_from: - - ../../.rubocop.yml - - CodeReuse/ActiveRecord: - Enabled: false - - AllCops: - TargetRubyVersion: 3.0 - - Naming/FileName: - Exclude: - - spec/**/*.rb - ``` - -1. Configure CI for a newly added Gem: - -- Add `gems/gitlab-<name-of-gem>/.gitlab-ci.yml`: - - ```yaml - workflow: - rules: - - if: $CI_MERGE_REQUEST_ID - - rspec: - image: "ruby:${RUBY_VERSION}" - cache: - key: gitlab-<name-of-gem> - paths: - - gitlab-<name-of-gem>/vendor/ruby - before_script: - - cd vendor/gems/bundler-checksum - - ruby -v # Print out ruby version for debugging - - gem install bundler --no-document # Bundler is not installed with the image - - bundle config set --local path 'vendor' # Install dependencies into ./vendor/ruby - - bundle config set with 'development' - - bundle config set --local frozen 'true' # Disallow Gemfile.lock changes on CI - - bundle config # Show bundler configuration - - bundle install -j $(nproc) - script: - - bundle exec rspec - parallel: - matrix: - - RUBY_VERSION: ["2.7", "3.0", "3.1", "3.2"] - ``` - -- To `.gitlab/ci/rules.gitlab-ci.yml` add: - - ```yaml - .gems:rules:gitlab-<name-of-gem>: - rules: - - <<: *if-merge-request - changes: ["gems/gitlab-<name-of-gem>/**/*"] - ``` - -- To `.gitlab/ci/gitlab-gems.gitlab-ci.yml` add: - - ```yaml - gems gitlab-<name-of-gem>: - extends: - - .gems:rules:gitlab-<name-of-gem> - needs: [] - trigger: - include: gems/gitlab-<name-of-gem>/.gitlab-ci.yml - strategy: depend - ``` - -1. Reference Gem in `Gemfile` with: - - ```ruby - gem 'gitlab-<name-of-gem>', path: 'gems/gitlab-<name-of-gem>' - ``` - ## In the external repo In general, we want to think carefully before doing this as there are @@ -319,3 +304,13 @@ to store them in monorepo: - It is expected that vendored gems might be published by third-party. - Those Gems will not be published by us to RubyGems. - Those Gems will be referenced via `path:` in `Gemfile`, since we cannot depend on RubyGems. + +## Reserve a gem name + +We reserve a gem name as a precaution **before publishing any public code that contains a new gem**, to avoid name-squatters taking over the name in RubyGems. + +To reserve a gem name, follow the steps to [Create and publish a Ruby gem](#create-and-publish-a-ruby-gem), with the following changes: + +- Use `0.0.0` as the version. +- Include a single file `lib/NAME.rb` with the content `raise "Reserved for GitLab"`. +- Perform the `build` and `publish`, and check <https://rubygems.org/gems/> to confirm it succeeded. diff --git a/doc/development/geo.md b/doc/development/geo.md index 5d09532afcb..a39f97f1241 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -111,7 +111,7 @@ projects that need updating. Those projects can be: timestamp that is more recent than the `last_repository_successful_sync_at` timestamp in the `Geo::ProjectRegistry` model. - Manual: The administrator can manually flag a repository to resync in the - [Geo Admin Area](../user/admin_area/geo_sites.md). + [Geo Admin Area](../administration/geo_sites.md). When we fail to fetch a repository on the secondary `RETRIES_BEFORE_REDOWNLOAD` times, Geo does a so-called _re-download_. It will do a clean clone @@ -425,18 +425,17 @@ not used, so sessions and so on, aren't shared between sites. ## Object Storage GitLab can optionally use Object Storage to store data it would -otherwise store on disk. These things can be: +otherwise store on disk. For example: - LFS Objects - CI Job Artifacts - Uploads -Objects that are stored in object storage, are not handled by Geo. Geo -ignores items in object storage. Either: +By default, Geo does not replicate objects that are stored in object storage. Depending on the situation and needs of the customer, they can: -- The object storage layer should take care of its own geographical - replication. -- All secondary sites should use the same storage site. +- [Enable GitLab-managed object storage replication](../administration/geo/replication/object_storage.md#enabling-gitlab-managed-object-storage-replication). +- Use their cloud provider's built-in services to replicate object storage across Geo sites. +- Configure secondary Geo sites to access the same object storage endpoint as the primary site. ## Verification @@ -467,7 +466,7 @@ basically hashes all Git refs together and stores that hash in the The **secondary** site does the same to calculate the hash of its clone, and compares the hash with the value the **primary** site calculated. If there is a mismatch, Geo will mark this as a mismatch -and the administrator can see this in the [Geo Admin Area](../user/admin_area/geo_sites.md). +and the administrator can see this in the [Geo Admin Area](../administration/geo_sites.md). ## Geo proxying @@ -679,7 +678,14 @@ on, check out our [self-service framework](geo/framework.md). ### GET:Geo pipeline -As part of the [e2e:package-and-test](testing_guide/end_to_end/index.md#using-the-package-and-test-job) pipeline, there is an option to manually trigger a job named `GET:Geo`. This pipeline uses [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to spin up a +After triggering a successful [e2e:package-and-test-ee](testing_guide/end_to_end/index.md#using-the-package-and-test-job) pipeline, you can manually trigger a job named `GET:Geo`: + +1. In the [GitLab project](https://gitlab.com/gitlab-org/gitlab), select the **Pipelines** tab of a merge request. +1. Select the `Stage: qa` stage on the latest pipeline to expand and list all the related jobs. +1. Select `trigger-omnibus` to view the [Omnibus GitLab Mirror](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror) pipeline corresponding to the merge request. +1. The `GET:Geo` job can be found and triggered under the `trigger-qa` stage. + +This pipeline uses [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to spin up a [1k](../administration/reference_architectures/1k_users.md) Geo installation, and run the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa) Geo scenario against the instance. When working on Geo features, it is a good idea to ensure the `qa-geo` job passes in a triggered `GET:Geo pipeline`. @@ -694,7 +700,7 @@ see the [QA documentation](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa The pipeline involves the interaction of multiple different projects: -- [GitLab](https://gitlab.com/gitlab-org/gitlab) - The [`e2e:package-and-test` job](testing_guide/end_to_end/index.md#using-the-package-and-test-job) is launched from merge requests in this project. +- [GitLab](https://gitlab.com/gitlab-org/gitlab) - The [`e2e:package-and-test-ee` job](testing_guide/end_to_end/index.md#using-the-package-and-test-job) is launched from merge requests in this project. - [`omnibus-gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab) - Builds relevant artifacts containing the changes from the triggering merge request pipeline. - [GET-Configs/Geo](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/Geo) - Coordinates the lifecycle of a short-lived Geo installation that can be evaluated. - [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) - Contains the necessary logic for creating and destroying Geo installations. Used by `GET-Configs/Geo`. diff --git a/doc/development/git_object_deduplication.md b/doc/development/git_object_deduplication.md index e98ebe5efe1..961bfca0d9b 100644 --- a/doc/development/git_object_deduplication.md +++ b/doc/development/git_object_deduplication.md @@ -56,7 +56,7 @@ identical to) the fork networks that get formed when users fork projects. At the Git level, pool repositories are created and managed using Gitaly -RPC calls. Just like with normal repositories, the authority on which +RPC calls. Just like with typical repositories, the authority on which pool repositories exist, and which repositories borrow from them, lies at the Rails application level in SQL. @@ -144,7 +144,7 @@ are as follows: ### Consequences -- If a normal Project participating in a pool gets moved to another +- If a typical Project participating in a pool gets moved to another Gitaly storage shard, its "belongs to PoolRepository" relation will be broken. Because of the way moving repositories between shard is implemented, we get a fresh self-contained copy diff --git a/doc/development/gitaly.md b/doc/development/gitaly.md index d2232d750b2..e6a853c107e 100644 --- a/doc/development/gitaly.md +++ b/doc/development/gitaly.md @@ -116,7 +116,7 @@ Please raise an issue in the GitLab CE or EE repositories to report the issue. I ~performance ~"technical debt". Please ensure that the issue contains the full stack trace and error message of the `TooManyInvocationsError`. Also include any known failing tests if possible. -Isolate the source of the n+1 problem. This is normally a loop that results in Gitaly being called for each +Isolate the source of the n+1 problem. This is usually a loop that results in Gitaly being called for each element in an array. If you are unable to isolate the problem, please contact a member of the [Gitaly Team](https://gitlab.com/groups/gl-gitaly/group_members) for assistance. @@ -152,7 +152,7 @@ end ## Running tests with a locally modified version of Gitaly -Normally, GitLab CE/EE tests use a local clone of Gitaly in +Usually, GitLab CE/EE tests use a local clone of Gitaly in `tmp/tests/gitaly` pinned at the version specified in `GITALY_SERVER_VERSION`. The `GITALY_SERVER_VERSION` file supports also branches and SHA to use a custom commit in [the repository](https://gitlab.com/gitlab-org/gitaly). @@ -185,7 +185,7 @@ to manually run `make` again. Note that CI tests do not use your locally modified version of Gitaly. To use a custom Gitaly version in CI, you must update -GITALY_SERVER_VERSION as described at the beginning of this section. +`GITALY_SERVER_VERSION` as described at the beginning of this section. To use a different Gitaly repository, such as if your changes are present on a fork, you can specify a `GITALY_REPO_URL` environment variable when diff --git a/doc/development/gitlab_flavored_markdown/specification_guide/index.md b/doc/development/gitlab_flavored_markdown/specification_guide/index.md index ae78daa3687..562fd445ab3 100644 --- a/doc/development/gitlab_flavored_markdown/specification_guide/index.md +++ b/doc/development/gitlab_flavored_markdown/specification_guide/index.md @@ -124,7 +124,7 @@ to report this. #### Official specifications vs internal extensions -Within GFM and GLFM respectively, both GitHub and GitLab have two "sets" of Markdown they support: +GFM for GitHub and GLFM for GitLab have two "sets" of Markdown they support: - Official specification - Internal extensions @@ -133,10 +133,10 @@ The following taxonomy chart shows the taxonomy and terminology of the various s ```mermaid graph TD -CM[CommonMark - spec.txt - e.g. headings] --- GFMS[GFM Specification - spec.txt - e.g. strikethrough extension] -GFMS --- GLFM[GLFM Specification - e.g. color chips] -GFMS --- GFMI[GFM internal extensions - e.g. GitHub-specific references] -GLFM --- GLFS[GLFM internal extensions - e.g. GitLab-specific references] +CM[CommonMark - spec.txt - for example, headings] --- GFMS[GFM Specification - spec.txt - for example, strikethrough extension] +GFMS --- GLFM[GLFM Specification - for example, color chips] +GFMS --- GFMI[GFM internal extensions - for example, GitHub-specific references] +GLFM --- GLFS[GLFM internal extensions - for example, GitLab-specific references] ``` ##### Official specifications @@ -297,9 +297,9 @@ The Markdown dialect used in the GitLab application has a dual requirement for r 1. Rendering to static read-only HTML format, to be displayed in various places throughout the application. 1. Rendering editable content in the - [Content Editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/), + [rich text editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/), a ["What You See Is What You Get" (WYSIWYG)](https://en.wikipedia.org/wiki/WYSIWYG) - editor. The Content Editor supports real-time instant switching between an editable + editor. The rich text editor supports real-time instant switching between an editable Markdown source and an editable WYSIWYG document. These requirements means that GitLab has two independent parser and renderer @@ -312,14 +312,14 @@ implementations: GitHub's fork of the reference parser for CommonMark. `libcmark-gfm` is an extended version of the C reference implementation of [CommonMark](https://commonmark.org/) 1. The frontend parser / renderer supports parsing and _WYSIWYG_ rendering for - the Content Editor. It is implemented in JavaScript. Parsing is based on the + the rich text editor. It is implemented in JavaScript. Parsing is based on the [Remark](https://github.com/remarkjs/remark) Markdown parser, which produces a MDAST Abstract Syntax Tree (MDAST). Rendering is the process of turning an MDAST into a [ProseMirror document](../../fe_guide/content_editor.md). Then, ProseMirror is used to render a ProseMirror document to WYSIWYG HTML. In this document, we refer to the process of turning Markdown into an MDAST as the _frontend / JavaScript parser_, and the entire process of rendering Markdown - to WYSIWYG HTML in ProseMirror as the _Content Editor_. Several + to WYSIWYG HTML in ProseMirror as the _rich text editor_. Several requirements drive the need for an independent frontend parser / renderer implementation, including: 1. Lack of necessary support for accurate source mapping in the HTML renderer @@ -356,7 +356,7 @@ used when running [Markdown snapshot testing](#markdown-snapshot-testing). #### WYSIWYG HTML -**WYSIWYG HTML** is HTML produced by the frontend (JavaScript) Content Editor, +**WYSIWYG HTML** is HTML produced by the frontend (JavaScript) rich text editor, which includes parsing and rendering logic. It is used to present an editable document in the ProseMirror WYSIWYG editor. @@ -472,7 +472,7 @@ generating the HTML for footnote examples. Even though it is in the production c no effect unless it is explicitly set, therefore it is innocuous. It allows us to avoid the more-complex regex-based normalization described below. -The current example of this is when normally random footnote IDs are overridden to be deterministic +The current example of this is when footnote IDs that are usually random are overridden to be deterministic by setting `GITLAB_TEST_FOOTNOTE_ID`. It is set along with the fixtures setup in the [`spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb`](https://gitlab.com/gitlab-org/gitlab/blob/master/spec/support/shared_contexts/glfm/example_snapshot_fixtures.rb) shared context. @@ -554,9 +554,9 @@ specification and testing infrastructure: 1. The frontend (JavaScript) parser and renderer (which includes GitLab custom code and Remark) can convert Markdown to the expected ProseMirror JSON representing a ProseMirror document. - 1. The **Content Editor** (which includes the frontend (JavaScript) parser and renderer, + 1. The **rich text editor** (which includes the frontend (JavaScript) parser and renderer, and ProseMirror) can convert Markdown to the expected custom WYSIWYG HTML as rendered by ProseMirror. - 1. The **Content Editor** can complete a round-trip test, which involves converting + 1. The **rich text editor** can complete a round-trip test, which involves converting from Markdown, to MDAST, to ProseMirror Document, then back to Markdown. It ensures the resulting Markdown is exactly identical, with no differences. @@ -734,10 +734,10 @@ Markdown snapshot testing RSpec and Jest `*_spec` files (from main app `spec` fo which are driven by `example_snapshot` YAML files. The actual RSpec and Jest test `*_spec` files (frontend and backend) live -under the normal relevant locations under `spec`, matching the location of their +under the usual relevant locations under `spec`, matching the location of their corresponding implementations. They can be run either: -- As part of the normal pipelines. +- As part of the standard pipelines. - From the command line or an IDE, just like any other file under `spec`. However, they are spread across four different locations: @@ -1329,7 +1329,7 @@ Three types of entries exist, with different HTML for each: - **WYSIWYG** - The WYSIWYG (frontend, JavaScript-generated) HTML for each entry in `glfm_specification/output_example_snapshots/examples_index.yml`. - - It is generated (or updated) from the frontend Content Editor implementation via the + - It is generated (or updated) from the frontend rich text editor implementation via the `update-example-snapshots.rb` script. It can be manually updated for WYSIWYG examples with incomplete implementations. @@ -1411,5 +1411,5 @@ This section describes how the scripts can be used to manage the GLFM specificat 1. Visually inspect and confirm any resulting changes to the [example snapshot files](#output-example-snapshot-files). 1. Run [`run-snapshot-tests.sh`](#run-snapshot-testssh-script) as a convenience script to run all relevant frontend (RSpec) and backend (Jest) tests which use the example snapshots. 1. Any frontend or backend snapshot test may also be run individually. - 1. All frontend and backend tests are also run as part of the continuous integration suite, as they normally are. + 1. All frontend and backend tests are also run as part of the continuous integration suite, as they typically are. 1. Commit any changes to the [input specification files](#input-specification-files), [output specification files](#output-specification-files), or [example snapshot files](#output-example-snapshot-files). diff --git a/doc/development/graphql_guide/batchloader.md b/doc/development/graphql_guide/batchloader.md index 6a716de61b8..5f06f1faf1e 100644 --- a/doc/development/graphql_guide/batchloader.md +++ b/doc/development/graphql_guide/batchloader.md @@ -41,8 +41,8 @@ class UserResolver < BaseResolver end ``` -- `project_id` is the `ID` of the current project being queried -- `loader.call` is used to map the result back to the input key (here a project ID) +- `username` is the username we want to query. It can be one name or multiple names. +- `loader.call` is used to map the result back to the input key (here user is mapped to its username) - `BatchLoader::GraphQL` returns a lazy object (suspended promise to fetch the data) Here an [example MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46549) illustrating how to use our `BatchLoading` mechanism. @@ -161,7 +161,7 @@ you do so, you do not need to manage the lifecycle of lazy values yourself, and you are assured accurate results. GraphQL fields that return lazy values may need these values forced in tests. -Forcing refers to explicit demands for evaluation, where this would normally +Forcing refers to explicit demands for evaluation, where this would usually be arranged by the framework. You can force a lazy value with the `GraphqlHelpers#batch_sync` method available in [GraphQLHelpers](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/helpers/graphql_helpers.rb), or by using `Gitlab::Graphql::Lazy.force`. For example: diff --git a/doc/development/import_export.md b/doc/development/import_export.md index b8493ef7a6e..9a7944ae308 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -46,8 +46,9 @@ The `AttributeConfigurationSpec` checks and confirms the addition of new columns <<-MSG It looks like #{relation_class}, which is exported using the project Import/Export, has new attributes: - Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if you consider this can be exported. - Otherwise, please blacklist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its correspondent + Please add the attribute(s) to SAFE_MODEL_ATTRIBUTES if they can be exported. + + Please denylist the attribute(s) in IMPORT_EXPORT_CONFIG by adding it to its corresponding model in the +excluded_attributes+ section. SAFE_MODEL_ATTRIBUTES: #{File.expand_path(safe_attributes_file)} diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md index b51a2799088..bd672c86b28 100644 --- a/doc/development/integrations/index.md +++ b/doc/development/integrations/index.md @@ -310,7 +310,7 @@ see the [feature deprecation guidelines](../../development/deprecation_guideline You must announce any deprecation [no later than the third milestone preceding intended removal](../../development/deprecation_guidelines/index.md#when-can-a-feature-be-deprecated). To deprecate an integration: -- [Add a deprecation entry](../../development/deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation-pages). +- [Add a deprecation entry](../../development/deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation). - [Mark the integration documentation as deprecated](../../development/documentation/versions.md#deprecate-a-page-or-topic). - Optional. To prevent any new project-level records from being created, add the integration to `Project#disabled_integrations` (see [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114835)). @@ -324,7 +324,6 @@ In the major milestone of intended removal (M.0), disable the integration and de - Remove the integration from `Integration::INTEGRATION_NAMES`. - Delete the integration model's `#execute` and `#test` methods (if defined), but keep the model. - Add a post-migration to delete the integration records from PostgreSQL (see [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114721)). -- [Add a removal entry](../../development/deprecation_guidelines/index.md#update-the-deprecations-and-removals-documentation-pages). - [Mark the integration documentation as removed](../../development/documentation/versions.md#remove-a-page). - [Update the integration API documentation](../../api/integrations.md). diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md index ee94e57a247..09778127050 100644 --- a/doc/development/integrations/secure.md +++ b/doc/development/integrations/secure.md @@ -577,8 +577,8 @@ All other attributes are optional. ##### SAST -The `location` of a SAST vulnerability must have a `file` and a `start_line` field, -giving the path of the affected file, and the affected line number, respectively. +The `location` of a SAST vulnerability must have a `file` that gives the path of the affected file and +a `start_line` field with the affected line number. It may also have an `end_line`, a `class`, and a `method`. For instance, here is the `location` object for a security flaw found @@ -626,19 +626,14 @@ This is addressed in [issue #7586](https://gitlab.com/gitlab-org/gitlab/-/issues See also [deduplication process](../../user/application_security/vulnerability_report/pipeline.md#deduplication-process). -##### Severity and confidence +##### Severity -The `severity` field describes how much the vulnerability impacts the software, -whereas the `confidence` field describes how reliable the assessment of the vulnerability is. +The `severity` field describes how badly the vulnerability impacts the software. The severity is used to sort the vulnerabilities in the security dashboard. The severity ranges from `Info` to `Critical`, but it can also be `Unknown`. Valid values are: `Unknown`, `Info`, `Low`, `Medium`, `High`, or `Critical` -The confidence ranges from `Low` to `Confirmed`, but it can also be `Unknown`, -`Experimental` or even `Ignore` if the vulnerability is to be ignored. -Valid values are: `Ignore`, `Unknown`, `Experimental`, `Low`, `Medium`, `High`, or `Confirmed` - `Unknown` values means that data is unavailable to determine it's actual value. Therefore, it may be `high`, `medium`, or `low`, and needs to be investigated. We have [provided a chart](../../user/application_security/sast/analyzers.md#data-provided-by-analyzers) of the available SAST Analyzers and what data is currently available. diff --git a/doc/development/internal_analytics/internal_events/index.md b/doc/development/internal_analytics/internal_events/index.md new file mode 100644 index 00000000000..cbc7b73a282 --- /dev/null +++ b/doc/development/internal_analytics/internal_events/index.md @@ -0,0 +1,98 @@ +--- +stage: Analytics +group: Analytics Instrumentation +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Internal Events development guidelines + +In an effort to provide a more efficient, scalable, and unified tracking API, GitLab is deprecating existing RedisHLL and Snowplow tracking. Instead, we're implementing a new `track_event` method. +With this approach, we can both update RedisHLL counters and send Snowplow events simultaneously, streamlining the tracking process. + +## Create and trigger events + +### Backend tracking + +To trigger an event, call the `Gitlab::InternalEvents.track_event` method with the desired arguments: + +```ruby +Gitlab::InternalEvents.track_event( + "i_code_review_user_apply_suggestion", + user_id: user_id, + namespace_id: namespace_id, + project_id: project_id + ) +``` + +This method automatically increments all RedisHLL metrics relating to the event `i_code_review_user_apply_suggestion`, and sends a corresponding Snowplow event with all named arguments and standard context (SaaS only). + +### Frontend tracking + +#### Vue components + +In Vue components, tracking can be done with [Vue mixin](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/tracking/internal_events.js#L29). + +To implement Vue component tracking: + +1. Import the `InternalEvents` library and call the `mixin` method: + + ```javascript + import { InternalEvents } from '~/tracking'; + const trackingMixin = InternalEvents.mixin(); + ``` + +1. Use the mixin in the component: + + ```javascript + export default { + mixins: [trackingMixin], + + data() { + return { + expanded: false, + }; + }, + }; + ``` + +1. Call the `track_event` method. Tracking options can be passed as the second parameter: + + ```javascript + this.track_event('i_code_review_user_apply_suggestion'); + ``` + + Or use the `track_event` method in the template: + + ```html + <template> + <div> + <button data-testid="toggle" @click="toggle">Toggle</button> + + <div v-if="expanded"> + <p>Hello world!</p> + <button @click="track_event('i_code_review_user_apply_suggestion')">Track another event</button> + </div> + </div> + </template> + ``` + +#### Raw JavaScript + +For tracking events directly from arbitrary frontend JavaScript code, a module for raw JavaScript is provided. This can be used outside of a component context where the Mixin cannot be utilized. + +```javascript +import { InternalEvents } from '~/tracking'; +InternalEvents.track_event('i_code_review_user_apply_suggestion'); +``` + +#### Data-track attribute + +This attribute ensures that if we want to track GitLab internal events for a button, we do not need to write JavaScript code on Click handler. Instead, we can just add a data-event-tracking attribute with event value and it should work. This can also be used with haml views. + +```html + <gl-button + data-event-tracking="i_analytics_dev_ops_adoption" + > + Click Me + </gl-button> +``` diff --git a/doc/development/internal_analytics/service_ping/implement.md b/doc/development/internal_analytics/service_ping/implement.md index 0dfc3806712..3b8243377a3 100644 --- a/doc/development/internal_analytics/service_ping/implement.md +++ b/doc/development/internal_analytics/service_ping/implement.md @@ -501,7 +501,7 @@ We can disable tracking completely by using the global flag: ##### Known events are added automatically in Service Data payload -Service Ping adds all events [`known_events/*.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events) to Service Data generation under the `redis_hll_counters` key. This column is stored in [version-app as a JSON](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L209). +Service Ping adds all events [`known_events/*.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events) to Service Data generation under the `redis_hll_counters` key. This column is stored in [version-app as a JSON](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/db/schema.rb#L213). For each event we add metrics for the weekly and monthly time frames, and totals for each where applicable: - `#{event_name}_weekly`: Data for 7 days for daily [aggregation](#add-new-events) events and data for the last complete week for weekly [aggregation](#add-new-events) events. @@ -680,7 +680,7 @@ See the [Metrics Dictionary guide](metrics_dictionary.md) for more information. ## Add the metric to the Versions Application -Check if the new metric must be added to the Versions Application. See the `usage_data` [schema](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/db/schema.rb#L147) and Service Data [parameters accepted](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/usage_ping.rb). Any metrics added under the `counts` key are saved in the `stats` column. +Check if the new metric must be added to the Versions Application. See the `usage_data` [schema](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/db/schema.rb#L152) and Service Data [parameters accepted](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/app/services/usage_ping.rb). Any metrics added under the `counts` key are saved in the `stats` column. ## Create a merge request @@ -715,7 +715,7 @@ To set up Service Ping locally, you must: ### Set up local repositories 1. Clone and start [GitLab](https://gitlab.com/gitlab-org/gitlab-development-kit). -1. Clone and start [Versions Application](https://gitlab.com/gitlab-services/version-gitlab-com). +1. Clone and start [Versions Application](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com). Make sure you run `docker-compose up` to start a PostgreSQL and Redis instance. 1. Point GitLab to the Versions Application endpoint instead of the default endpoint: 1. Open [service_ping/submit_service.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L5) locally and modify `STAGING_BASE_URL`. @@ -783,7 +783,7 @@ By default, it comes with a fully configured Prometheus service that is set up t However, it has the following limitations: - It does not run a `gitlab-exporter` instance, so several `process_*` metrics from services such as Gitaly may be missing. -- While it runs a `node_exporter`, `docker-compose` services emulate hosts, meaning that it normally reports itself as not associated +- While it runs a `node_exporter`, `docker-compose` services emulate hosts, meaning that it usually reports itself as not associated with any of the other running services. That is not how node metrics are reported in a production setup, where `node_exporter` always runs as a process alongside other GitLab components on any given node. For Service Ping, none of the node data would therefore appear to be associated to any of the services running, because they all appear to be running on different hosts. To alleviate this problem, the `node_exporter` in GCK was arbitrarily "assigned" to the `web` service, meaning only for this service `node_*` metrics appears in Service Ping. diff --git a/doc/development/internal_analytics/service_ping/index.md b/doc/development/internal_analytics/service_ping/index.md index 69d37f0dae2..22e66a247c9 100644 --- a/doc/development/internal_analytics/service_ping/index.md +++ b/doc/development/internal_analytics/service_ping/index.md @@ -22,7 +22,7 @@ and sales teams understand how GitLab is used. The data helps to: Service Ping information is not anonymous. It's linked to the instance's hostname, but does not contain project names, usernames, or any other specific data. -Service Ping is enabled by default. However, you can [disable](../../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) it on any self-managed instance. When Service Ping is enabled, GitLab gathers data from the other instances and can show your instance's usage statistics to your users. +Service Ping is enabled by default. However, you can [disable](../../../administration/settings/usage_statistics.md#enable-or-disable-usage-statistics) it on any self-managed instance. When Service Ping is enabled, GitLab gathers data from the other instances and can show your instance's usage statistics to your users. ## Service Ping terminology diff --git a/doc/development/internal_analytics/service_ping/metrics_dictionary.md b/doc/development/internal_analytics/service_ping/metrics_dictionary.md index d1f1c0b595a..f56955ed422 100644 --- a/doc/development/internal_analytics/service_ping/metrics_dictionary.md +++ b/doc/development/internal_analytics/service_ping/metrics_dictionary.md @@ -40,7 +40,7 @@ Each metric is defined in a separate YAML file consisting of a number of fields: | `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html). | | `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | | `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | -| `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`. | +| `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`, `internal_events`. | | `data_category` | yes | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.| | `instrumentation_class` | yes | `string`; [the class that implements the metric](metrics_instrumentation.md). | | `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. | @@ -135,7 +135,7 @@ For more information about the aggregation type of each feature, see the [`commo | data_source | time_frame | aggregation | Description | |------------------------|------------|----------------|-------------------------------------------------| -| any | `none` | not applicable | A type of data that’s not tracked over time, such as settings and configuration information | +| any | `none` | not applicable | A type of data that's not tracked over time, such as settings and configuration information | | `database` | `all` | not applicable | The whole time the metric has been active (all-time interval) | | `database` | `7d` | not applicable | 9 days ago to 2 days ago | | `database` | `28d` | not applicable | 30 days ago to 2 days ago | @@ -150,7 +150,7 @@ For more information about the aggregation type of each feature, see the [`commo We use the following categories to classify a metric: - `operational`: Required data for operational purposes. -- `optional`: Default value for a metric. Data that is optional to collect. This can be [enabled or disabled](../../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics) in the Admin Area. +- `optional`: Default value for a metric. Data that is optional to collect. This can be [enabled or disabled](../../../administration/settings/usage_statistics.md#enable-or-disable-usage-statistics) in the Admin Area. - `subscription`: Data related to licensing. - `standard`: Standard set of identifiers that are included when collecting data. diff --git a/doc/development/internal_analytics/service_ping/metrics_instrumentation.md b/doc/development/internal_analytics/service_ping/metrics_instrumentation.md index b6ca773a572..dc225a40d1b 100644 --- a/doc/development/internal_analytics/service_ping/metrics_instrumentation.md +++ b/doc/development/internal_analytics/service_ping/metrics_instrumentation.md @@ -403,6 +403,36 @@ module Gitlab end ``` +## Prometheus metrics + +This instrumentation class lets you handle Prometheus queries by passing a Prometheus client object as an argument to the `value` block. +Any Prometheus error handling should be done in the block itself. + +- `value`: Specifies the value of the metric. A Prometheus client object is passed as the first argument. +- `available?`: Specifies whether the metric should be reported. The default is `true`. + +[Example of a merge request that adds a Prometheus metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/122400). + +```ruby +module Gitlab + module Usage + module Metrics + module Instrumentations + class GitalyApdexMetric < PrometheusMetric + value do |client| + result = client.query('avg_over_time(gitlab_usage_ping:gitaly_apdex:ratio_avg_over_time_5m[1w])').first + + break FALLBACK unless result + + result['value'].last.to_f + end + end + end + end + end +end +``` + ## Support for instrumentation classes There is support for: diff --git a/doc/development/internal_analytics/service_ping/metrics_lifecycle.md b/doc/development/internal_analytics/service_ping/metrics_lifecycle.md index cc56863690c..7aa0eaa1554 100644 --- a/doc/development/internal_analytics/service_ping/metrics_lifecycle.md +++ b/doc/development/internal_analytics/service_ping/metrics_lifecycle.md @@ -51,16 +51,16 @@ To remove a metric: to inform administrators of the progress of DevOps adoption for the instance. You can check - [`CalculateConvIndexService`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/app/services/calculate_conv_index_service.rb) + [`CalculateConvIndexService`](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/app/services/calculate_conv_index_service.rb) to view the metrics that are used. The metrics are represented as the keys that are passed as a field argument into the `get_value` method. 1. Verify that removing the metric from the Service Ping payload does not cause - errors in [Version App](https://gitlab.com/gitlab-services/version-gitlab-com) + errors in [Version App](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com) when the updated payload is collected and processed. Version App collects and persists all Service Ping reports. To verify Service Ping processing in your local development environment, follow this [guide](https://www.youtube.com/watch?v=FS5emplabRU). - Alternatively, you can modify [fixtures](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/master/spec/support/usage_data_helpers.rb#L540) - used to test the [`UsageDataController#create`](https://gitlab.com/gitlab-services/version-gitlab-com/-/blob/3760ef28/spec/controllers/usage_data_controller_spec.rb#L75) + Alternatively, you can modify [fixtures](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/spec/support/usage_data_helpers.rb) + used to test the [`UsageDataController#create`](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/blob/main/spec/controllers/usage_data_controller_spec.rb) endpoint, and assure that test suite does not fail when metric that you wish to remove is not included into test payload. 1. Remove data from Redis diff --git a/doc/development/internal_analytics/service_ping/troubleshooting.md b/doc/development/internal_analytics/service_ping/troubleshooting.md index 2b285b85bd0..7f7b4099d48 100644 --- a/doc/development/internal_analytics/service_ping/troubleshooting.md +++ b/doc/development/internal_analytics/service_ping/troubleshooting.md @@ -22,11 +22,13 @@ You can use [this query](https://gitlab.com/gitlab-org/gitlab/-/issues/347298#no ### Troubleshoot the GitLab application layer -For results about an investigation conducted into an unexpected drop in Service ping Payload events volume, see [this issue](https://gitlab.com/gitlab-data/analytics/-/issues/11071). +We conducted an investigation into an unexpected drop in Service ping Payload events volume. +GitLab team members can view more information in this confidential issue: +`https://gitlab.com/gitlab-data/analytics/-/issues/11071` ### Troubleshoot VersionApp layer -Check if the [export jobs](https://gitlab.com/gitlab-services/version-gitlab-com#data-export-using-pipeline-schedules) are successful. +Check if the [export jobs](https://gitlab.com/gitlab-org/gitlab-services/version.gitlab.com/-/tree/main/#data-export-using-pipeline-schedules) are successful. Check [Service Ping errors](https://app.periscopedata.com/app/gitlab/968489?widget=14609989&udv=0) in the [Service Ping Health Dashboard](https://app.periscopedata.com/app/gitlab/968489). diff --git a/doc/development/internal_analytics/snowplow/infrastructure.md b/doc/development/internal_analytics/snowplow/infrastructure.md index 9679abac6b7..462dee2c39b 100644 --- a/doc/development/internal_analytics/snowplow/infrastructure.md +++ b/doc/development/internal_analytics/snowplow/infrastructure.md @@ -55,7 +55,7 @@ In contrast to a typical Snowplow pipeline, after enrichment, GitLab Snowplow ev #### Why events need to be pseudonymized GitLab is bound by its [obligations to community](https://about.gitlab.com/handbook/product/analytics-instrumentation-guide/service-usage-data-commitment/) -and by [legal regulations](https://about.gitlab.com/handbook/legal/privacy/services-usage-data/) to protect the privacy of its users. +and by [legal regulations](https://about.gitlab.com/handbook/legal/privacy/customer-product-usage-information/) to protect the privacy of its users. GitLab must provide valuable insights for business decisions, and there is a need for a better understanding of different users' behavior patterns. The diff --git a/doc/development/internal_analytics/snowplow/troubleshooting.md b/doc/development/internal_analytics/snowplow/troubleshooting.md index 885f4e0c16f..2f59543e0f4 100644 --- a/doc/development/internal_analytics/snowplow/troubleshooting.md +++ b/doc/development/internal_analytics/snowplow/troubleshooting.md @@ -31,7 +31,7 @@ While on CloudWatch dashboard set time range to last 4 weeks, to get better pict ### Troubleshooting GitLab application layer -Drop occurring at application layer can be symptom of some issue, but it might be also a result of normal application lifecycle, intended changes done to analytics instrumentation or experiments tracking +Drop occurring at application layer can be symptom of some issue, but it might be also a result of typical application lifecycle, intended changes done to analytics instrumentation or experiments tracking or even a result of a public holiday in some regions of the world with a larger user-base. To verify if there is an underlying problem to solve, you can check following things: 1. Check `about.gitlab.com` website traffic on [Google Analytics](https://analytics.google.com/analytics/web/) to verify if some public holiday might impact overall use of GitLab system @@ -42,7 +42,9 @@ or even a result of a public holiday in some regions of the world with a larger 1. Check (via [Grafana explore tab](https://dashboards.gitlab.net/explore) ) following Prometheus counters `gitlab_snowplow_events_total`, `gitlab_snowplow_failed_events_total` and `gitlab_snowplow_successful_events_total` to see how many events were fired correctly from GitLab.com. Example query to use `sum(rate(gitlab_snowplow_successful_events_total{env="gprd"}[5m])) / sum(rate(gitlab_snowplow_events_total{env="gprd"}[5m]))` would chart rate at which number of good events rose in comparison to events sent in total. If number drops from 1 it means that problem might be in communication between GitLab and AWS collectors fleet. 1. Check [logs in Kibana](https://log.gprd.gitlab.net/app/discover#) and filter with `{ "query": { "match_phrase": { "json.message": "failed to be reported to collector at" } } }` if there are some failed events logged -For results about an investigation conducted into an unexpected drop in snowplow events volume, see [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/335206). +We conducted an investigation into an unexpected drop in snowplow events volume. + +GitLab team members can view more information in this confidential issue: `https://gitlab.com/gitlab-org/gitlab/-/issues/335206` ### Troubleshooting AWS layer diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md index 5fceb9013da..538b66124ba 100644 --- a/doc/development/internal_api/index.md +++ b/doc/development/internal_api/index.md @@ -492,14 +492,21 @@ curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \ Called from GitLab agent server (`kas`) to increase the usage metric counters. -| Attribute | Type | Required | Description | -|:-------------------------------------------------|:--------------|:---------|:---------------------------------------------------------------------------------------------------------------------| -| `counters` | hash | no | Hash of counters | -| `counters["k8s_api_proxy_request"]` | integer | no | The number to increase the `k8s_api_proxy_request` counter by | -| `counters["gitops_sync"]` | integer | no | The number to increase the `gitops_sync` counter by | -| `counters["flux_git_push_notifications_total"]` | integer | no | The number to increase the `flux_git_push_notifications_total` counter by | -| `unique_counters` | hash | no | Array of unique numbers | -| `unique_counters["agent_users_using_ci_tunnel"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel to track the `agent_users_using_ci_tunnel` metric event | +| Attribute | Type | Required | Description | +|:--------------------------------------------------------------------------|:--------------|:---------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `counters` | hash | no | Hash of counters | +| `counters["k8s_api_proxy_request"]` | integer | no | The number to increase the `k8s_api_proxy_request` counter by | +| `counters["gitops_sync"]` | integer | no | The number to increase the `gitops_sync` counter by | +| `counters["flux_git_push_notifications_total"]` | integer | no | The number to increase the `flux_git_push_notifications_total` counter by | +| `counters["k8s_api_proxy_requests_via_ci_access"]` | integer | no | The number to increase the `k8s_api_proxy_requests_via_ci_access` counter by | +| `counters["k8s_api_proxy_requests_via_user_access"]` | integer | no | The number to increase the `k8s_api_proxy_requests_via_user_access` counter by | +| `unique_counters` | hash | no | Array of unique numbers | +| `unique_counters["agent_users_using_ci_tunnel"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel to track the `agent_users_using_ci_tunnel` metric event | +| `unique_counters["k8s_api_proxy_requests_unique_users_via_ci_access"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel via `ci_access` to track the `k8s_api_proxy_requests_unique_users_via_ci_access` metric event | +| `unique_counters["k8s_api_proxy_requests_unique_agents_via_ci_access"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel via `ci_access` to track the `k8s_api_proxy_requests_unique_agents_via_ci_access` metric event | +| `unique_counters["k8s_api_proxy_requests_unique_users_via_user_access"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel via `user_access` to track the `k8s_api_proxy_requests_unique_users_via_user_access` metric event | +| `unique_counters["k8s_api_proxy_requests_unique_agents_via_user_access"]` | integer array | no | The set of unique user ids that have interacted a CI Tunnel via `user_access` to track the `k8s_api_proxy_requests_unique_agents_via_user_access` metric event | +| `unique_counters["flux_git_push_notified_unique_projects"]` | integer array | no | The set of unique projects ids that have been notified to reconcile their Flux workloads to track the `flux_git_push_notified_unique_projects` metric event | ```plaintext POST /internal/kubernetes/usage_metrics @@ -857,9 +864,9 @@ PUT /namespaces/:id/subscription_add_on_purchase/:add_on_name | Attribute | Type | Required | Description | |:------------|:--------|:---------|:------------| -| `quantity` | integer | yes | Amount of units in the subscription add-on purchase (Example: Number of seats for a code suggestions add-on) | +| `quantity` | integer | no | Amount of units in the subscription add-on purchase (Example: Number of seats for a code suggestions add-on) | | `expires_on` | date | yes | Expiration date of the subscription add-on purchase | -| `purchase_xid` | string | yes | Identifier for the subscription add-on purchase (Example: Subscription name for a code suggestions add-on) | +| `purchase_xid` | string | no | Identifier for the subscription add-on purchase (Example: Subscription name for a code suggestions add-on) | Example request: @@ -1014,10 +1021,10 @@ Example response: ## Compute quota provisioning -> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "units of compute" in GitLab 16.1. +> [Renamed](https://gitlab.com/groups/gitlab-com/-/epics/2150) from "CI/CD minutes" to "compute quota" and "compute minutes" in GitLab 16.1. The compute quota endpoints are used by [CustomersDot](https://gitlab.com/gitlab-org/customers-gitlab-com) (`customers.gitlab.com`) -to apply additional packs of units of compute, for personal namespaces or top-level groups in GitLab.com. +to apply additional packs of compute minutes, for personal namespaces or top-level groups in GitLab.com. ### Create an additional pack @@ -1031,7 +1038,7 @@ POST /namespaces/:id/minutes |:------------|:--------|:---------|:------------| | `packs` | array | yes | An array of purchased compute packs | | `packs[expires_at]` | date | yes | Expiry date of the purchased pack| -| `packs[number_of_minutes]` | integer | yes | Number of additional units of compute | +| `packs[number_of_minutes]` | integer | yes | Number of additional compute minutes | | `packs[purchase_xid]` | string | yes | The unique ID of the purchase | Example request: diff --git a/doc/development/internal_api/internal_api_allowed.md b/doc/development/internal_api/internal_api_allowed.md index 89c76c88df4..d5e4e8cf63d 100644 --- a/doc/development/internal_api/internal_api_allowed.md +++ b/doc/development/internal_api/internal_api_allowed.md @@ -91,7 +91,7 @@ same manner as the standard repositories, and is more prone to the refs issue. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `parallel_push_checks`. +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `parallel_push_checks`. On GitLab.com, by default this feature is not available. To make it available per project, ask GitLab.com administrator to [enable the feature flag](../../administration/feature_flags.md) named `parallel_push_checks`. diff --git a/doc/development/internal_users.md b/doc/development/internal_users.md index ce13324507d..d9f40d6f5d4 100644 --- a/doc/development/internal_users.md +++ b/doc/development/internal_users.md @@ -40,6 +40,7 @@ For this bot: Other examples of internal users: +- [GitLab Admin Bot](https://gitlab.com/gitlab-org/gitlab/-/blob/278bc9018dd1515a10cbf15b6c6cd55cb5431407/app/models/user.rb#L950-960) - [Alert Bot](../operations/incident_management/alerts.md#trigger-actions-from-alerts) - [Ghost User](../user/profile/account/delete_account.md#associated-records) - [Support Bot](../user/project/service_desk.md#support-bot-user) diff --git a/doc/development/jh_features_review.md b/doc/development/jh_features_review.md index 7e90d79c067..f4ed7070948 100644 --- a/doc/development/jh_features_review.md +++ b/doc/development/jh_features_review.md @@ -99,6 +99,20 @@ the relevant EE and JH modules by the name of the receiver module. If reviewing the corresponding JH file is needed, it should be found at [JH repository](https://jihulab.com/gitlab-cn/gitlab). +NOTE: +In some cases, JH does need to override something we don't need, and in that +case it is ok to also add `prepend_mod` for the modules. When we do this, +also add a comment mentioning it, and a link to the JH module using it. +This way we know where it's used and when we might not need it anymore, +and we do not remove them only because we're not using it, accidentally +breaking JH. An example of this: + +```ruby +# Added for JiHu +# Used in https://jihulab.com/gitlab-cn/gitlab/-/blob/main-jh/jh/lib/jh/api/integrations.rb +API::Integrations.prepend_mod +``` + ### General guidance for writing JH extensions See [Guidelines for implementing Enterprise Edition features](ee_features.md) diff --git a/doc/development/merge_request_concepts/diffs/development.md b/doc/development/merge_request_concepts/diffs/development.md index 37d87d4e4eb..75f961e41de 100644 --- a/doc/development/merge_request_concepts/diffs/development.md +++ b/doc/development/merge_request_concepts/diffs/development.md @@ -178,7 +178,7 @@ of options for access and working with diffs, focusing solely on the most common ### `batch_diffs.json` The most common avenue for viewing diffs is the **Changes** -tab in the top navigation bar of merge request pages in the GitLab UI. When selected, the +tab at the top of merge request pages in the GitLab UI. When selected, the diffs themselves are loaded via a paginated request to `/-/merge_requests/:id/batch_diffs.json`, which is served by [`Projects::MergeRequests::DiffsController#diffs_batch`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/merge_requests/diffs_controller.rb): diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 33f51c20446..8cbf18ce9f2 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -138,7 +138,7 @@ This will generate: Changes to the schema should be committed to `db/structure.sql`. This file is automatically generated by Rails when you run -`bundle exec rails db:migrate`, so you normally should not +`bundle exec rails db:migrate`, so you typically should not edit this file by hand. If your migration is adding a column to a table, that column is added at the bottom. Please do not reorder columns manually for existing tables as this causes confusion to @@ -203,6 +203,11 @@ and explains how to perform them without requiring downtime. Your migration **must be** reversible. This is very important, as it should be possible to downgrade in case of a vulnerability or bugs. +**Note**: On GitLab production environments, if a problem occurs, a roll-forward strategy is used instead of rolling back migrations using `db:rollback`. +On self-managed instances we advise users to restore the backup which was created before the upgrade process started. +The `down` method is used primarily in the development environment, for example, when a developer wants to ensure +their local copy of `structure.sql` file and database are in a consistent state when switching between commits or branches. + In your migration, add a comment describing how the reversibility of the migration was tested. @@ -224,13 +229,13 @@ end Migrations like this are inherently risky and [additional actions](database_review.md#preparation-when-adding-data-migrations) are required when preparing the migration for review. -## Atomicity +## Atomicity and transaction -By default, migrations are single transaction. That is, a transaction is opened +By default, migrations are a single transaction: it's opened at the beginning of the migration, and committed after all steps are processed. Running migrations in a single transaction makes sure that if one of the steps fails, -none of the steps are executed, leaving the database in valid state. +none of the steps are executed, leaving the database in a valid state. Therefore, either: - Put all migrations in one single-transaction migration. @@ -238,11 +243,141 @@ Therefore, either: for the steps that cannot be done in a single transaction. For example, if you create an empty table and need to build an index for it, -it is recommended to use a regular single-transaction migration and the default +you should use a regular single-transaction migration and the default rails schema statement: [`add_index`](https://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/SchemaStatements.html#method-i-add_index). -This is a blocking operation, but it doesn't cause problems because the table is not yet used, +This operation is a blocking operation, but it doesn't cause problems because the table is not yet used, and therefore it does not have any records yet. +NOTE: +Subtransactions are [disallowed](https://about.gitlab.com/blog/2021/09/29/why-we-spent-the-last-month-eliminating-postgresql-subtransactions/) in general. +Use multiple, separate transactions +if needed as described in [Heavy operations in a single transaction](#heavy-operations-in-a-single-transaction). + +### Heavy operations in a single transaction + +When using a single-transaction migration, a transaction holds a database connection +for the duration of the migration, so you must make sure the actions in the migration +do not take too much time. +In general, transactions must [execute quickly](database/transaction_guidelines.md#transaction-speed). +To that end, observe [the maximum query time limit](database/query_performance.md#timing-guidelines-for-queries) +for each query run in the migration. + +If your single-transaction migration takes long to finish, you have several options. +In all cases, remember to select the appropriate migration type +depending on [how long a migration takes](#how-long-a-migration-should-take) + +- Split the migration into **multiple single-transaction migrations**. + +- Use **multiple transactions** by [using `disable_ddl_transaction!`](#disable-transaction-wrapped-migration). + +- Keep using a single-transaction migration after **adjusting statement and lock timeout settings**. + If your heavy workload must use the guarantees of a transaction, + you should check your migration can execute without hitting the timeout limits. + The same advice applies to both single-transaction migrations and individual transactions. + + - Statement timeout: the statement timeout is configured to be `15s` for GitLab.com's production database + but creating an index often takes more than 15 seconds. + When you use the existing helpers including `add_concurrent_index`, + they automatically turn off the statement timeout as needed. + In rare cases, you might need to set the timeout limit yourself by [using `disable_statement_timeout`](#temporarily-turn-off-the-statement-timeout-limit). + - Lock timeout: if your migration must execute as a transaction but can possibly time out while + acquiring a lock, [use `enable_lock_retries!`](#usage-with-transactional-migrations). + +NOTE: +To run migrations, we directly connect to the primary database, bypassing PgBouncer +to control settings like `statement_timeout` and `lock_wait_timeout`. + +#### Temporarily turn off the statement timeout limit + +The migration helper `disable_statement_timeout` enables you to +temporarily set the statement timeout to `0` per transaction or per connection. + +- You use the per-connection option when your statement does not support + running inside an explicit transaction, like `CREATE INDEX CONCURRENTLY`. + +- If your statement does support an explicit transaction block, + like `ALTER TABLE ... VALIDATE CONSTRAINT`, + the per-transaction option should be used. + +Using `disable_statement_timeout` is rarely needed, because +the most migration helpers already use them internally when needed. +For example, creating an index usually takes more than 15 seconds, +which is the default statement timeout configured for GitLab.com's production database. +The helper `add_concurrent_index` creates an index inside the block +passed to `disable_statement_timeout` to disable the statement timeout per connection. + +If you are writing raw SQL statements in a migration, +you may need to manually use `disable_statement_timeout`. +Consult the database reviewers and maintainers when you do. + +### Disable transaction-wrapped migration + +You can opt out of running your migration as a single transaction by using +`disable_ddl_transaction!`, an ActiveRecord method. +The method might be called in other database systems, with different results. +At GitLab we exclusively use PostgreSQL. +You should always read `disable_ddl_transaction!` as meaning: + +"Do not execute this migration in a single PostgreSQL transaction. I'll open PostgreSQL transaction(s) only _when_ and _if_ I need them." + +NOTE: +Even if you don't use an explicit PostgreSQL transaction `.transaction` (or `BEGIN; COMMIT;`), +every SQL statement is still executed as a transaction. +See [the PostgreSQL documentation on transactions](https://www.postgresql.org/docs/current/tutorial-transactions.html). + +NOTE: +In GitLab, we've sometimes referred to +the migrations that used `disable_ddl_transaction!` as non-transactional migrations. +It just meant the migrations were not executed as _single_ transactions. + +When should you use `disable_ddl_transaction!`? In most cases, +the existing RuboCop rules or migration helpers can detect if you should be +using `disable_ddl_transaction!`. +Skip `disable_ddl_transaction!` if you are unsure whether to use it or not in your migration, +and let the RuboCop rules and database reviews guide you. + +Use `disable_ddl_transaction!` when PostgreSQL requires an operation to be executed outside an explicit transaction. + +- The most prominent example of such operation is the command `CREATE INDEX CONCURRENTLY`. + PostgreSQL allows the blocking version (`CREATE INDEX`) to be run inside a transaction. + Unlike `CREATE INDEX`, `CREATE INDEX CONCURRENTLY` must be performed outside a transaction. + Therefore, even though a migration may run just one statement `CREATE INDEX CONCURRENTLY`, + you should disable `disable_ddl_transaction!`. + It's also the reason why the use of the helper `add_concurrent_index` requires `disable_ddl_transaction!` + `CREATE INDEX CONCURRENTLY` is more of the exception than the rule. + +Use `disable_ddl_transaction!` when you need to run multiple transactions in a migration for any reason. +Most of the time you would be using multiple transactions to avoid [running one slow transaction](#heavy-operations-in-a-single-transaction). + +- For example, when you insert, update, or delete (DML) a large amount of data, + you should [perform them in batches](database/iterating_tables_in_batches.md#eachbatch-in-data-migrations). + Should you need to group operations for each batch, + you can explicitly open a transaction block when processing a batch. + Consider using a [batched background migration](database/batched_background_migrations.md) for + any reasonably large workload. + +Use `disable_ddl_transaction!` when migration helpers require them. +Various migration helpers need to run with `disable_ddl_transaction!` +because they require a precise control on when and how to open transactions. + +- A foreign key _can_ be added inside a transaction, unlike `CREATE INDEX CONCURRENTLY`. + However, PostgreSQL does not provide an option similar to `CREATE INDEX CONCURRENTLY`. + The helper [`add_concurrent_foreign_key`](database/foreign_keys.md#adding-foreign-keys-in-migrations) + instead opens its own transactions to lock the source and target table + in a manner that minimizes locking while adding and validating the foreign key. +- As advised earlier, skip `disable_ddl_transaction!` if you are unsure + and see if any RuboCop check is violated. + +Use `disable_ddl_transaction!` when your migration does not actually touch PostgreSQL databases +or does touch _multiple_ PostgreSQL databases. + +- For example, your migration might target a Redis server. As a rule, + you cannot [interact with an external service](database/transaction_guidelines.md#dangerous-example-third-party-api-calls) + inside a PostgreSQL transaction. +- A transaction is used for a single database connection. + If your migrations are targeting multiple databases, such as both `ci` and `main` database, + follow [Migrations for multiple databases](database/migrations_for_multiple_databases.md). + ## Naming conventions Names for database objects (such as tables, indexes, and views) must be lowercase. @@ -285,19 +420,6 @@ minimum acceptable timestamp would be 20230424000000. While the above should be considered a hard rule, it is a best practice to try to keep migration timestamps to within three weeks of the date it is anticipated that the migration will be merged upstream, regardless of how much time has elapsed since the last hard stop. -## Heavy operations in a single transaction - -When using a single-transaction migration, a transaction holds a database connection -for the duration of the migration, so you must make sure the actions in the migration -do not take too much time: GitLab.com's production database has a `15s` timeout, so -in general, the cumulative execution time in a migration should aim to fit comfortably -in that limit. Singular query timings should fit within the [standard limit](database/query_performance.md#timing-guidelines-for-queries) - -In case you need to insert, update, or delete a significant amount of data, you: - -- Must disable the single transaction with `disable_ddl_transaction!`. -- Should consider doing it in a [batched background migration](database/batched_background_migrations.md). - ## Migration helpers and versioning > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339115) in GitLab 14.3. @@ -605,6 +727,71 @@ like a standard migration invocation. The migration might fail if there is a very long running transaction (40+ minutes) accessing the `users` table. +#### Lock-retry methodology at the SQL level + +In this section, we provide a simplified SQL example that demonstrates the use of `lock_timeout`. +You can follow along by running the given snippets in multiple `psql` sessions. + +When altering a table to add a column, +`AccessExclusiveLock`, which conflicts with most lock types, is required on the table. +If the target table is a very busy one, the transaction adding the column +may fail to acquire `AccessExclusiveLock` in a timely fashion. + +Suppose a transaction is attempting to insert a row into a table: + +```sql +-- Transaction 1 +BEGIN; +INSERT INTO my_notes (id) VALUES (1); +``` + +At this point Transaction 1 acquired `RowExclusiveLock` on `my_notes`. +Transaction 1 could still execute more statements prior to committing or aborting. +There could be other similar, concurrent transactions that touch `my_notes`. + +Suppose a transactional migration is attempting to add a column to the table +without using any lock retry helper: + +```sql +-- Transaction 2 +BEGIN; +ALTER TABLE my_notes ADD COLUMN title text; +``` + +Transaction 2 is now blocked because it cannot acquire +`AccessExclusiveLock` on `my_notes` table +as Transaction 1 is still executing and holding the `RowExclusiveLock` +on `my_notes`. + +A more pernicious effect is blocking the transactions that would +normally not conflict with Transaction 1 because Transaction 2 +is queueing to acquire `AccessExclusiveLock`. +In a normal situation, if another transaction attempted to read from and write +to the same table `my_notes` at the same time as Transaction 1, +the transaction would go through +since the locks needed for reading and writing would not +conflict with `RowExclusiveLock` held by Transaction 1. +However, when the request to acquire `AccessExclusiveLock` is queued, +the subsequent requests for conflicting locks on the table would block although +they could be executed concurrently alongside Transaction 1. + +If we used `with_lock_retries`, Transaction 2 would instead quickly +timeout after failing to acquire the lock within the specified time period +and allow other transactions to proceed: + +```sql +-- Transaction 2 (version with with lock timeout) +BEGIN; +SET LOCAL lock_timeout to '100ms'; -- added by the lock retry helper. +ALTER TABLE my_notes ADD COLUMN title text; +``` + +The lock retry helper would repeatedly try the same transaction +at different time intervals until it succeeded. + +Note that `SET LOCAL` scopes the parameter (`lock_timeout`) change to +the transaction. + ## Removing indexes If the table is not empty when removing an index, make sure to use the method @@ -717,7 +904,7 @@ same migration, as the migrations run before the model code is updated and models will have an old schema cache, meaning they won't know about this column and won't be able to set it. In this case it's recommended to: -1. Add the column with default value in a normal migration. +1. Add the column with default value in a standard migration. 1. Remove the default in a post-deployment migration. The post-deployment migration happens after the application restarts, @@ -872,20 +1059,21 @@ If you want to reduce risk slightly, consider putting the migrations into a second merge request after the application changes are merged. This approach provides an opportunity to roll back. -Removing the foreign key on the `projects` table: +Removing the foreign key on the `projects` table using a non-transactional migration: ```ruby # first migration file +class RemovingForeignKeyMigrationClass < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! -def up - with_lock_retries do - remove_foreign_key :my_table, :projects + def up + with_lock_retries do + remove_foreign_key :my_table, :projects + end end -end -def down - with_lock_retries do - add_foreign_key :my_table, :projects + def down + add_concurrent_foreign_key :my_table, :projects, column: COLUMN_NAME end end ``` @@ -894,17 +1082,19 @@ Dropping the table: ```ruby # second migration file +class DroppingTableMigrationClass < Gitlab::Database::Migration[2.1] + def up + drop_table :my_table + end -def up - drop_table :my_table -end - -def down - # create_table ... + def down + # create_table with the same schema but without the removed foreign key ... + end end ``` -After a table has been dropped, it should be added to the database dictionary, following the steps in the [database dictionary guide](database/database_dictionary.md#dropping-tables). +After a table has been dropped, it should be added to the database dictionary, following the +steps in the [database dictionary guide](database/database_dictionary.md#dropping-tables). ## Dropping a sequence @@ -953,10 +1143,10 @@ Truncating a table is uncommon, but you can use the `truncate_tables!` method pr Under the hood, it works like this: - Finds the `gitlab_schema` for the tables to be truncated. -- If the `gitlab_schema` for the tables is included in the connection's gitlab_schemas, +- If the `gitlab_schema` for the tables is included in the connection's `gitlab_schema`s, it then executes the `TRUNCATE` statement. - If the `gitlab_schema` for the tables is not included in the connection's - gitlab_schemas, it does nothing. + `gitlab_schema`s, it does nothing. ## Swapping primary key diff --git a/doc/development/packages/new_format_development.md b/doc/development/packages/new_format_development.md index f730d4f9476..66e6cb89661 100644 --- a/doc/development/packages/new_format_development.md +++ b/doc/development/packages/new_format_development.md @@ -221,7 +221,7 @@ implemented in the same file. After the route has been added, you must add an additional `/authorize` version of the upload endpoint to your API file. [This example](https://gitlab.com/gitlab-org/gitlab/-/blob/398fef1ca26ae2b2c3dc89750f6b20455a1e5507/ee/lib/api/maven_packages.rb#L164) shows the additional endpoint added for Maven. The `/authorize` endpoint verifies and authorizes the request from workhorse, -then the normal upload endpoint is implemented below, consuming the metadata that Workhorse provides to +then the typical upload endpoint is implemented below, consuming the metadata that Workhorse provides to create the package record. Workhorse provides a variety of file metadata such as type, size, and different checksum formats. For testing purposes, you may want to [enable object storage](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/object_storage.md) diff --git a/doc/development/performance.md b/doc/development/performance.md index 291165d8125..fd7e9a85fba 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -217,7 +217,7 @@ Finished in 18.19 seconds (files took 4.8 seconds to load) ``` You can limit the specs that are run by passing any arguments `RSpec` would -normally take. +usually take. ### Using Stackprof in production diff --git a/doc/development/permissions/custom_roles.md b/doc/development/permissions/custom_roles.md index 73a8c920894..9aba4035ec9 100644 --- a/doc/development/permissions/custom_roles.md +++ b/doc/development/permissions/custom_roles.md @@ -8,6 +8,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w Users can create custom roles and define those roles by assigning specific abilities. For example, a user could create an "Engineer" role with `read code` and `admin merge requests` abilities, but without abilities like `admin issues`. +In this context: + +- "Ability" is an action a user can do. +- "Permission" defines the policy classes. + +## Custom roles vs static roles + +In GitLab 15.9 and earlier, GitLab only had [static roles](predefined_roles.md) as a permission system. In this system, there are a few predefined roles that are statically assigned to certain abilities. These static roles are not customizable by customers. + +With custom roles, the customers can decide which abilities they want to assign to certain user groups. For example: + +- In the static role system, reading of vulnerabilities is limited to a Developer role. +- In the custom role system, a customer can assign this ability to a new custom role based on the Reporter role. + ## Technical overview Individual custom roles are stored in the `member_roles` table (`MemberRole` model) and can be defined only for top-level groups. This table includes individual abilities and a `base_access_level` value. This value defines the minimum access level of: diff --git a/doc/development/pipelines/index.md b/doc/development/pipelines/index.md index cadba17f03e..316ae3c83cd 100644 --- a/doc/development/pipelines/index.md +++ b/doc/development/pipelines/index.md @@ -214,7 +214,7 @@ graph LR ## Merge Trains -### Why do we need to have a “stable” master branch to enable merge trains? +### Why do we need to have a "stable" master branch to enable merge trains? If the master branch is unstable (i.e. CI/CD pipelines for the master branch are failing frequently), all of the merge requests pipelines that were added AFTER a faulty merge request pipeline would have to be **cancelled** and **added back to the train**, which would create a lot of delays if the merge train is long. @@ -609,18 +609,18 @@ Exceptions to this general guideline should be motivated and documented. ### Ruby versions testing We're running Ruby 3.0 on GitLab.com, as well as for merge requests and the default branch. -However, there are older versions for which we need to support Ruby 2.7, so we also run our -test suite against Ruby 2.7 on a dedicated 2-hourly scheduled pipelines. +To prepare for the next release, Ruby 3.1, we also run our test suite against Ruby 3.1 on +a dedicated 2-hourly scheduled pipelines. -For merge requests, you can add the `pipeline:run-in-ruby2` label to switch -the Ruby version used for running the whole test suite to 2.7. When you do +For merge requests, you can add the `pipeline:run-in-ruby3_1` label to switch +the Ruby version used for running the whole test suite to 3.1. When you do this, the test suite will no longer run in Ruby 3.0 (default), and an additional job `verify-ruby-3.0` will also run and always fail to remind us to remove the label and run in Ruby 3.0 before merging the merge request. This should let us: -- Test changes for Ruby 2.7 +- Test changes for Ruby 3.1 - Make sure it will not break anything when it's merged into the default branch ### PostgreSQL versions testing @@ -634,33 +634,33 @@ We also run our test suite against PostgreSQL 12 and PostgreSQL 13 upon specific #### Current versions testing -| Where? | PostgreSQL version | Ruby version | -|------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------| -| Merge requests | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `master` branch commits | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 14 (default version), 13 for DB library changes | 3.0 (default version) | -| `maintenance` scheduled pipelines for the `ruby2` branch (every odd-numbered hour), see below. | 14 (default version), 13 for DB library changes | 2.7 | -| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 12, 13, 15 | 3.0 (default version) | +| Where? | PostgreSQL version | Ruby version | +|--------------------------------------------------------------------------------------------------|-------------------------------------------------|-----------------------| +| Merge requests | 14 (default version), 13 for DB library changes | 3.0 (default version) | +| `master` branch commits | 14 (default version), 13 for DB library changes | 3.0 (default version) | +| `maintenance` scheduled pipelines for the `master` branch (every even-numbered hour) | 14 (default version), 13 for DB library changes | 3.0 (default version) | +| `maintenance` scheduled pipelines for the `ruby3_1` branch (every odd-numbered hour), see below. | 14 (default version), 13 for DB library changes | 3.1 | +| `nightly` scheduled pipelines for the `master` branch | 14 (default version), 12, 13, 15 | 3.0 (default version) | -There are 2 pipeline schedules used for testing Ruby 2.7. One is triggering a -pipeline in `ruby2-sync` branch, which updates the `ruby2` branch with latest +There are 2 pipeline schedules used for testing Ruby 3.1. One is triggering a +pipeline in `ruby3_1-sync` branch, which updates the `ruby3_1` branch with latest `master`, and no pipelines will be triggered by this push. The other schedule -is triggering a pipeline in `ruby2` 5 minutes after it, which is considered +is triggering a pipeline in `ruby3_1` 5 minutes after it, which is considered the maintenance schedule to run test suites and update cache. -The `ruby2` branch must not have any changes. The branch is only there to set -`RUBY_VERSION` to `2.7` in the maintenance pipeline schedule. +The `ruby3_1` branch must not have any changes. The branch is only there to set +`RUBY_VERSION` to `3.1` in the maintenance pipeline schedule. -The `gitlab` job in the `ruby2-sync` branch uses a `gitlab-org/gitlab` project +The `gitlab` job in the `ruby3_1-sync` branch uses a `gitlab-org/gitlab` project token with `write_repository` scope and `Maintainer` role with no expiration. -The token is stored in the `RUBY2_SYNC_TOKEN` variable in `gitlab-org/gitlab`. +The token is stored in the `RUBY3_1_SYNC_TOKEN` variable in `gitlab-org/gitlab`. ### Redis versions testing Our test suite runs against Redis 6 as GitLab.com runs on Redis 6 and [Omnibus defaults to Redis 6 for new installs and upgrades](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/redis.rb). -We do run our test suite against Redis 5 on `nightly` scheduled pipelines, specifically when running backward-compatible and forward-compatible PostgreSQL jobs. +We do run our test suite against Redis 7 on `nightly` scheduled pipelines, specifically when running forward-compatible PostgreSQL 15 jobs. #### Current versions testing @@ -668,7 +668,7 @@ We do run our test suite against Redis 5 on `nightly` scheduled pipelines, speci | ------ | ------------------ | | MRs | 6 | | `default branch` (non-scheduled pipelines) | 6 | -| `nightly` scheduled pipelines | 5 | +| `nightly` scheduled pipelines | 7 | ### Single database testing @@ -755,10 +755,14 @@ graph RL; click 2_5-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations" 2_5-1 --> 1-3 & 1-6 & 1-14 & 1-15; - 3_2-1["rspec:coverage (5 minutes)"]; + ac-1["rspec:artifact-collector (2 minutes)<br/>(workaround for 'needs' limitation)"]; + class ac-1 criticalPath; + ac-1 --> 2_5-1; + + 3_2-1["rspec:coverage (3 minutes)"]; class 3_2-1 criticalPath; click 3_2-1 "https://app.periscopedata.com/app/gitlab/652085/Engineering-Productivity---Pipeline-Build-Durations?widget=7248745&udv=0" - 3_2-1 -.->|"(don't use needs<br/>because of limitations)"| 2_5-1; + 3_2-1 --> ac-1; 4_3-1["rspec:undercoverage (1.3 minutes)"]; class 4_3-1 criticalPath; diff --git a/doc/development/pipelines/internals.md b/doc/development/pipelines/internals.md index 35881db8c6d..c0d7bbd3713 100644 --- a/doc/development/pipelines/internals.md +++ b/doc/development/pipelines/internals.md @@ -20,7 +20,8 @@ Pipelines are always created for the following scenarios: Pipeline creation is also affected by the following CI/CD variables: -- If `$FORCE_GITLAB_CI` is set, pipelines are created. +- If `$FORCE_GITLAB_CI` is set, pipelines are created. Not recommended to use. + See [Avoid `$FORCE_GITLAB_CI`](#avoid-force_gitlab_ci). - If `$GITLAB_INTERNAL` is not set, pipelines are not created. No pipeline is created in any other cases (for example, when pushing a branch with no @@ -28,6 +29,25 @@ MR for it). The source of truth for these workflow rules is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml). +### Avoid `$FORCE_GITLAB_CI` + +The pipeline is very complex and we need to clearly understand the kind of +pipeline we want to trigger. We need to know which jobs we should run and +which ones we shouldn't. + +If we use `$FORCE_GITLAB_CI` to force trigger a pipeline, +we don't really know what kind of pipeline it is. The result can be that we don't +run the jobs we want, or we run too many jobs we don't care about. + +Some more context and background can be found at: +[Avoid blanket changes to avoid unexpected run](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102881) + +Here's a list of where we're using this right now, and should try to move away +from using `$FORCE_GITLAB_CI`. + +- [JiHu validation pipeline](https://about.gitlab.com/handbook/ceo/chief-of-staff-team/jihu-support/jihu-validation-pipelines.html) +- [Gitaly downstream GitLab pipeline](https://gitlab.com/gitlab-org/gitaly/-/issues/4615) + ## Default image The default image is defined in [`.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab-ci.yml). @@ -102,7 +122,7 @@ Docker Hub unless `${GITLAB_DEPENDENCY_PROXY}` is also defined there. ### Work around for when a pipeline is started by a Project access token user -When a pipeline is started by a Project access token user (e.g. the `release-tools approver bot` user which +When a pipeline is started by a Project access token user (for example, the `release-tools approver bot` user which automatically updates the Gitaly version used in the main project), [the Dependency proxy isn't accessible](https://gitlab.com/gitlab-org/gitlab/-/issues/332411#note_1130388163) and the job fails at the `Preparing the "docker+machine" executor` step. @@ -165,7 +185,7 @@ and included in `rules` definitions via [YAML anchors](../../ci/yaml/yaml_optimi | `if:` conditions | Description | Notes | |------------------|-------------|-------| -| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success` or `when: manual`), or **not** create a job for forks (by using `when: never`). | +| `if-not-canonical-namespace` | Matches if the project isn't in the canonical (`gitlab-org/` and `gitlab-cn/`) or security (`gitlab-org/security`) namespace. | Use to create a job for forks (by using `when: on_success` or `when: manual`), or **not** create a job for forks (by using `when: never`). | | `if-not-ee` | Matches if the project isn't EE (that is, project name isn't `gitlab` or `gitlab-ee`). | Use to create a job only in the FOSS project (by using `when: on_success` or `when: manual`), or **not** create a job if the project is EE (by using `when: never`). | | `if-not-foss` | Matches if the project isn't FOSS (that is, project name isn't `gitlab-foss`, `gitlab-ce`, or `gitlabhq`). | Use to create a job only in the EE project (by using `when: on_success` or `when: manual`), or **not** create a job if the project is FOSS (by using `when: never`). | | `if-default-refs` | Matches if the pipeline is for `master`, `main`, `/^[\d-]+-stable(-ee)?$/` (stable branches), `/^\d+-\d+-auto-deploy-\d+$/` (auto-deploy branches), `/^security\//` (security branches), merge requests, and tags. | Note that jobs aren't created for branches with this default configuration. | @@ -224,7 +244,7 @@ and included in `rules` definitions via [YAML anchors](../../ci/yaml/yaml_optimi - If you need to **extend a hash**, you should use `extends` - If you need to **extend an array**, you'll need to use `!reference`, or `YAML anchors` as last resort -- For more complex cases (e.g. extend hash inside array, extend array inside hash, ...), you'll have to use `!reference` or `YAML anchors` +- For more complex cases (for example, extend hash inside array, extend array inside hash, ...), you'll have to use `!reference` or `YAML anchors` #### What can `extends` and `YAML anchors` do? diff --git a/doc/development/policies.md b/doc/development/policies.md index b50d654ed28..38f1fc54bf4 100644 --- a/doc/development/policies.md +++ b/doc/development/policies.md @@ -18,7 +18,7 @@ Permissions are broken into two parts: `conditions` and `rules`. Conditions are ### Conditions -Conditions are defined by the `condition` method, and are given a name and a block. The block is executed in the context of the policy object - so it can access `@user` and `@subject`, as well as call any methods defined on the policy. Note that `@user` may be nil (in the anonymous case), but `@subject` is guaranteed to be a real instance of the subject class. +Conditions are defined by the `condition` method, and are given a name and a block. The block is executed in the context of the policy object - so it can access `@user` and `@subject`, as well as call any methods defined on the policy. `@user` may be nil (in the anonymous case), but `@subject` is guaranteed to be a real instance of the subject class. ```ruby class FooPolicy < BasePolicy @@ -42,7 +42,7 @@ Conditions are cached according to their scope. Scope and ordering is covered la ### Rules -A `rule` is a logical combination of conditions and other rules, that are configured to enable or prevent certain abilities. It is important to note that the rule configuration is static - a rule's logic cannot touch the database or know about `@user` or `@subject`. This allows us to cache only at the condition level. Rules are specified through the `rule` method, which takes a block of DSL configuration, and returns an object that responds to `#enable` or `#prevent`: +A `rule` is a logical combination of conditions and other rules, that are configured to enable or prevent certain abilities. The rule configuration is static - a rule's logic cannot touch the database or know about `@user` or `@subject`. This allows us to cache only at the condition level. Rules are specified through the `rule` method, which takes a block of DSL configuration, and returns an object that responds to `#enable` or `#prevent`: ```ruby class FooPolicy < BasePolicy @@ -67,7 +67,7 @@ Within the rule DSL, you can use: - A regular word mentions a condition by name - a rule that is in effect when that condition is truthy. - `~` indicates negation, also available as `negate`. - `&` and `|` are logical combinations, also available as `all?(...)` and `any?(...)`. -- `can?(:other_ability)` delegates to the rules that apply to `:other_ability`. Note that this is distinct from the instance method `can?`, which can check dynamically - this only configures a delegation to another ability. +- `can?(:other_ability)` delegates to the rules that apply to `:other_ability`. This is distinct from the instance method `can?`, which can check dynamically - this only configures a delegation to another ability. `~`, `&` and `|` operators are overridden methods in [`DeclarativePolicy::Rule::Base`](https://gitlab.com/gitlab-org/declarative-policy/-/blob/main/lib/declarative_policy/rule.rb). @@ -104,8 +104,9 @@ An example debug output would look as follows: Each line represents a rule that was evaluated. There are a few things to note: -1. The `-` or `+` symbol indicates whether the rule block was evaluated to be - `false` or `true`, respectively. +1. The `-` symbol indicates the rule block was evaluated to be + `false`. A `+` symbol indicates the rule block was evaluated to be + `true`. 1. The number inside the brackets indicates the score. 1. The last part of the line (for example, `@john : Issue/1`) shows the username and subject for that rule. @@ -124,7 +125,7 @@ heuristic of how expensive they are to calculate. The sorting is dynamic and cache-aware, so that previously calculated conditions are considered first, before computing other conditions. -Note that the score is chosen by a developer via the `score:` parameter +The score is chosen by a developer via the `score:` parameter in a `condition` to denote how expensive evaluating this rule would be relative to other rules. @@ -173,7 +174,7 @@ class FooPolicy < BasePolicy end ``` -includes all rules from `ProjectPolicy`. The delegated conditions are evaluated with the correct delegated subject, and are sorted along with the regular rules in the policy. Note that only the relevant rules for a particular ability are actually considered. +includes all rules from `ProjectPolicy`. The delegated conditions are evaluated with the correct delegated subject, and are sorted along with the regular rules in the policy. Only the relevant rules for a particular ability are actually considered. ### Overrides diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md index 9f5a1a1110f..cf25d83c39a 100644 --- a/doc/development/product_qualified_lead_guide/index.md +++ b/doc/development/product_qualified_lead_guide/index.md @@ -15,7 +15,7 @@ A hand-raise PQL is a user who requests to speak to sales from within the produc 1. Set up GDK with a connection to your local CustomersDot instance. 1. Set up CustomersDot to talk to a staging instance of Workato. -1. Set up CustomersDot using the [normal install instructions](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/doc/setup/installation_steps.md). +1. Set up CustomersDot using the [standard install instructions](https://gitlab.com/gitlab-org/customers-gitlab-com/-/blob/staging/doc/setup/installation_steps.md). 1. Set the `CUSTOMER_PORTAL_URL` environment variable to your local (or ngrok) URL of your CustomersDot instance. 1. Place `export CUSTOMER_PORTAL_URL='https://XXX.ngrok.io/'` in your shell `rc` script (`~/.zshrc` or `~/.bash_profile` or `~/.bashrc`) and restart GDK. 1. Enter the credentials on CustomersDot development to Workato in your `/config/secrets.yml` and restart. Credentials for the Workato Staging are in the 1Password Subscription portal vault. The URL for staging is `https://apim.workato.com/gitlab-dev/services/marketo/lead`. @@ -131,8 +131,8 @@ The flow of a PQL lead is as follows: ```mermaid sequenceDiagram Trial Frontend Forms ->>TrialsController#create_lead: GitLab.com frontend sends [lead] to backend - TrialsController#create_lead->>CreateLeadService: [lead] - TrialsController#create_lead->>ApplyTrialService: [lead] Apply the trial + TrialsController#create->>CreateLeadService: [lead] + TrialsController#create->>ApplyTrialService: [lead] Apply the trial CreateLeadService->>SubscriptionPortalClient#generate_trial(sync_to_gl=false): [lead] Creates customer account on CustomersDot ApplyTrialService->>SubscriptionPortalClient#generate_trial(sync_to_gl=true): [lead] Asks CustomersDot to apply the trial on namespace SubscriptionPortalClient#generate_trial(sync_to_gl=false)->>CustomersDot|TrialsController#create(sync_to_gl=false): GitLab.com sends [lead] to CustomersDot @@ -169,8 +169,8 @@ sequenceDiagram ```mermaid sequenceDiagram - HandRaiseForm Vue Component->>TrialsController#create_hand_raise_lead: GitLab.com frontend sends [lead] to backend - Subscriptions::HandRaiseLeadsController#create->>CreateHandRaiseLeadService: [lead] + HandRaiseForm Vue Component->>HandRaiseLeadsController#create: GitLab.com frontend sends [lead] to backend + HandRaiseLeadsController#create->>CreateHandRaiseLeadService: [lead] CreateHandRaiseLeadService->>SubscriptionPortalClient: [lead] SubscriptionPortalClient->>CustomersDot|TrialsController#create_hand_raise_lead: GitLab.com sends [lead] to CustomersDot ``` diff --git a/doc/development/project_templates.md b/doc/development/project_templates.md index cc53ef77c62..da933c8a009 100644 --- a/doc/development/project_templates.md +++ b/doc/development/project_templates.md @@ -19,8 +19,8 @@ To make the project template available when creating a new project, the vendorin 1. Create a working template ([example](https://gitlab.com/gitlab-org/project-templates/dotnetcore)) - 2 types of built-in templates are available within GitLab: - - **Normal templates**: Available in GitLab Core, Starter and above (this is the most common type of built-in template). - - To contribute a normal template: + - **Standard templates**: Available in GitLab Core, Starter and above (this is the most common type of built-in template). + - To contribute a standard template: - Add details of the template in the `localized_templates_table` method in `gitlab/lib/gitlab/project_template.rb`, - Add details of the template in `spec/lib/gitlab/project_template_spec.rb`, in the test for the `all` method, and - Add details of the template in `gitlab/app/assets/javascripts/projects/default_project_templates.js`. diff --git a/doc/development/reactive_caching.md b/doc/development/reactive_caching.md index 9e3b1f58abe..d0652c85c6d 100644 --- a/doc/development/reactive_caching.md +++ b/doc/development/reactive_caching.md @@ -197,9 +197,8 @@ There are some `class_attribute` options which can be tweaked. self.reactive_cache_key = -> (record) { [model_name.singular, record.id] } ``` -- The `data` and `alive` cache keys in this case are `"ExampleModel:1:arg1:arg2"` - and `"ExampleModel:1:arg1:arg2:alive"` respectively, where `ExampleModel` is the - name of the model, `1` is the ID of the record, `arg1` and `arg2` are parameters +- The `data` cache key is `"ExampleModel:1:arg1:arg2"` and `alive` cache keys is `"ExampleModel:1:arg1:arg2:alive"`, + where `ExampleModel` is the name of the model, `1` is the ID of the record, `arg1` and `arg2` are parameters passed to `with_reactive_cache`. - If you're including this concern in an integration (`app/models/integrations/`) instead, you must override the default by adding the following to your integration: diff --git a/doc/development/redis.md b/doc/development/redis.md index 5073d9350e8..ebc7c0271a1 100644 --- a/doc/development/redis.md +++ b/doc/development/redis.md @@ -285,3 +285,48 @@ This is used by the [`RepositorySetCache`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/repository_set_cache.rb) to provide a convenient way to use sets to cache repository data like branch names. + +## Background migration + +Redis-based migrations involve using the `SCAN` command to scan the entire Redis instance for certain key patterns. +For large Redis instances, the migration might [exceed the time limit](migration_style_guide.md#how-long-a-migration-should-take) +for regular or post-deployment migrations. [`RedisMigrationWorker`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/redis_migration_worker.rb) +performs long-running Redis migrations as a background migration. + +To perform a background migration by creating a class: + +```ruby +module Gitlab + module BackgroundMigration + module Redis + class BackfillCertainKey + def perform(keys) + # implement logic to clean up or backfill keys + end + + def scan_match_pattern + # define the match pattern for the `SCAN` command + end + + def redis + # define the exact Redis instance + end + end + end + end +end +``` + +To trigger the worker through a post-deployment migration: + +```ruby +class ExampleBackfill < Gitlab::Database::Migration[2.1] + disable_ddl_transaction! + + MIGRATION='BackfillCertainKey' + + def up + queue_redis_migration_job(MIGRATION) + end +end +``` diff --git a/doc/development/sec/gemnasium_analyzer_data.md b/doc/development/sec/gemnasium_analyzer_data.md new file mode 100644 index 00000000000..2da787a277a --- /dev/null +++ b/doc/development/sec/gemnasium_analyzer_data.md @@ -0,0 +1,33 @@ +--- +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Gemnasium analyzer data + +The following table lists the data available for the Gemnasium analyzer. + +| Property \ Tool | Gemnasium | +|:----------------------------------------------|:-----------------------:| +| Severity | **{check-circle}** Yes | +| Title | **{check-circle}** Yes | +| File | **{check-circle}** Yes | +| Start line | **{dotted-circle}** No | +| End line | **{dotted-circle}** No | +| External ID (for example, CVE) | **{check-circle}** Yes | +| URLs | **{check-circle}** Yes | +| Internal doc/explanation | **{check-circle}** Yes | +| Solution | **{check-circle}** Yes | +| Confidence | **{dotted-circle}** No | +| Affected item (for example, class or package) | **{check-circle}** Yes | +| Source code extract | **{dotted-circle}** No | +| Internal ID | **{check-circle}** Yes | +| Date | **{check-circle}** Yes | +| Credits | **{check-circle}** Yes | + +- **{check-circle}** Yes => we have that data +- **{dotted-circle}** No => we don't have that data, or it would need to develop specific or inefficient/unreliable logic to obtain it. + +The values provided by these tools are heterogeneous, so they are sometimes normalized into common +values (for example, `severity`, `confidence`, etc). diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index c5e7a58af0d..8d6f36bb189 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -333,7 +333,7 @@ XSS issues are commonly classified in three categories, by their delivery method ### Impact -The injected client-side code is executed on the victim's browser in the context of their current session. This means the attacker could perform any same action the victim would normally be able to do through a browser. The attacker would also have the ability to: +The injected client-side code is executed on the victim's browser in the context of their current session. This means the attacker could perform any same action the victim would typically be able to do through a browser. The attacker would also have the ability to: - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [log victim keystrokes](https://youtu.be/2VFavqfDS6w?t=1367) - launch a network scan from the victim's browser @@ -524,7 +524,7 @@ of these behaviors. The Ruby method [`Pathname.join`](https://ruby-doc.org/stdlib-2.7.4/libdoc/pathname/rdoc/Pathname.html#method-i-join) joins path names. Using methods in a specific way can result in a path name typically prohibited in -normal use. In the examples below, we see attempts to access `/etc/passwd`, which is a sensitive file: +typical use. In the examples below, we see attempts to access `/etc/passwd`, which is a sensitive file: ```ruby require 'pathname' diff --git a/doc/development/sidekiq/compatibility_across_updates.md b/doc/development/sidekiq/compatibility_across_updates.md index d20f4230fc8..1d8b9d15cc6 100644 --- a/doc/development/sidekiq/compatibility_across_updates.md +++ b/doc/development/sidekiq/compatibility_across_updates.md @@ -205,6 +205,6 @@ end ``` -You must rename the queue in a post-deployment migration not in a normal +You must rename the queue in a post-deployment migration not in a standard migration. Otherwise, it runs too early, before all the workers that schedule these jobs have stopped running. See also [other examples](../database/post_deployment_migrations.md#use-cases). diff --git a/doc/development/sidekiq/idempotent_jobs.md b/doc/development/sidekiq/idempotent_jobs.md index 1c4f4ba44a8..8ccbeee283d 100644 --- a/doc/development/sidekiq/idempotent_jobs.md +++ b/doc/development/sidekiq/idempotent_jobs.md @@ -156,7 +156,7 @@ end ## Setting the deduplication time-to-live (TTL) -Deduplication depends on an idempotent key that is stored in Redis. This is normally +Deduplication depends on an idempotent key that is stored in Redis. This is usually cleared by the configured deduplication strategy. However, the key can remain until its TTL in certain cases like: diff --git a/doc/development/sidekiq/index.md b/doc/development/sidekiq/index.md index d510414baa3..936388a14fe 100644 --- a/doc/development/sidekiq/index.md +++ b/doc/development/sidekiq/index.md @@ -121,7 +121,7 @@ Sidekiq workers are deferred by two ways, 1. Manual: Feature flags can be used to explicitly defer a particular worker, more details can be found [here](../feature_flags/index.md#deferring-sidekiq-jobs). 1. Automatic: Similar to the [throttling mechanism](../database/batched_background_migrations.md#throttling-batched-migrations) in batched migrations, database health indicators are used to defer a Sidekiq worker. - To use the automatic deferring mechanism, worker has to opt-in by calling `defer_on_database_health_signal` with gitlab_schema, delay_by (time to delay) and tables (which is used by autovacuum db indicator) as it's parameters. + To use the automatic deferring mechanism, worker has to opt-in by calling `defer_on_database_health_signal` with `gitlab_schema`, delay_by (time to delay) and tables (which is used by autovacuum db indicator) as it's parameters. **Example:** diff --git a/doc/development/software_design.md b/doc/development/software_design.md index d4edbaa72be..33a6dfcb4a7 100644 --- a/doc/development/software_design.md +++ b/doc/development/software_design.md @@ -139,3 +139,165 @@ end If classes that are defined into a namespace have a lot in common with classes in other namespaces, chances are that these two namespaces are part of the same bounded context. + +## Taming Omniscient classes + +We must consider not adding new data and behavior to [omniscient classes](https://en.wikipedia.org/wiki/God_object) (also known as god objects). +We consider `Project`, `User`, `MergeRequest`, `Ci::Pipeline` and any classes above 1000 LOC to be omniscient. + +Such classes are overloaded with responsibilities. New data and behavior can most of the time be added +as a separate and dedicated class. + +Guidelines: + +- If you mostly need a reference to the object ID (for example `Project#id`) you could add a new model + that uses the foreign key or a thin wrapper around the object to add special behavior. +- If you find out that by adding a method to the omniscient class you also end up adding a couple of other methods + (private or public) it's a sign that these methods should be encapsulated in a dedicated class. +- It's temping to add a method to `Project` because that's the starting point of data and associations. + Try to define behavior in the bounded context where it belongs, not where the data (or some of it) is. + This helps creating facets of the omniscient object that are much more relevant in the bounded context than + having generic and overloaded objects which bring more coupling and complexity. + +### Example: Define a thin domain object around a generic model + +Instead of adding multiple methods to `User` because it has an association to `abuse_trust_scores`, +try inverting the dependency. + +```ruby +## +# BAD: Behavior added to User object. +class User + def spam_score + abuse_trust_scores.spamcheck.average(:score) || 0.0 + end + + def spammer? + # Warning sign: we use a constant that belongs to a specific bounded context! + spam_score > Abuse::TrustScore::SPAMCHECK_HAM_THRESHOLD + end + + def telesign_score + abuse_trust_scores.telesign.recent_first.first&.score || 0.0 + end + + def arkose_global_score + abuse_trust_scores.arkose_global_score.recent_first.first&.score || 0.0 + end + + def arkose_custom_score + abuse_trust_scores.arkose_custom_score.recent_first.first&.score || 0.0 + end +end + +# Usage: +user = User.find(1) +user.spam_score +user.telesign_score +user.arkose_global_score +``` + +```ruby +## +# GOOD: Define a thin class that represents a user trust score +class Abuse::UserTrustScore + def initialize(user) + @user = user + end + + def spam + scores.spamcheck.average(:score) || 0.0 + end + + def spammer? + spam > Abuse::TrustScore::SPAMCHECK_HAM_THRESHOLD + end + + def telesign + scores.telesign.recent_first.first&.score || 0.0 + end + + def arkose_global + scores.arkose_global_score.recent_first.first&.score || 0.0 + end + + def arkose_custom + scores.arkose_custom_score.recent_first.first&.score || 0.0 + end + + private + + def scores + Abuse::TrustScore.for_user(@user) + end +end + +# Usage: +user = User.find(1) +user_score = Abuse::UserTrustScore.new(user) +user_score.spam +user_score.spammer? +user_score.telesign +user_score.arkose_global +``` + +See a real example [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117853#note_1423070054). + +### Example: Use Dependency Inversion to extract a domain concept + +```ruby +## +# BAD: methods related to integrations defined in Project. +class Project + has_many :integrations + + def find_or_initialize_integrations + # ... + end + + def find_or_initialize_integration(name) + # ... + end + + def disabled_integrations + # ... + end + + def ci_integrations + # ... + end + + # many more methods... +end +``` + +```ruby +## +# GOOD: All logic related to Integrations is enclosed inside the `Integrations::` +# bounded context. +module Integrations + class ProjectIntegrations + def initialize(project) + @project = project + end + + def all_integrations + @project.integrations # can still leverage caching of AR associations + end + + def find_or_initialize(name) + # ... + end + + def all_disabled + all_integrations.disabled + end + + def all_ci + all_integrations.ci_integration + end + end +end +``` + +Real example of [similar refactoring](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/92985). diff --git a/doc/development/spam_protection_and_captcha/web_ui.md b/doc/development/spam_protection_and_captcha/web_ui.md index 0cc17854010..9f92105b18d 100644 --- a/doc/development/spam_protection_and_captcha/web_ui.md +++ b/doc/development/spam_protection_and_captcha/web_ui.md @@ -48,11 +48,11 @@ However, even though the actual handling of the request interception and modal is transparent, without any mandatory changes to the involved JavaScript or Vue components for the form or page, changes in request or error handling may be required. Changes are needed because the existing behavior may not work correctly: for example, if a failed or cancelled -CAPTCHA display interrupts the normal request flow or UI updates. +CAPTCHA display interrupts the standard request flow or UI updates. Careful exploratory testing of all scenarios is important to uncover any potential problems. -This sequence diagram illustrates the normal CAPTCHA flow for JavaScript XHR/Fetch requests +This sequence diagram illustrates the standard CAPTCHA flow for JavaScript XHR/Fetch requests on the frontend: ```mermaid @@ -73,7 +73,7 @@ sequenceDiagram ``` The backend is also cleanly abstracted via mixin modules and helper methods. The three main -changes required to the relevant backend controller actions (normally just `create`/`update`) are: +changes required to the relevant backend controller actions (typically just `create`/`update`) are: 1. Pass `perform_spam_check: true` to the Update Service class constructor. It is set to `true` by default in the Create Service. @@ -86,7 +86,7 @@ changes required to the relevant backend controller actions (normally just `crea 1. Checking if there the model contains an error, and the `needs_recaptcha` flag is true. - If yes: Add the appropriate spam or CAPTCHA fields to the JSON response, and return a `409 - Conflict` HTTP status code. - - If no (if CAPTCHA is disabled or if no spam was detected): The normal request return + - If no (if CAPTCHA is disabled or if no spam was detected): The standard request return logic passed in the block is run. Thanks to the abstractions, it's more straightforward to implement than it is to explain it. diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 90267a517b8..5a4ac99f5c5 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -50,6 +50,32 @@ bundle exec guard When using spring and guard together, use `SPRING=1 bundle exec guard` instead to make use of spring. +### General guidelines + +- Use a single, top-level `RSpec.describe ClassName` block. +- Use `.method` to describe class methods and `#method` to describe instance + methods. +- Use `context` to test branching logic (`RSpec/AvoidConditionalStatements` Rubocop Cop - [MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/117152)). +- Try to match the ordering of tests to the ordering in the class. +- Try to follow the [Four-Phase Test](https://thoughtbot.com/blog/four-phase-test) pattern, using newlines + to separate phases. +- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'`. +- Don't assert against the absolute value of a sequence-generated attribute (see + [Gotchas](../gotchas.md#do-not-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). +- Avoid using `expect_any_instance_of` or `allow_any_instance_of` (see + [Gotchas](../gotchas.md#avoid-using-expect_any_instance_of-or-allow_any_instance_of-in-rspec)). +- Don't supply the `:each` argument to hooks because it's the default. +- On `before` and `after` hooks, prefer it scoped to `:context` over `:all`. +- When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element, + use a Capybara matcher beforehand (such as `find('.js-foo')`) to ensure the element actually exists. +- Use `focus: true` to isolate parts of the specs you want to run. +- Use [`:aggregate_failures`](https://rspec.info/features/3-12/rspec-core/expectation-framework-integration/aggregating-failures/) when there is more than one expectation in a test. +- For [empty test description blocks](https://github.com/rubocop-hq/rspec-style-guide#it-and-specify), use `specify` rather than `it do` if the test is self-explanatory. +- Use `non_existing_record_id`/`non_existing_record_iid`/`non_existing_record_access_level` + when you need an ID/IID/access level that doesn't actually exist. Using 123, 1234, + or even 999 is brittle as these IDs could actually exist in the database in the + context of a CI run. + ### Eager loading the application code By default, the application code: @@ -132,9 +158,44 @@ We should reduce test dependencies, and avoiding capabilities also reduces the amount of set-up needed. `:js` is particularly important to avoid. This must only be used if the feature -test requires JavaScript reactivity in the browser. Using a headless +test requires JavaScript reactivity in the browser (for example, clicking a Vue.js component). Using a headless browser is much slower than parsing the HTML response from the app. +#### Profiling: see where your test spend its time + +[`rspec-stackprof`](https://github.com/dkhroad/rspec-stackprof) can be used to generate a flame graph that shows you where you test spend its time. + +The gem generates a JSON report that we can upload to <https://www.speedscope.app> for an interactive visualization. + +##### Installation + +`stackprof` gem is [already installed with GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/695fcee0c5541b4ead0a90eb9b8bf0b69bee796c/Gemfile#L487), and we also have a script available that generates the JSON report (`bin/rspec-stackprof`). + +```shell +# Optional: install the `speedscope` package to easily upload the JSON report to https://www.speedscope.app +npm install -g speedscope +``` + +##### Generate the JSON report + +```shell +bin/rspec-stackprof --speedscope=true <your_slow_spec> +# There will be the name of the report displayed when the script ends. + +# Upload the JSON report to speedscope.app +speedscope tmp/<your-json-report>.json +``` + +##### How to interpret the flamegraph + +Below are some useful tips to interpret and navigate the flamegraph: + +- There are [several views available](https://github.com/jlfwong/speedscope#views) for the flamegraph. `Left Heavy` is particularly useful when there are a lot of function calls (for example, feature specs). +- You can zoom in or out! See [the navigation documentation](https://github.com/jlfwong/speedscope#navigation) +- If you are working on a slow feature test, search for `Capybara::DSL#` in the search to see the capybara actions that are made, and how long they take! + +See [#414929](https://gitlab.com/gitlab-org/gitlab/-/issues/414929#note_1425239887) or [#375004](https://gitlab.com/gitlab-org/gitlab/-/issues/375004#note_1109867718) for some analysis examples. + #### Optimize factory usage A common cause of slow tests is excessive creation of objects, and thus @@ -240,9 +301,21 @@ There are various ways to create objects and store them in variables in your tes - `let` lazily creates the object. It isn't created until the object is called. `let` is generally inefficient as it creates a new object for every example. `let` is fine for simple values. However, more efficient variants of `let` are best when dealing with database models such as factories. - `let_it_be_with_refind` works similar to `let_it_be_with_reload`, but the [former calls `ActiveRecord::Base#find`](https://github.com/test-prof/test-prof/blob/936b29f87b36f88a134e064aa6d8ade143ae7a13/lib/test_prof/ext/active_record_refind.rb#L15) instead of `ActiveRecord::Base#reload`. `reload` is usually faster than `refind`. - `let_it_be_with_reload` creates an object one time for all examples in the same context, but after each example, the database changes are rolled back, and `object.reload` will be called to restore the object to its original state. This means you can make changes to the object before or during an example. However, there are cases where [state leaks across other models](https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#state-leakage-detection) can occur. In these cases, `let` may be an easier option, especially if only a few examples exist. -- `let_it_be` creates an immutable object one time for all of the examples in the same context. This is a great alternative to `let` and `let!` for objects that do not need to change from one example to another. Using `let_it_be` can dramatically speed up tests that create database models. See <https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#let-it-be> for more details and examples. +- `let_it_be` creates an object one time for all of the examples in the same context. This is a great alternative to `let` and `let!` for objects that do not need to change from one example to another. Using `let_it_be` can dramatically speed up tests that create database models. See <https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#let-it-be> for more details and examples. + +Pro-tip: When writing tests, it is best to consider the objects inside a `let_it_be` as **immutable**, as there are some important caveats when modifying objects inside a `let_it_be` declaration ([1](https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#database-is-rolled-back-to-a-pristine-state-but-the-objects-are-not), [2](https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#modifiers)). To make your `let_it_be` objects immutable, consider using `.freeze`: + +```shell +# Before +let_it_be(:namespace) { create_default(:namespace) + +# After +let_it_be(:namespace) { create_default(:namespace).freeze +``` + +See <https://github.com/test-prof/test-prof/blob/master/docs/recipes/let_it_be.md#state-leakage-detection> for more information on `let_it_be` freezing. -`let_it_be` is the most optimized option since it instantiates an object once and does not change it. If you find yourself needing `let` instead of `let_it_be`, try `let_it_be_with_reload`. +`let_it_be` is the most optimized option since it instantiates an object once and shares its instance across examples. If you find yourself needing `let` instead of `let_it_be`, try `let_it_be_with_reload`. ```ruby # Old @@ -336,6 +409,35 @@ NOTE: Refrain from using this stub helper if the test code relies on persisting `project_authorizations` or `Member` records. Use `Project#add_member` or `Group#add_member` instead. +#### Additional profiling metrics + +We can use the `rspec_profiling` gem to diagnose, for instance, the number of SQL queries we're making when running a test. + +This could be caused by some application side SQL queries **triggered by a test that could mock parts that are not under test** (for example, [!123810](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123810)). + +[See the instructions in the performance docs](../performance.md#rspec-profiling). + +#### Troubleshoot slow feature test + +A slow feature test can generally be optimized the same way as any other test. However, there are some specific techniques that can make the troubleshooting session more fruitful. + +##### See what the feature test is doing in the UI + +```shell +# Before +bin/rspec ./spec/features/admin/admin_settings_spec.rb:992 + +# After +WEBDRIVER_HEADLESS=0 bin/rspec ./spec/features/admin/admin_settings_spec.rb:992 +``` + +See [Run `:js` spec in a visible browser](#run-js-spec-in-a-visible-browser) for more info. + +##### Search for `Capybara::DSL#` when using profiling + +<!-- TODO: Add the search keywords --> +When using [`stackprof` flamegraphs](#profiling-see-where-your-test-spend-its-time), search for `Capybara::DSL#` in the search to see the capybara actions that are made, and how long they take! + #### Identify slow tests Running a spec with profiling is a good way to start optimizing a spec. This can @@ -437,31 +539,11 @@ performance gains. When combining tests, consider using `:aggregate_failures`, so that the full results are available, and not just the first failure. -### General guidelines +#### In case you're stuck -- Use a single, top-level `RSpec.describe ClassName` block. -- Use `.method` to describe class methods and `#method` to describe instance - methods. -- Use `context` to test branching logic. -- Try to match the ordering of tests to the ordering in the class. -- Try to follow the [Four-Phase Test](https://thoughtbot.com/blog/four-phase-test) pattern, using newlines - to separate phases. -- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'` -- Don't assert against the absolute value of a sequence-generated attribute (see - [Gotchas](../gotchas.md#do-not-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). -- Avoid using `expect_any_instance_of` or `allow_any_instance_of` (see - [Gotchas](../gotchas.md#avoid-using-expect_any_instance_of-or-allow_any_instance_of-in-rspec)). -- Don't supply the `:each` argument to hooks because it's the default. -- On `before` and `after` hooks, prefer it scoped to `:context` over `:all` -- When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element, - use a Capybara matcher beforehand (such as `find('.js-foo')`) to ensure the element actually exists. -- Use `focus: true` to isolate parts of the specs you want to run. -- Use [`:aggregate_failures`](https://rspec.info/features/3-12/rspec-core/expectation-framework-integration/aggregating-failures/) when there is more than one expectation in a test. -- For [empty test description blocks](https://github.com/rubocop-hq/rspec-style-guide#it-and-specify), use `specify` rather than `it do` if the test is self-explanatory. -- Use `non_existing_record_id`/`non_existing_record_iid`/`non_existing_record_access_level` - when you need an ID/IID/access level that doesn't actually exists. Using 123, 1234, - or even 999 is brittle as these IDs could actually exist in the database in the - context of a CI run. +We have a `backend_testing_performance` [domain expertise](https://about.gitlab.com/handbook/engineering/workflow/code-review/#domain-experts) to list people that could help refactor slow backend specs. + +To find people that could help, search for `backend testing performance` on the [Engineering Projects page](https://about.gitlab.com/handbook/engineering/projects/), or look directly in [the `www-gitlab-org` project](https://gitlab.com/search?group_id=6543&nav_source=navbar&project_id=7764&repository_ref=master&scope=blobs&search=backend_testing_performance+path%3Adata%2Fteam_members%2F*&search_code=true). ### Feature category metadata @@ -918,7 +1000,7 @@ describe 'specs which require time to be frozen to a specific date and/or time', end ``` -[Under the hood](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/time_travel.rb), these helpers use the `around(:each)` hook and the block syntax of the +[Under the hood](https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/gitlab-rspec/lib/gitlab/rspec/configurations/time_travel.rb), these helpers use the `around(:each)` hook and the block syntax of the [`ActiveSupport::Testing::TimeHelpers`](https://api.rubyonrails.org/classes/ActiveSupport/Testing/TimeHelpers.html) methods: @@ -932,6 +1014,19 @@ around(:each) do |example| end ``` +Remember that any objects created before the examples run (such as objects created via `let_it_be`) will be outside spec scope. +If the time for everything needs to be frozen, `before :all` can be used to encapsulate the setup as well. + +```ruby +before :all do + freeze_time +end + +after :all do + unfreeze_time +end +``` + ### Feature flags in tests This section was moved to [developing with feature flags](../feature_flags/index.md). @@ -1292,6 +1387,19 @@ describe "#==" do end ``` +If, after creating a table-based test, you see an error that looks like this: + +```ruby +NoMethodError: + undefined method `to_params' + + param_sets = extracted.is_a?(Array) ? extracted : extracted.to_params + ^^^^^^^^^^ + Did you mean? to_param +``` + +That indicates that you need to include the line `using RSpec::Parameterized::TableSyntax` in the spec file. + <!-- vale gitlab.Spelling = NO --> WARNING: @@ -1470,19 +1578,17 @@ under `spec/support/helpers/`. Helpers can be placed in a subfolder if they appl to a certain type of specs only (such as features or requests) but shouldn't be if they apply to multiple type of specs. -Helpers should follow the Rails naming / namespacing convention. For instance -`spec/support/helpers/cycle_analytics_helpers.rb` should define: +Helpers should follow the Rails naming / namespacing convention, where +`spec/support/helpers/` is the root. For instance +`spec/support/helpers/features/iteration_helpers.rb` should define: ```ruby -module Spec - module Support - module Helpers - module CycleAnalyticsHelpers - def create_commit_referencing_issue(issue, branch_name: random_git_name) - project.repository.add_branch(user, branch_name, 'main') - create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name) - end - end +# frozen_string_literal: true + +module Features + module IterationHelpers + def iteration_period(iteration) + "#{iteration.start_date.to_fs(:medium)} - #{iteration.due_date.to_fs(:medium)}" end end end @@ -1492,8 +1598,14 @@ Helpers should not change the RSpec configuration. For instance, the helpers mod described above should not include: ```ruby +# bad RSpec.configure do |config| - config.include Spec::Support::Helpers::CycleAnalyticsHelpers + config.include Features::IterationHelpers +end + +# good, include in specific spec +RSpec.describe 'Issue Sidebar', feature_category: :team_planning do + include Features::IterationHelpers end ``` diff --git a/doc/development/testing_guide/contract/index.md b/doc/development/testing_guide/contract/index.md index cf23792e239..577699fa828 100644 --- a/doc/development/testing_guide/contract/index.md +++ b/doc/development/testing_guide/contract/index.md @@ -78,7 +78,9 @@ As mentioned in the [folder structure section](#consumer-tests), consumer tests #### Provider naming -These are the API endpoints that provides the data to the consumer so they are named according to the API endpoint they pertain to. Be mindful that this begins with the HTTP request method and the rest of the name is as descriptive as possible. For example, if we're writing a test for the `GET /groups/:id/projects` endpoint, we don't want to name it "GET projects endpoint" as there is a `GET /projects` endpoint as well that also fetches a list of projects the user has access to across all of GitLab. To choose an appropriate name, we can start by checking out our [API documentation](../../../api/api_resources.md) and naming it the same way it is named in there while making sure to keep the name in sentence case. So the [`GET /groups/:id/projects`](../../../api/groups.md#list-a-groups-projects) would be called `GET list a group's projects` and [`GET /projects`](../../../api/projects.md#list-all-projects) would be called `GET list all projects`. Subsequently, the test files are named `get_list_a_groups_projects_helper.rb` and `get_list_all_projects_helper.rb` respectively. +These are the API endpoints that provide the data to the consumer so they are named according to the API endpoint they pertain to. Be mindful that this begins with the HTTP request method and the rest of the name is as descriptive as possible. For example, if we're writing a test for the `GET /groups/:id/projects` endpoint, we don't want to name it "GET projects endpoint" as there is a `GET /projects` endpoint as well that also fetches a list of projects the user has access to across all of GitLab. + +To choose an appropriate name, we can start by checking out our [API documentation](../../../api/api_resources.md) and naming it the same way it is named in there while making sure to keep the name in sentence case. So [`GET /groups/:id/projects`](../../../api/groups.md#list-a-groups-projects) would be called `GET list a group's projects` and the test file name is `get_list_a_groups_projects_helper.rb`. [`GET /projects`](../../../api/projects.md#list-all-projects) would be called `GET list all projects`, and the test file name `get_list_all_projects_helper.rb`. There are some cases where the provider being tested may not be documented so, in those cases, fall back to starting with the HTTP request method followed by a name that is as descriptive as possible to ensure it's easy to tell what the provider is for. diff --git a/doc/development/testing_guide/contract/provider_tests.md b/doc/development/testing_guide/contract/provider_tests.md index cb3aeae529d..71940941d51 100644 --- a/doc/development/testing_guide/contract/provider_tests.md +++ b/doc/development/testing_guide/contract/provider_tests.md @@ -10,7 +10,7 @@ This tutorial guides you through writing a provider test from scratch. It is a c ## Create the skeleton -Provider tests are quite simple. The goal is to set up the test data and then link that with the corresponding contract. Start by creating a file called `get_discussions_helper.rb` under `spec/contracts/provider/pact_helpers/project/merge_request`. Note that the files are called `helpers` to match how they are called by Pact in the Rake tasks, which are set up at the end of this tutorial. +Provider tests are quite simple. The goal is to set up the test data and then link that with the corresponding contract. Start by creating a file called `get_discussions_helper.rb` under `spec/contracts/provider/pact_helpers/project/merge_request`. The files are called `helpers` to match how they are called by Pact in the Rake tasks, which are set up at the end of this tutorial. For more information about how the contract test directory is structured, see [Test suite folder structure](index.md#test-suite-folder-structure). @@ -122,7 +122,7 @@ To create the test data, create `show_state.rb` under `spec/contracts/provider/s ### Default user in `spec/contracts/provider/spec_helper.rb` -Before you create the test data, note that a default user is created in the [`spec_helper`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/spec/contracts/provider/spec_helper.rb), which is the user being used for the test runs. This user is configured using `RSpec.configure`, as Pact actually is built on top of RSpec. This step allows us to configure the user before any of the test runs. +Before you create the test data, a default user is created in the [`spec_helper`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/spec/contracts/provider/spec_helper.rb), which is the user being used for the test runs. This user is configured using `RSpec.configure`, as Pact actually is built on top of RSpec. This step allows us to configure the user before any of the test runs. ```ruby RSpec.configure do |config| diff --git a/doc/development/testing_guide/end_to_end/beginners_guide.md b/doc/development/testing_guide/end_to_end/beginners_guide.md index 78e6af4a347..4627d5d29cb 100644 --- a/doc/development/testing_guide/end_to_end/beginners_guide.md +++ b/doc/development/testing_guide/end_to_end/beginners_guide.md @@ -219,8 +219,8 @@ end **How do we test?** -1. Check if the user avatar appears in the top navigation. -1. Check if the user avatar *does not* appear in the top navigation. +1. Check if the user avatar appears in the left sidebar. +1. Check if the user avatar *does not* appear in the left sidebar. Behind the scenes, `be_signed_in` is a [predicate matcher](https://rspec.info/features/3-12/rspec-expectations/built-in-matchers/predicates/) diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index c1f06bb9a66..96bd02d235b 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -99,6 +99,43 @@ end We recommend creating four associated test cases, two for each shared example. +## Test naming + +Test names should form a readable sentence defining the purpose of the test. Our [testing guide](index.md) extends the [Thoughtbot testing style guide](https://github.com/thoughtbot/guides/tree/master/testing-rspec). This page clarifies the guidelines, along with input from [https://www.betterspecs.org/](https://www.betterspecs.org/) and [the RSpec naming guide](https://rspec.rubystyle.guide/#naming.) + +### Recommended approach + +The following block generates a test named `Plan wiki content creation in a project adds a home page` + +``` ruby +# `RSpec.describe` is the DevOps Stage being covered +RSpec.describe 'Plan', product_group: :knowledge do + # `describe` is the feature being tested + describe 'wiki content creation' do + # `context` provides the condition being covered + context 'in a project' + # `it` defines the expected result of the test + it 'adds a home page' + ... + end + ... + end + ... + end +end +``` + +1. Every `describe`, `context`, and `it` blocks should have a short description attached +1. Keep descriptions as concise as possible. + 1. Long descriptions or multiple conditionals could be a sign it should be split up (additional `context` blocks). + 1. The [Documentation Style Guide](../../documentation/styleguide/index.md) gives recommendations on how to write concisely and with [active voice](../../documentation/styleguide/word_list.md#active-voice). +1. The outermost `Rspec.describe` block should be [the DevOps stage name](https://about.gitlab.com/handbook/product/categories/#devops-stages) +1. Inside the `Rspec.describe` block is a `describe` block with the name of the feature being tested +1. Optional `context` blocks define what the conditions being tested are + 1. `context` blocks descriptions should begin with `when`, `with`, `without`, `for`, `and`, `on`, `in`, `as`, or `if` to match the [rubocop rule](https://www.rubydoc.info/gems/rubocop-rspec/RuboCop/Cop/RSpec/ContextWording) +1. The `it` block describes the pass/fail criteria for the test + 1. In `shared_examples` with a single example a `specify` block can be used instead of a named `it` block + ## Prefer API over UI The end-to-end testing framework has the ability to fabricate its resources on a case-by-case basis. diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md index e473b158087..228f63d2354 100644 --- a/doc/development/testing_guide/end_to_end/feature_flags.md +++ b/doc/development/testing_guide/end_to_end/feature_flags.md @@ -55,18 +55,16 @@ RSpec.describe "with feature flag enabled", feature_flag: { let(:project) { Resource::Project.fabricate_via_api! } - before do + around do |example| Runtime::Feature.enable(:feature_flag_name, project: project) + example.run + Runtime::Feature.disable(:feature_flag_name, project: project) end it "feature flag test" do # Execute the test with the feature flag enabled. # It will only affect the project created in this test. end - - after do - Runtime::Feature.disable(:feature_flag_name, project: project) - end end ``` diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index b698eb53f75..f13686981a9 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -15,7 +15,7 @@ This is a partial list of the [RSpec metadata](https://rspec.info/features/3-12/ |-----------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `:elasticsearch` | The test requires an Elasticsearch service. It is used by the [instance-level scenario](https://gitlab.com/gitlab-org/gitlab-qa#definitions) [`Test::Integration::Elasticsearch`](https://gitlab.com/gitlab-org/gitlab/-/blob/72b62b51bdf513e2936301cb6c7c91ec27c35b4d/qa/qa/ee/scenario/test/integration/elasticsearch.rb) to include only tests that require Elasticsearch. | | `:except` | The test is to be run in their typical execution contexts _except_ as specified. See [test execution context selection](execution_context_selection.md) for more information. | -| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Pre-production. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. | +| `:feature_flag` | The test uses a feature flag and therefore requires an administrator account to run. When `scope` is set to `:global`, the test will be skipped on all live .com environments. Otherwise, it will be skipped only on Canary, Production, and Pre-production. See [testing with feature flags](../../../development/testing_guide/end_to_end/feature_flags.md) for more details. | | `:framework` | The test makes sanity assertions around the QA framework itself | | `:geo` | The test requires two GitLab Geo instances - a primary and a secondary - to be spun up. | | `:gitaly_cluster` | The test runs against a GitLab instance where repositories are stored on redundant Gitaly nodes behind a Praefect node. All nodes are [separate containers](../../../administration/gitaly/praefect.md#requirements). Tests that use this tag have a longer setup time since there are three additional containers that need to be started. | @@ -33,10 +33,11 @@ This is a partial list of the [RSpec metadata](https://rspec.info/features/3-12/ | `:mattermost` | The test requires a GitLab Mattermost service on the GitLab instance. | | `:metrics` | The test requires a GitLab instance where [dedicated metrics exporters](../../../administration/monitoring/prometheus/web_exporter.md) are running alongside Puma and Sidekiq. | | `:mixed_env` | The test should only be executed in environments that have a paired canary version available through traffic routing based on the existence of the `gitlab_canary=true` cookie. Tests in this category are switching the cookie mid-test to validate mixed deployment environments. | +| `:oauth` | The test uses and external OmniAuth provider to log in to GitLab | | `:object_storage` | The test requires a GitLab instance to be configured to use multiple [object storage types](../../../administration/object_storage.md). Uses MinIO as the object storage server. | | `:only` | The test is only to be run in specific execution contexts. See [test execution context selection](execution_context_selection.md) for more information. | | `:orchestrated` | The GitLab instance under test may be [configured by `gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/docs/what_tests_can_be_run.md#orchestrated-tests) to be different to the default GitLab configuration, or `gitlab-qa` may launch additional services in separate Docker containers, or both. Tests tagged with `:orchestrated` are excluded when testing environments where we can't dynamically modify the GitLab configuration (for example, Staging). | | -| `:product_group` | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories/) for the comprehensive groups list. | +| `:product_group` | Specifies what product group the test belongs to. See [Product sections, stages, groups, and categories](https://about.gitlab.com/handbook/product/categories/) for the comprehensive groups list. | | `:quarantine` | The test has been [quarantined](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/debugging-qa-test-failures/#quarantining-tests), runs in a separate job that only includes quarantined tests, and is allowed to fail. The test is skipped in its regular job so that if it fails it doesn't hold up the pipeline. Note that you can also [quarantine a test only when it runs in a specific context](execution_context_selection.md#quarantine-a-test-for-a-specific-environment). | | `:relative_url` | The test requires a GitLab instance to be installed under a [relative URL](../../../install/relative_url.md). | | `:reliable` | The test has been [promoted to a reliable test](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/reliable-tests/#promoting-an-existing-test-to-reliable) meaning it passes consistently in all pipelines, including merge requests. | diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md index 32d8bf339ed..f5a3fa2fc51 100644 --- a/doc/development/testing_guide/end_to_end/style_guide.md +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w This document describes the conventions used at GitLab for writing End-to-end (E2E) tests using the GitLab QA project. -Please note that this guide is an extension of the primary [testing standards and style guidelines](../index.md). If this guide defines a rule that contradicts the primary guide, this guide takes precedence. +This guide is an extension of the primary [testing standards and style guidelines](../index.md). If this guide defines a rule that contradicts the primary guide, this guide takes precedence. ## `click_` versus `go_to_` diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index 9114ec3d179..2d36377967e 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -63,7 +63,7 @@ difficult to achieve locally. Ordering issues are easier to reproduce by repeate `master` if the order of tests changes. - [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91016/diffs): A test asserts that trying to find a record with an nonexistent ID returns an error message. The test uses an - hardcoded ID that's supposed to not exist (e.g. `42`). If the test is run early in the test + hardcoded ID that's supposed to not exist (for example, `42`). If the test is run early in the test suite, it might pass as not enough records were created before it, but as soon as it would run later in the suite, there could be a record that actually has the ID `42`, hence the test would start to fail. @@ -185,7 +185,7 @@ After the long-term quarantining MR has reached production, you should revert th For Jest specs, you can use the `.skip` method along with the `eslint-disable-next-line` comment to disable the `jest/no-disabled-tests` ESLint rule and include the issue URL. Here's an example: ```javascript -// https://gitlab.com/gitlab-org/gitlab/-/issues/56789 +// quarantine: https://gitlab.com/gitlab-org/gitlab/-/issues/56789 // eslint-disable-next-line jest/no-disabled-tests it.skip('should throw an error', () => { expect(response).toThrowError(expected_error) @@ -198,6 +198,12 @@ This means it is skipped unless the test suit is run with `--runInBand` Jest com jest --runInBand ``` +A list of files with quarantined specs in them can be found with the command: + +```shell +yarn jest:quarantine +``` + For both test frameworks, make sure to add the `~"quarantined test"` label to the issue. Once a test is in quarantine, there are 3 choices: @@ -210,10 +216,10 @@ Once a test is in quarantine, there are 3 choices: ## Automatic retries and flaky tests detection -On our CI, we use [RSpec::Retry](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few +On our CI, we use [`RSpec::Retry`](https://github.com/NoRedInk/rspec-retry) to automatically retry a failing example a few times (see [`spec/spec_helper.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/spec_helper.rb) for the precise retries count). -We also use a custom [`RspecFlaky::Listener`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/tooling/rspec_flaky/listener.rb). +We also use a custom [`RspecFlaky::Listener`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/rspec_flaky/lib/rspec_flaky/listener.rb). This listener runs in the `update-tests-metadata` job in `maintenance` scheduled pipelines on the `master` branch, and saves flaky examples to `rspec/flaky/report-suite.json`. The report file is then retrieved by the `retrieve-tests-metadata` job in all pipelines. diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index b1fde5ed139..1b50bd410c2 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -51,17 +51,28 @@ The default timeout for Jest is set in If your test exceeds that time, it fails. If you cannot improve the performance of the tests, you can increase the timeout -for a specific test using [`jest.setTimeout`](https://jestjs.io/docs/27.x/jest-object#jestsettimeouttimeout) +for the whole suite using [`jest.setTimeout`](https://jestjs.io/docs/28.x/jest-object#jestsettimeouttimeout) ```javascript +jest.setTimeout(500); + describe('Component', () => { it('does something amazing', () => { - jest.setTimeout(500); // ... }); }); ``` +or for a specific test by providing a third argument to [`it`](https://jestjs.io/docs/28.x/api#testname-fn-timeout) + +```javascript +describe('Component', () => { + it('does something amazing', () => { + // ... + }, 500) +}) +``` + Remember that the performance of each test depends on the environment. ### Test-specific stylesheets @@ -540,7 +551,7 @@ describe('when logged in', () => { ### Ensuring that tests are isolated -Tests are normally architected in a pattern which requires a recurring setup of the component under test. This is often achieved by making use of the `beforeEach` hook. +Tests are typically architected in a pattern which requires a recurring setup of the component under test. This is often achieved by making use of the `beforeEach` hook. Example @@ -1225,7 +1236,7 @@ You can download any older version of Firefox from the releases FTP server, <htt 1. Rename the application to something like `Firefox_Old`. 1. Move the application to the `Applications` folder. 1. Open up a terminal and run `/Applications/Firefox_Old.app/Contents/MacOS/firefox-bin -profilemanager` to create a new profile specific to that Firefox version. -1. Once the profile has been created, quit the app, and run it again like normal. You now have a working older Firefox version. +1. Once the profile has been created, quit the app, and run it again like usual. You now have a working older Firefox version. ## Snapshots @@ -1742,7 +1753,7 @@ If you are stubbing an `ee` feature flag, then use: You can run your spec with the prefix `WEBDRIVER_HEADLESS=0` to open an actual browser. However, the specs goes though the commands quickly and leaves you no time to look around. -To avoid this problem, you can write `binding.pry` on the line where you want Capybara to stop execution. You are then inside the browser with normal usage. To understand why you cannot find certain elements, you can: +To avoid this problem, you can write `binding.pry` on the line where you want Capybara to stop execution. You are then inside the browser with standard usage. To understand why you cannot find certain elements, you can: - Select elements. - Use the console and network tab. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index 1e5ee9f3003..b4ae23336d5 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -204,10 +204,10 @@ subgraph "CNG-mirror pipeline" - If the `review-deploy` job keeps failing (and a manual retry didn't help), please post a message in the `#g_qe_engineering_productivity` channel and/or create a `~"Engineering Productivity"` `~"ep::review apps"` `~"type::bug"` - issue with a link to your merge request. Note that the deployment failure can + issue with a link to your merge request. The deployment failure can reveal an actual problem introduced in your merge request (that is, this isn't necessarily a transient failure)! -- If the `review-qa-smoke` or `review-qa-reliable` job keeps failing (note that we already retry them once), +- If the `review-qa-smoke` or `review-qa-reliable` job keeps failing (we already retry them once), please check the job's logs: you could discover an actual problem introduced in your merge request. You can also download the artifacts to see screenshots of the page at the time the failures occurred. If you don't find the cause of the diff --git a/doc/development/testing_guide/test_results_tracking.md b/doc/development/testing_guide/test_results_tracking.md index 28407e806a3..92c1e0917c7 100644 --- a/doc/development/testing_guide/test_results_tracking.md +++ b/doc/development/testing_guide/test_results_tracking.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w We developed the [`gitlab_quality-test_tooling`](https://gitlab.com/gitlab-org/ruby/gems/gitlab_quality-test_tooling) gem that includes several commands to automate test results tracking. -The goal of this gem is to have a consolidated set of tooling that we use accross our various test suite (e.g. GitLab Rails & E2E test suites). +The goal of this gem is to have a consolidated set of tooling that we use across our various test suite (for example, GitLab Rails & E2E test suites). The initial motivation and development was tracked by [this epic](https://gitlab.com/groups/gitlab-org/-/epics/10536). diff --git a/doc/development/value_stream_analytics.md b/doc/development/value_stream_analytics.md index e056ce9b064..30244407e38 100644 --- a/doc/development/value_stream_analytics.md +++ b/doc/development/value_stream_analytics.md @@ -41,8 +41,8 @@ feature-full. |Total time chart|Yes|No|No| |Task by type chart|Yes|No|No| |DORA Metrics|Yes|Yes|No| -|Cycle time and lead time summary (Key metrics)|Yes|Yes|No| -|New issues, commits and deploys (Key metrics)|Yes, excluding commits|Yes|Yes| +|Cycle time and lead time summary (Lifecycle metrics)|Yes|Yes|No| +|New issues, commits and deploys (Lifecycle metrics)|Yes, excluding commits|Yes|Yes| |Uses aggregated backend|Yes|No|No| |Date filter behavior|Filters items [finished within the date range](https://gitlab.com/groups/gitlab-org/-/epics/6046)|Filters items by creation date.|Filters items by creation date.| |Authorization|At least reporter|At least reporter|Can be public.| diff --git a/doc/development/work_items_widgets.md b/doc/development/work_items_widgets.md index 6c453f49db1..2e5a09af911 100644 --- a/doc/development/work_items_widgets.md +++ b/doc/development/work_items_widgets.md @@ -123,20 +123,20 @@ Because information about what widgets are assigned to each work item type is st ### Structure of widget definitions table -Each record in the table defines mapping of a widget to a work item type. Currently only "global" definitions (definitions with NULL namespace_id) are used. In next iterations we plan to allow customization of these mappings. For example table below defines that: +Each record in the table defines mapping of a widget to a work item type. Currently only "global" definitions (definitions with NULL `namespace_id`) are used. In next iterations we plan to allow customization of these mappings. For example table below defines that: - Weight widget is enabled for work item types 0 and 1 - in namespace 1 Weight widget is renamed to MyWeight. When user renames widget's name, it makes sense to rename all widget mappings in the namespace - because `name` attribute is denormalized, we have to create namespaced mappings for all work item types for this widget type. - Weight widget can be disabled for specific work item types (in namespace 3 it's disabled for work item type 0, while still left enabled for work item type 1) -| ID | Namespace_id | Work_item_type_id | Widget_type_enum | Position | Name | Disabled | -|----| ------------ | ----------------- |----------------- |--------- |---------- |-------| -| 1 | | 0 | 1 | 1 | Weight | false | -| 2 | | 1 | 1 | 1 | Weight | false | -| 3 | 1 | 0 | 1 | 0 | MyWeight | false | -| 4 | 1 | 1 | 1 | 0 | MyWeight | false | -| 5 | 2 | 0 | 1 | 1 | Other Weight | false | -| 6 | 3 | 0 | 1 | 1 | Weight | true | +| ID | `namespace_id` | `work_item_type_id` | `widget_type_enum` | Position | Name | Disabled | +|:---|:---------------|:--------------------|:-------------------|:---------|:-------------|:---------| +| 1 | | 0 | 1 | 1 | Weight | false | +| 2 | | 1 | 1 | 1 | Weight | false | +| 3 | 1 | 0 | 1 | 0 | MyWeight | false | +| 4 | 1 | 1 | 1 | 0 | MyWeight | false | +| 5 | 2 | 0 | 1 | 1 | Other Weight | false | +| 6 | 3 | 0 | 1 | 1 | Weight | true | ## Backend architecture diff --git a/doc/development/workhorse/channel.md b/doc/development/workhorse/channel.md index f5693b57f7a..2c28cea42a3 100644 --- a/doc/development/workhorse/channel.md +++ b/doc/development/workhorse/channel.md @@ -37,7 +37,7 @@ Sec-WebSocket-Protocol: terminal.gitlab.com ``` At this point, the connection is still HTTP, so this is a request. -The server can send a normal HTTP response, such as `404 Not Found` or +The server can send a standard HTTP response, such as `404 Not Found` or `500 Internal Server Error`. If the server decides to permit the upgrade, it sends a HTTP @@ -116,7 +116,7 @@ contain ANSI terminal control codes, and may be in any encoding. ## Workhorse to GitLab Using the terminal as an example, before upgrading the browser, -Workhorse sends a normal HTTP request to GitLab on a URL like +Workhorse sends a standard HTTP request to GitLab on a URL like `https://gitlab.com/group/project/environments/1/terminal.ws/authorize`. This returns a JSON response containing details of where the terminal can be found, and how to connect it. In particular, diff --git a/doc/development/workhorse/configuration.md b/doc/development/workhorse/configuration.md index e69f16c41f8..84fb557d9ec 100644 --- a/doc/development/workhorse/configuration.md +++ b/doc/development/workhorse/configuration.md @@ -53,9 +53,9 @@ Options: -logFormat string Log format to use defaults to text (text, json, structured, none) (default "text") -pprofListenAddr string - pprof listening address, e.g. 'localhost:6060' + pprof listening address, for example, 'localhost:6060' -prometheusListenAddr string - Prometheus listening address, e.g. 'localhost:9229' + Prometheus listening address, for example, 'localhost:9229' -propagateCorrelationID X-Request-ID Reuse existing Correlation-ID from the incoming request header X-Request-ID if present -proxyHeadersTimeout duration diff --git a/doc/gitlab-basics/feature_branch_workflow.md b/doc/gitlab-basics/feature_branch_workflow.md index 65445c226ca..ed84d584f81 100644 --- a/doc/gitlab-basics/feature_branch_workflow.md +++ b/doc/gitlab-basics/feature_branch_workflow.md @@ -34,6 +34,6 @@ To merge changes from a local branch to a feature branch, follow this workflow. git push origin feature_name ``` -1. Review your code: On the left sidebar, go to **Repository > Commits**. +1. Review your code: On the left sidebar, go to **Code > Commits**. 1. [Create a merge request](../user/project/merge_requests/creating_merge_requests.md). 1. Your team lead reviews the code and merges it to the main branch. diff --git a/doc/index.md b/doc/index.md index 367f18ec159..2c90d2f0970 100644 --- a/doc/index.md +++ b/doc/index.md @@ -38,8 +38,8 @@ Have a look at some of our most popular topics: | [Two-factor authentication](user/profile/account/two_factor_authentication.md) | Improve the security of your GitLab account. | | [GitLab groups](user/group/index.md) | Manage projects together. | | [Keyword reference for the `.gitlab-ci.yml` file](ci/yaml/index.md) | Available configuration options for `.gitlab-ci.yml` files. | -| [Activate GitLab EE with a license](user/admin_area/license.md) | Activate GitLab Enterprise Edition functionality with a license. | -| [Back up and restore GitLab](raketasks/backup_restore.md) | Rake tasks for backing up and restoring GitLab self-managed instances. | +| [Activate GitLab EE with a license](administration/license.md) | Activate GitLab Enterprise Edition functionality with a license. | +| [Back up and restore GitLab](administration/backup_restore/index.md) | Backing up and restoring GitLab self-managed instances. | | [GitLab release and maintenance policy](policy/maintenance.md) | Policies for version naming and cadence, and also upgrade recommendations. | | [Elasticsearch integration](integration/advanced_search/elasticsearch.md) | Integrate Elasticsearch with GitLab to enable advanced search. | | [Database settings for Linux package installations](https://docs.gitlab.com/omnibus/settings/database.html) | Database settings for self-managed instances installed using Linux packages. | diff --git a/doc/install/aws/gitlab_hybrid_on_aws.md b/doc/install/aws/gitlab_hybrid_on_aws.md index 75b39a387dc..b4d55e30ab1 100644 --- a/doc/install/aws/gitlab_hybrid_on_aws.md +++ b/doc/install/aws/gitlab_hybrid_on_aws.md @@ -157,8 +157,8 @@ On Demand pricing is used in this table for comparisons, but should not be used | Sidekiq | 2 vCPU, 8 GB | | | | Supporting services such as NGINX, Prometheus, etc | 2 vCPU, 8 GB | | | | **GitLab Ref Arch Raw Total K8s Node Capacity** | 16 vCPU, 32 GB | | | -| One Node for Overhead and Miscellaneous (EKS Cluster AutoScaler, Grafana, Prometheus, etc) | + 8 vCPU, 16GB | | | -| **Grand Total w/ Overheads**<br />Minimum hosts = 3 | 24 vCPU, 48 GB | **c5.2xlarge** <br />(8vCPU/16GB) x 3 nodes<br />24 vCPU, 48 GB | $1.02/hr | +| One Node for Overhead and Miscellaneous (EKS Cluster AutoScaler, Grafana, Prometheus, etc) | + 8 vCPU, 16 GB | | | +| **Grand Total w/ Overheads**<br />Minimum hosts = 3 | 24 vCPU, 48 GB | **c5.2xlarge** <br />(8vCPU/16 GB) x 3 nodes<br />24 vCPU, 48 GB | $1.02/hr | | **Idle Configuration (Scaled-In)** | 16 vCPU, 32 GB | **c5.2xlarge** x 2 | $0.68/hr | NOTE: @@ -168,7 +168,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------- | ------------------------------- | ------------------------------- | | **Bastion Host (Quick Start)** | 1 HA instance in ASG | **t2.micro** for prod, **m4.2xlarge** for performance testing | | | | **PostgreSQL**<br />AWS Amazon RDS PostgreSQL Nodes Configuration (GPT tested) | 2vCPU, 7.5 GB<br />Tested with Graviton ARM | **db.r6g.large** x 3 nodes <br />(6vCPU, 48 GB) | 3 nodes x $0.26 = $0.78/hr | 3 nodes x $0.26 = $0.78/hr | -| **Redis** | 1vCPU, 3.75GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m6g.large** x 3 nodes<br />(6vCPU, 19GB) | 3 nodes x $0.15 = $0.45/hr | 2 nodes x $0.15 = $0.30/hr | +| **Redis** | 1vCPU, 3.75GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m6g.large** x 3 nodes<br />(6vCPU, 19 GB) | 3 nodes x $0.15 = $0.45/hr | 2 nodes x $0.15 = $0.30/hr | | **<u>Gitaly Cluster</u>** [Details](gitlab_sre_for_aws.md#gitaly-sre-considerations) | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | | | | | Gitaly Instances (in ASG) | 12 vCPU, 45GB<br />(across 3 nodes) | **m5.xlarge** x 3 nodes<br />(48 vCPU, 180 GB) | $0.192 x 3 = $0.58/hr | $0.192 x 3 = $0.58/hr | | | The GitLab Reference architecture for 2K is not Highly Available and therefore has a single Gitaly no Praefect. AWS Quick Starts MUST be HA, so it implements Praefect from the 3K Ref Architecture to meet that requirement | | | | @@ -186,7 +186,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will - [3K AutoScale from 25% GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/3K/3k-QuickStart-AutoScale-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_194200/3k-QuickStart-AutoScale-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_194200_results.txt) - Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven my normal production workloads. + Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven by standard production workloads. **Deploy Now** @@ -211,7 +211,7 @@ On Demand pricing is used in this table for comparisons, but should not be used | Supporting services such as NGINX, Prometheus, etc | [2 allocations](../../administration/reference_architectures/3k_users.md#cluster-topology) x ([2 vCPU and 7.5 GB](../../administration/reference_architectures/3k_users.md#cluster-topology)) = <br />4 vCPU, 15 GB | | | | **GitLab Ref Arch Raw Total K8s Node Capacity** | 32 vCPU, 56 GB | | | | One Node for Overhead and Miscellaneous (EKS Cluster AutoScaler, Grafana, Prometheus, etc) | + 16 vCPU, 32GB | | | -| **Grand Total w/ Overheads Full Scale**<br />Minimum hosts = 3 | 48 vCPU, 88 GB | **c5.2xlarge** (8vCPU/16GB) x 5 nodes<br />40 vCPU, 80 GB<br />[Full Fixed Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/3K/3k-QuickStart-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_124216/3k-QuickStart-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_124216_results.txt) | $1.70/hr | +| **Grand Total w/ Overheads Full Scale**<br />Minimum hosts = 3 | 48 vCPU, 88 GB | **c5.2xlarge** (8vCPU/16 GB) x 5 nodes<br />40 vCPU, 80 GB<br />[Full Fixed Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/3K/3k-QuickStart-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_124216/3k-QuickStart-ARM-RDS-Cache_v13-12-3-ee_2021-07-23_124216_results.txt) | $1.70/hr | | **Possible Idle Configuration (Scaled-In 75% - round up)**<br />Pod autoscaling must be also adjusted to enable lower idling configuration. | 24 vCPU, 48 GB | c5.2xlarge x 4 | $1.36/hr | Other combinations of node type and quantity can be used to meet the Grand Total. Due to the properties of pods, hosts that are overly small may have significant unused capacity. @@ -223,7 +223,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------ | | **Bastion Host (Quick Start)** | 1 HA instance in ASG | **t2.micro** for prod, **m4.2xlarge** for performance testing | | | | **PostgreSQL**<br />Amazon RDS PostgreSQL Nodes Configuration (GPT tested) | 18vCPU, 36 GB <br />(across 9 nodes for PostgreSQL, PgBouncer, Consul)<br />Tested with Graviton ARM | **db.r6g.xlarge** x 3 nodes <br />(12vCPU, 96 GB) | 3 nodes x $0.52 = $1.56/hr | 3 nodes x $0.52 = $1.56/hr | -| **Redis** | 6vCPU, 18GB<br />(across 6 nodes for Redis Cache, Sentinel) | **cache.m6g.large** x 3 nodes<br />(6vCPU, 19GB) | 3 nodes x $0.15 = $0.45/hr | 2 nodes x $0.15 = $0.30/hr | +| **Redis** | 6vCPU, 18 GB<br />(across 6 nodes for Redis Cache, Sentinel) | **cache.m6g.large** x 3 nodes<br />(6vCPU, 19 GB) | 3 nodes x $0.15 = $0.45/hr | 2 nodes x $0.15 = $0.30/hr | | **<u>Gitaly Cluster</u>** [Details](gitlab_sre_for_aws.md#gitaly-sre-considerations) | | | | | | Gitaly Instances (in ASG) | 12 vCPU, 45GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | **m5.large** x 3 nodes<br />(12 vCPU, 48 GB) | $0.192 x 3 = $0.58/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | | Praefect (Instances in ASG with load balancer) | 6 vCPU, 5.4 GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | **c5.large** x 3 nodes<br />(6 vCPU, 12 GB) | $0.09 x 3 = $0.21/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | @@ -240,7 +240,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will - [5K AutoScale from 25% GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/5K/5k-QuickStart-AutoScale-From-25Percent-ARM-RDS-Redis_v13-12-3-ee_2021-07-24_102717/5k-QuickStart-AutoScale-From-25Percent-ARM-RDS-Redis_v13-12-3-ee_2021-07-24_102717_results.txt) - Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven my normal production workloads. + Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven by standard production workloads. **Deploy Now** @@ -264,8 +264,8 @@ On Demand pricing is used in this table for comparisons, but should not be used | Sidekiq | [8 pods](https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/examples/ref/5k.yaml#L24) x ([1 vCPU & 2 GB](../../administration/reference_architectures/5k_users.md#sidekiq)) = <br />8 vCPU, 16 GB | | | | Supporting services such as NGINX, Prometheus, etc | [2 allocations](../../administration/reference_architectures/5k_users.md#cluster-topology) x ([2 vCPU and 7.5 GB](../../administration/reference_architectures/5k_users.md#cluster-topology)) = <br />4 vCPU, 15 GB | | | | **GitLab Ref Arch Raw Total K8s Node Capacity** | 62 vCPU, 96.5 GB | | | -| One Node for Quick Start Overhead and Miscellaneous (EKS Cluster AutoScaler, Grafana, Prometheus, etc) | + 8 vCPU, 16GB | | | -| **Grand Total w/ Overheads Full Scale**<br />Minimum hosts = 3 | 70 vCPU, 112.5 GB | **c5.2xlarge** (8vCPU/16GB) x 9 nodes<br />72 vCPU, 144 GB<br />[Full Fixed Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/5K/5k-QuickStart-ARM-RDS-Redis_v13-12-3-ee_2021-07-23_140128/5k-QuickStart-ARM-RDS-Redis_v13-12-3-ee_2021-07-23_140128_results.txt) | $2.38/hr | +| One Node for Quick Start Overhead and Miscellaneous (EKS Cluster AutoScaler, Grafana, Prometheus, etc) | + 8 vCPU, 16 GB | | | +| **Grand Total w/ Overheads Full Scale**<br />Minimum hosts = 3 | 70 vCPU, 112.5 GB | **c5.2xlarge** (8vCPU/16 GB) x 9 nodes<br />72 vCPU, 144 GB<br />[Full Fixed Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/5K/5k-QuickStart-ARM-RDS-Redis_v13-12-3-ee_2021-07-23_140128/5k-QuickStart-ARM-RDS-Redis_v13-12-3-ee_2021-07-23_140128_results.txt) | $2.38/hr | | **Possible Idle Configuration (Scaled-In 75% - round up)**<br />Pod autoscaling must be also adjusted to enable lower idling configuration. | 24 vCPU, 48 GB | c5.2xlarge x 7 | $1.85/hr | Other combinations of node type and quantity can be used to meet the Grand Total. Due to the CPU and memory requirements of pods, hosts that are overly small may have significant unused capacity. @@ -294,7 +294,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will - [10K Elastic Auto Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/10K/GL-CloudNative-10k-AutoScaling-Test_v13-12-3-ee_2021-07-09_115139/GL-CloudNative-10k-AutoScaling-Test_v13-12-3-ee_2021-07-09_115139_results.txt) - Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven my normal production workloads. + Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven by standard production workloads. **Deploy Now** @@ -330,9 +330,9 @@ If EKS node autoscaling is employed, it is likely that your average loading will | ------------------------------------------------------------ | ------------------------------ | ------------------------------- | ------------------------------------------------------------ | ------------------------------------------------------------ | | **Bastion Host (Quick Start)** | 1 HA instance in ASG | **t2.micro** for prod, **m4.2xlarge** for performance testing | | | | **PostgreSQL**<br />Amazon RDS PostgreSQL Nodes Configuration (GPT tested) | 36vCPU, 102 GB <br />(across 9 nodes for PostgreSQL, PgBouncer, Consul) | **db.r6g.2xlarge** x 3 nodes <br />(24vCPU, 192 GB) | 3 nodes x $1.04 = $3.12/hr | 3 nodes x $1.04 = $3.12/hr | -| **Redis** | 30vCPU, 114GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m5.2xlarge** x 3 nodes<br />(24vCPU, 78GB) | 3 nodes x $0.62 = $1.86/hr | 2 nodes x $0.62 = $1.24/hr | +| **Redis** | 30vCPU, 114 GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m5.2xlarge** x 3 nodes<br />(24vCPU, 78GB) | 3 nodes x $0.62 = $1.86/hr | 2 nodes x $0.62 = $1.24/hr | | **<u>Gitaly Cluster</u>** [Details](gitlab_sre_for_aws.md#gitaly-sre-considerations) | | | | | -| Gitaly Instances (in ASG) | 48 vCPU, 180GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | **m5.4xlarge** x 3 nodes<br />(48 vCPU, 180 GB) | $0.77 x 3 = $2.31/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | +| Gitaly Instances (in ASG) | 48 vCPU, 180 GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | **m5.4xlarge** x 3 nodes<br />(48 vCPU, 180 GB) | $0.77 x 3 = $2.31/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | | Praefect (Instances in ASG with load balancer) | 6 vCPU, 5.4 GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | **c5.large** x 3 nodes<br />(6 vCPU, 12 GB) | $0.09 x 3 = $0.21/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | | Praefect PostgreSQL(1) (Amazon RDS) | 6 vCPU, 5.4 GB<br />([across 3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections)) | Not applicable; reuses GitLab PostgreSQL | $0 | | | Internal Load Balancing Node | 2 vCPU, 1.8 GB | AWS ELB | $0.10/hr | $0.10/hr | @@ -347,7 +347,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will - [50K Elastic Auto Scale GPT Test Results](https://gitlab.com/guided-explorations/aws/implementation-patterns/gitlab-cloud-native-hybrid-on-eks/-/blob/master/gitlab-alliances-testing/50K/50k-AutoScale-Test_v13-12-3-ee_2021-08-13_192633/50k-AutoScale-Test_v13-12-3-ee_2021-08-13_192633.txt) - Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven my normal production workloads. + Elastic Auto Scale GPT Test Results start with an idle scaled cluster and then start the standard GPT test to determine if the EKS Auto Scaler performs well enough to keep up with performance test demands. In general this is substantially harder ramping than the scaling required when the ramping is driven by standard production workloads. **Deploy Now** @@ -383,7 +383,7 @@ If EKS node autoscaling is employed, it is likely that your average loading will | ------------------------------------------------------------ | ------------------------------------------------------------ | --------------------------------------------------------- | ------------------------------- | ------------------------------------------------------------ | | **Bastion Host (Quick Start)** | 1 HA instance in ASG | **t2.micro** for prod, **m4.2xlarge** for performance testing | | | | **PostgreSQL**<br />Amazon RDS PostgreSQL Nodes Configuration (GPT tested) | 96vCPU, 360 GB <br />(across 3 nodes) | **db.r6g.8xlarge** x 3 nodes <br />(96vCPU, 768 GB total) | 3 nodes x $4.15 = $12.45/hr | 3 nodes x $4.15 = $12.45/hr | -| **Redis** | 30vCPU, 114GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m6g.2xlarge** x 3 nodes<br />(24vCPU, 78GB total) | 3 nodes x $0.60 = $1.80/hr | 2 nodes x $0.60 = $1.20/hr | +| **Redis** | 30vCPU, 114 GB<br />(across 12 nodes for Redis Cache, Redis Queues/Shared State, Sentinel Cache, Sentinel Queues/Shared State) | **cache.m6g.2xlarge** x 3 nodes<br />(24vCPU, 78GB total) | 3 nodes x $0.60 = $1.80/hr | 2 nodes x $0.60 = $1.20/hr | | **<u>Gitaly Cluster</u>** [Details](gitlab_sre_for_aws.md#gitaly-sre-considerations) | | | | | | Gitaly Instances (in ASG) | 64 vCPU, 240GB x [3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | **m5.16xlarge** x 3 nodes<br />(64 vCPU, 256 GB each) | $3.07 x 3 = $9.21/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | | Praefect (Instances in ASG with load balancer) | 4 vCPU, 3.6 GB x [3 nodes](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | **c5.xlarge** x 3 nodes<br />(4 vCPU, 8 GB each) | $0.17 x 3 = $0.51/hr | [Gitaly & Praefect Must Have an Uneven Node Count for HA](gitlab_sre_for_aws.md#gitaly-and-praefect-elections) | diff --git a/doc/install/aws/gitlab_sre_for_aws.md b/doc/install/aws/gitlab_sre_for_aws.md index f957dfa8a65..512857b87b5 100644 --- a/doc/install/aws/gitlab_sre_for_aws.md +++ b/doc/install/aws/gitlab_sre_for_aws.md @@ -80,11 +80,11 @@ All recommendations are for production configurations, including performance tes ### AWS Gitaly backup -Due to the nature of how Praefect tracks the replication metadata of Gitaly disk information, the best backup method is [the official backup and restore Rake tasks](../../raketasks/backup_restore.md). +Due to the nature of how Praefect tracks the replication metadata of Gitaly disk information, the best backup method is [the official backup and restore Rake tasks](../../administration/backup_restore/index.md). ### AWS Gitaly recovery -Gitaly Cluster does not support snapshot backups as these can cause issues where the Praefect database becomes out of syn with the disk storage. Due to the nature of how Praefect rebuilds the replication metadata of Gitaly disk information during a restore, the best recovery method is [the official backup and restore Rake tasks](../../raketasks/backup_restore.md). +Gitaly Cluster does not support snapshot backups as these can cause issues where the Praefect database becomes out of syn with the disk storage. Due to the nature of how Praefect rebuilds the replication metadata of Gitaly disk information during a restore, the best recovery method is [the official backup and restore Rake tasks](../../administration/backup_restore/index.md). ### Gitaly HA in EKS quick start diff --git a/doc/install/aws/manual_install_aws.md b/doc/install/aws/manual_install_aws.md index 92375fff59e..92ef08c2447 100644 --- a/doc/install/aws/manual_install_aws.md +++ b/doc/install/aws/manual_install_aws.md @@ -268,7 +268,7 @@ On the EC2 dashboard, look for Load Balancer in the left navigation bar: 1. Select **Configure Health Check** and set up a health check for your EC2 instances. 1. For **Ping Protocol**, select HTTP. 1. For **Ping Port**, enter 80. - 1. For **Ping Path** - we recommend that you [use the Readiness check endpoint](../../administration/load_balancer.md#readiness-check). You must add [the VPC IP Address Range (CIDR)](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-groups.html#elb-vpc-nacl) to the [IP allowlist](../../administration/monitoring/ip_allowlist.md) for the [Health Check endpoints](../../user/admin_area/monitoring/health_check.md) + 1. For **Ping Path** - we recommend that you [use the Readiness check endpoint](../../administration/load_balancer.md#readiness-check). You must add [the VPC IP Address Range (CIDR)](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/elb-security-groups.html#elb-vpc-nacl) to the [IP allowlist](../../administration/monitoring/ip_allowlist.md) for the [Health Check endpoints](../../administration/monitoring/health_check.md) 1. Keep the default **Advanced Details** or adjust them according to your needs. 1. Select **Add EC2 Instances** - don't add anything as we create an Auto Scaling Group later to manage instances for us. 1. Select **Add Tags** and add any tags you need. @@ -741,7 +741,7 @@ GitLab provides its own integrated monitoring solution based on Prometheus. For more information about how to set it up, see [GitLab Prometheus](../../administration/monitoring/prometheus/index.md). -GitLab also has various [health check endpoints](../../user/admin_area/monitoring/health_check.md) +GitLab also has various [health check endpoints](../../administration/monitoring/health_check.md) that you can ping and get reports. ## GitLab Runner @@ -754,16 +754,16 @@ Read more on configuring an ## Backup and restore -GitLab provides [a tool to back up](../../raketasks/backup_restore.md) +GitLab provides [a tool to back up](../../administration/backup_restore/index.md) and restore its Git data, database, attachments, LFS objects, and so on. Some important things to know: - The backup/restore tool **does not** store some configuration files, like secrets; you - must [configure this yourself](../../raketasks/backup_gitlab.md#storing-configuration-files). + must [configure this yourself](../../administration/backup_restore/backup_gitlab.md#storing-configuration-files). - By default, the backup files are stored locally, but you can - [backup GitLab using S3](../../raketasks/backup_gitlab.md#using-amazon-s3). -- You can [exclude specific directories form the backup](../../raketasks/backup_gitlab.md#excluding-specific-directories-from-the-backup). + [backup GitLab using S3](../../administration/backup_restore/backup_gitlab.md#using-amazon-s3). +- You can [exclude specific directories form the backup](../../administration/backup_restore/backup_gitlab.md#excluding-specific-directories-from-the-backup). ### Backing up GitLab @@ -781,9 +781,9 @@ For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`. ### Restoring GitLab from a backup -To restore GitLab, first review the [restore documentation](../../raketasks/backup_restore.md#restore-gitlab), +To restore GitLab, first review the [restore documentation](../../administration/backup_restore/index.md#restore-gitlab), and primarily the restore prerequisites. Then, follow the steps under the -[Linux package installations section](../../raketasks/restore_gitlab.md#restore-for-omnibus-gitlab-installations). +[Linux package installations section](../../administration/backup_restore/restore_gitlab.md#restore-for-omnibus-gitlab-installations). ## Updating GitLab @@ -833,7 +833,7 @@ to request additional material: Geo is the solution for widely distributed development teams. - [Linux package](https://docs.gitlab.com/omnibus/) - Everything you must know about administering your GitLab instance. -- [Add a license](../../user/admin_area/license.md): +- [Add a license](../../administration/license.md): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. diff --git a/doc/install/docker.md b/doc/install/docker.md index ab1aec98b16..925da8a2b4a 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -282,10 +282,6 @@ Here's an example that deploys GitLab with four runners as a [stack](https://doc docker stack deploy --compose-file docker-compose.yml mystack ``` -### Install the product documentation - -This is an optional step. See how to [self-host the product documentation](../administration/docs_self_host.md#self-host-the-product-documentation-with-docker). - ## Configuration This container uses the official Omnibus GitLab package, so all configuration @@ -487,6 +483,12 @@ If, for any reason, you wish to switch back to single database connection: sudo docker restart gitlab ``` +## Recommended next steps + +After completing your installation, consider taking the +[recommended next steps](next_steps.md), including authentication options +and sign-up restrictions. + ## Upgrade In most cases, upgrading GitLab is as easy as downloading the newest Docker @@ -580,11 +582,6 @@ The following steps assume that you are upgrading the same version. replace `ce` with `ee` in your `docker run` command or `docker-compose.yml` file. However, reuse the CE container name, port and file mappings, and version. -### Upgrade the product documentation - -This is an optional step. If you [installed the documentation site](#install-the-product-documentation), -see how to [upgrade to another version](../administration/docs_self_host.md#upgrade-using-docker). - ### Downgrade GitLab To downgrade GitLab after an upgrade: @@ -596,7 +593,7 @@ To downgrade GitLab after an upgrade: - Restoring is required to back out database data and schema changes (migrations) made as part of the upgrade. - GitLab backups must be restored to the exact same version and edition. - - [Follow the restore steps for Docker images](../raketasks/restore_gitlab.md#restore-for-docker-image-and-gitlab-helm-chart-installations), including + - [Follow the restore steps for Docker images](../administration/backup_restore/restore_gitlab.md#restore-for-docker-image-and-gitlab-helm-chart-installations), including stopping Puma and Sidekiq. Only the database must be restored, so add `SKIP=artifacts,repositories,registry,uploads,builds,pages,lfs,packages,terraform_state` to the `gitlab-backup restore` command line arguments. @@ -609,7 +606,7 @@ You can create a GitLab backup with: docker exec -t <container name> gitlab-backup create ``` -Read more on how to [back up and restore GitLab](../raketasks/backup_restore.md). +Read more on how to [back up and restore GitLab](../administration/backup_restore/index.md). NOTE: If configuration is provided entirely via the `GITLAB_OMNIBUS_CONFIG` environment variable @@ -618,8 +615,8 @@ meaning no configuration is set directly in the `gitlab.rb` file, then there is to back up the `gitlab.rb` file. WARNING: -[Backing up the GitLab secrets file](../raketasks/backup_gitlab.md#storing-configuration-files) is required -to avoid [complicated steps](../raketasks/backup_restore.md#when-the-secrets-file-is-lost) when recovering +[Backing up the GitLab secrets file](../administration/backup_restore/backup_gitlab.md#storing-configuration-files) is required +to avoid [complicated steps](../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost) when recovering GitLab from backup. The secrets file is stored at `/etc/gitlab/gitlab-secrets.json` inside the container, or `$GITLAB_HOME/config/gitlab-secrets.json` [on the container host](#set-up-the-volumes-location). diff --git a/doc/install/installation.md b/doc/install/installation.md index 4d80a02c9f1..45c6b398a76 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -47,7 +47,7 @@ If the highest number stable branch is unclear, check the [GitLab blog](https:// |:------------------------|:----------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Ruby](#2-ruby) | `3.0.x` | From GitLab 15.10, Ruby 3.0 is required. You must use the standard MRI implementation of Ruby. We love [JRuby](https://www.jruby.org/) and [Rubinius](https://github.com/rubinius/rubinius#the-rubinius-language-platform), but GitLab needs several Gems that have native extensions. | | [RubyGems](#3-rubygems) | `3.4.x` | A specific RubyGems version is not fully needed, but it's recommended to update so you can enjoy some known performance improvements. | -| [Go](#4-go) | `1.18.x` | From GitLab 15.6, Go 1.18 or later is required. | +| [Go](#4-go) | `1.19.x` | From GitLab 16.1, Go 1.19 or later is required. | | [Git](#git) | `2.38.x` | From GitLab 15.8, Git 2.38.x and later is required. It's highly recommended that you use the [Git version provided by Gitaly](#git). | | [Node.js](#5-node) | `18.16.x` | From GitLab 16.1, Node.js 18.16 or later is required. | @@ -72,7 +72,7 @@ When following the instructions on this page, you create this directory structur - `/home/git/repositories` - Bare repositories for all projects organized by namespace. This is where the Git repositories which are pushed/pulled are maintained for all projects. **This area contains critical data for projects. - [Keep a backup](../raketasks/backup_restore.md).** + [Keep a backup](../administration/backup_restore/index.md).** The default locations for repositories can be configured in `config/gitlab.yml` of GitLab and `config.yml` of GitLab Shell. @@ -174,7 +174,7 @@ the Git path: ### GraphicsMagick -For the [Custom Favicon](../user/admin_area/appearance.md#favicon) to work, GraphicsMagick +For the [Custom Favicon](../administration/appearance.md#favicon) to work, GraphicsMagick must be installed. ```shell @@ -247,11 +247,11 @@ Linux. You can find downloads for other platforms at the # Remove former Go installation folder sudo rm -rf /usr/local/go -curl --remote-name --location --progress-bar "https://go.dev/dl/go1.18.8.linux-amd64.tar.gz" -echo '4d854c7bad52d53470cf32f1b287a5c0c441dc6b98306dea27358e099698142a go1.18.8.linux-amd64.tar.gz' | shasum -a256 -c - && \ - sudo tar -C /usr/local -xzf go1.18.8.linux-amd64.tar.gz +curl --remote-name --location --progress-bar "https://go.dev/dl/go1.19.10.linux-amd64.tar.gz" +echo '8b045a483d3895c6edba2e90a9189262876190dbbd21756870cdd63821810677 go1.19.10.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.19.10.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/ -rm go1.18.8.linux-amd64.tar.gz +rm go1.19.10.linux-amd64.tar.gz ``` ## 5. Node @@ -260,7 +260,7 @@ GitLab requires the use of Node to compile JavaScript assets, and Yarn to manage JavaScript dependencies. The current minimum requirements for these are: -- `node` 18.x releases (v18.16.0 or later). +- `node` 18.x releases (v18.16.1 or later). [Other LTS versions of Node.js](https://github.com/nodejs/release#release-schedule) might be able to build assets, but we only guarantee Node.js 18.x. - `yarn` = v1.22.x (Yarn 2 is not supported yet) @@ -1038,9 +1038,11 @@ To start and stop GitLab when using: - systemd units: use `sudo systemctl start gitlab.target` or `sudo systemctl stop gitlab.target`. - The SysV init script: use `sudo service gitlab start` or `sudo service gitlab stop`. -### Install the product documentation +### Recommended next steps -This is an optional step. See how to [self-host the product documentation](../administration/docs_self_host.md). +After completing your installation, consider taking the +[recommended next steps](next_steps.md), including authentication options +and sign-up restrictions. ## Advanced Setup Tips diff --git a/doc/install/migrate/compare_sm_to_saas.md b/doc/install/migrate/compare_sm_to_saas.md index a83c4a6865f..4cdc849be2c 100644 --- a/doc/install/migrate/compare_sm_to_saas.md +++ b/doc/install/migrate/compare_sm_to_saas.md @@ -17,14 +17,14 @@ In GitLab SaaS, administration tasks are limited compared to a self-managed appl In a self-managed instance: -- You have complete access and administrative control over the application, including the [Admin Area](../../user/admin_area/settings/index.md). +- You have complete access and administrative control over the application, including the [Admin Area](../../administration/settings/index.md). - You can impersonate, create, add, and remove users. - You can assign the [`Auditor`](../../administration/auditor_users.md) user type and `External` role. On GitLab SaaS: - You have limited administrative control. For example, you cannot impersonate, create, add, or remove users. -- You cannot access the [Admin Area](../../user/admin_area/settings/index.md). +- You cannot access the [Admin Area](../../administration/settings/index.md). - You cannot assign the `Auditor` user type and `External` role. ## Logs diff --git a/doc/install/next_steps.md b/doc/install/next_steps.md index 70b6101b1eb..ecc456cd3ec 100644 --- a/doc/install/next_steps.md +++ b/doc/install/next_steps.md @@ -40,7 +40,7 @@ installation. ## Backup and upgrade -- [Back up and restore GitLab](../raketasks/backup_restore.md): Learn the different +- [Back up and restore GitLab](../administration/backup_restore/index.md): Learn the different ways you can back up or restore GitLab. - [Upgrade GitLab](../update/index.md): Every 22nd of the month, a new feature-rich GitLab version is released. Learn how to upgrade to it, or to an interim release that contains a security fix. @@ -50,7 +50,7 @@ installation. ## License -- [Add a license](../user/admin_area/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): +- [Add a license](../administration/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. @@ -65,3 +65,8 @@ installation. GitLab supports several different types of clustering. - [Geo replication](../administration/geo/index.md): Geo is the solution for widely distributed development teams. + +## Install the product documentation + +This is an optional step. If you want to host the documentation on your own +server, see how to [self-host the product documentation](../administration/docs_self_host.md). diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 64dfd3e6044..5e13628e815 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -81,9 +81,10 @@ the following table) as these were used for development and testing: | GitLab version | Minimum PostgreSQL version | |----------------|----------------------------| | 13.0 | 11 | -| 14.0 | 12.7 | +| 14.0 | 12.7 | | 15.0 | 12.10 | | 16.0 | 13.6 | +| 17.0 (planned) | 14.8 | You must also ensure the following extensions are loaded into every GitLab database. [Read more about this requirement, and troubleshooting](postgresql_extensions.md). diff --git a/doc/integration/advanced_search/elasticsearch.md b/doc/integration/advanced_search/elasticsearch.md index 7b23eaa278a..cdda85bb259 100644 --- a/doc/integration/advanced_search/elasticsearch.md +++ b/doc/integration/advanced_search/elasticsearch.md @@ -79,10 +79,11 @@ To index Git repository data, GitLab uses an [indexer written in Go](https://git Depending on your GitLab version, there are different installation procedures for the Go indexer: -- For Omnibus GitLab 11.8 or greater, see [Omnibus GitLab](#omnibus-gitlab). -- For installations from source or older versions of Omnibus GitLab, +- For Omnibus GitLab 11.8 and later, see [Omnibus GitLab](#omnibus-gitlab). +- For installations from source or Omnibus GitLab 11.7 and earlier, [install the indexer from source](#from-source). -- If you are using GitLab Development Kit, see [GDK Elasticsearch how-to](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/elasticsearch.md). +- If you're using the GitLab Development Kit, see [Elasticsearch in the GDK](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/elasticsearch.md). +- If you're running a Helm deployment of GitLab 11.10 and later, [the indexer is already included](https://gitlab.com/gitlab-org/build/CNG/-/merge_requests/213). ### Omnibus GitLab @@ -178,7 +179,7 @@ To enable advanced search: NOTE: To see the **Advanced Search** section, you need an active GitLab Premium - [license](../../user/admin_area/license.md). + [license](../../administration/license.md). 1. Configure the [advanced search settings](#advanced-search-configuration) for your Elasticsearch cluster. Do not enable **Search with Elasticsearch enabled** @@ -186,8 +187,7 @@ To enable advanced search: 1. Enable **Elasticsearch indexing** and select **Save changes**. This creates an empty index if one does not already exist. 1. Select **Index all projects**. -1. Select **Check progress** in the confirmation message to see the status of - the background jobs. +1. Optional. Select **Check progress** to see the status of background jobs. 1. Personal snippets must be indexed using another Rake task: ```shell @@ -217,6 +217,7 @@ The following Elasticsearch settings are available: | `Elasticsearch indexing` | Enables or disables Elasticsearch indexing and creates an empty index if one does not already exist. You may want to enable indexing but disable search to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. | | `Pause Elasticsearch indexing` | Enables or disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until resumed. | | `Search with Elasticsearch enabled` | Enables or disables using Elasticsearch in search. | +| `Requeue indexing workers` | Enable automatic requeuing of indexing workers. This improves non-code indexing throughput by enqueuing Sidekiq jobs until all documents are processed. Requeuing indexing workers is not recommended for smaller instances or instances with few Sidekiq processes. | | `URL` | The URL of your Elasticsearch instance. Use a comma-separated list to support clustering (for example, `http://host1, https://host2:9200`). If your Elasticsearch instance is password-protected, use the `Username` and `Password` fields described below. Alternatively, use inline credentials such as `http://<username>:<password>@<elastic_host>:9200/`. | | `Username` | The `username` of your Elasticsearch instance. | | `Password` | The password of your Elasticsearch instance. | @@ -228,6 +229,7 @@ The following Elasticsearch settings are available: | `AWS Secret Access Key` | The AWS secret access key. | | `Maximum file size indexed` | See [the explanation in instance limits.](../../administration/instance_limits.md#maximum-file-size-indexed). | | `Maximum field length` | See [the explanation in instance limits.](../../administration/instance_limits.md#maximum-field-length). | +| `Number of shards for non-code indexing` | Number of indexing worker shards. This improves non-code indexing throughput by enqueuing more parallel Sidekiq jobs. Increasing the number of shards is not recommended for smaller instances or instances with few Sidekiq processes. Default is `2`. | | `Maximum bulk request size (MiB)` | Used by the GitLab Ruby and Go-based indexer processes. This setting indicates how much data must be collected (and stored in memory) in a given indexing process before submitting the payload to the Elasticsearch Bulk API. For the GitLab Go-based indexer, you should use this setting with `Bulk request concurrency`. `Maximum bulk request size (MiB)` must accommodate the resource constraints of both the Elasticsearch hosts and the hosts running the GitLab Go-based indexer from either the `gitlab-rake` command or the Sidekiq tasks. | | `Bulk request concurrency` | The Bulk request concurrency indicates how many of the GitLab Go-based indexer processes (or threads) can run in parallel to collect data to subsequently submit to the Elasticsearch Bulk API. This increases indexing performance, but fills the Elasticsearch bulk requests queue faster. This setting should be used together with the Maximum bulk request size setting (see above) and needs to accommodate the resource constraints of both the Elasticsearch hosts and the hosts running the GitLab Go-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | | `Client request timeout` | Elasticsearch HTTP client request timeout value in seconds. `0` means using the system default timeout value, which depends on the libraries that GitLab application is built upon. | diff --git a/doc/integration/advanced_search/elasticsearch_troubleshooting.md b/doc/integration/advanced_search/elasticsearch_troubleshooting.md index e8eace7bd16..e8634cf5ef9 100644 --- a/doc/integration/advanced_search/elasticsearch_troubleshooting.md +++ b/doc/integration/advanced_search/elasticsearch_troubleshooting.md @@ -256,6 +256,24 @@ Bulk requests getting rejected by the Elasticsearch nodes are likely due to load Ensure that your Elasticsearch cluster meets the [system requirements](elasticsearch.md#system-requirements) and has enough resources to perform bulk operations. See also the error ["429 (Too Many Requests)"](#indexing-fails-with-error-elastic-error-429-too-many-requests). +### Indexing fails with `strict_dynamic_mapping_exception` + +Indexing might fail if all [advanced search migrations were not finished before doing a major upgrade](elasticsearch.md#all-migrations-must-be-finished-before-doing-a-major-upgrade). +A large Sidekiq backlog might accompany this error. To fix the indexing failures, you must re-index the database, repositories, and wikis. + +1. Pause indexing so Sidekiq can catch up: + + ```shell + sudo gitlab-rake gitlab:elastic:pause_indexing + ``` + +1. [Recreate the index from scratch](#last-resort-to-recreate-an-index). +1. Resume indexing: + + ```shell + sudo gitlab-rake gitlab:elastic:resume_indexing + ``` + ### Last resort to recreate an index There may be cases where somehow data never got indexed and it's not in the @@ -469,7 +487,7 @@ $ jq --raw-output 'select(.severity == "ERROR") | [.error_class, .error_message] sort | uniq -c ``` -`Elastic` workers and [Sidekiq jobs](../../user/admin_area/index.md#background-jobs) could also appear much more often +`Elastic` workers and [Sidekiq jobs](../../administration/admin_area.md#background-jobs) could also appear much more often because Elasticsearch frequently attempts to reindex if a previous job fails. You can use [`fast-stats`](https://gitlab.com/gitlab-com/support/toolbox/fast-stats#usage) or `jq` to count workers in the [Sidekiq logs](../../administration/logs/index.md#sidekiq-logs): diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index b8cffc449ec..ef17b56d38f 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -6,24 +6,24 @@ info: To determine the technical writer assigned to the Stage/Group associated w # External issue trackers **(FREE)** -GitLab has an [issue tracker](../user/project/issues/index.md), but you can -configure an external issue tracker per GitLab project. +GitLab has its own [issue tracker](../user/project/issues/index.md), +but you can also configure an external issue tracker per GitLab project. +You can then use: -After you configure the external tracker, you can reference external issues -in GitLab merge requests, commits, and comments -using the format `CODE-123`, where: +- The external issue tracker only +- The external issue tracker with the GitLab issue tracker + +With an external tracker, you can use the format `CODE-123` to mention +external issues in GitLab merge requests, commits, and comments where: - `CODE` is a unique code for the tracker. - `123` is the issue number in the tracker. -The references are automatically converted to links to the issues. - -You can keep the GitLab issue tracker enabled in parallel or disable it. When enabled, the **Issues** link in the -GitLab menu always opens the internal issue tracker. When disabled, the link is not visible in the menu. +References are displayed as issue links. ## Disable the GitLab issue tracker -To disable the GitLab issue tracker: +To disable the GitLab issue tracker for a project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > General**. @@ -31,16 +31,16 @@ To disable the GitLab issue tracker: 1. Under **Issues**, turn off the toggle. 1. Select **Save changes**. -## Configure an external issue tracker +After you disable the GitLab issue tracker, **Issues** is not visible on the left sidebar of your project. -To enable an external issue tracker, you must configure the appropriate [integration](../user/project/integrations/index.md). +## Configure an external issue tracker -The following external issue tracker integrations are available: +You can configure any of the following external issue trackers: - [Bugzilla](../user/project/integrations/bugzilla.md) - [ClickUp](../user/project/integrations/clickup.md) -- [Custom Issue Tracker](../user/project/integrations/custom_issue_tracker.md) -- [Engineering Workflow Management](../user/project/integrations/ewm.md) +- [Custom issue tracker](../user/project/integrations/custom_issue_tracker.md) +- [Engineering Workflow Management (EWM)](../user/project/integrations/ewm.md) - [Jira](../integration/jira/index.md) - [Redmine](../user/project/integrations/redmine.md) - [YouTrack](../user/project/integrations/youtrack.md) diff --git a/doc/integration/index.md b/doc/integration/index.md index f9865199505..866c68e6d25 100644 --- a/doc/integration/index.md +++ b/doc/integration/index.md @@ -53,8 +53,26 @@ For more details, see [Secure your application](../user/application_security/ind ## Security partners -You can integrate GitLab with several security partners. For more information, see -[Security partner integrations](security_partners/index.md). +You can integrate GitLab with the following security partners: + +<!-- vale gitlab.Spelling = NO --> + +- [Anchore](https://docs.anchore.com/current/docs/configuration/integration/ci_cd/gitlab/) +- [Bridgecrew](https://docs.bridgecrew.io/docs/integrate-with-gitlab-self-managed) +- [Checkmarx](https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1929937052/GitLab+Integration) +- [Deepfactor](https://www.deepfactor.io/docs/integrate-deepfactor-scanner-in-your-ci-cd-pipelines/#gitlab) +- [Fortify](https://www.microfocus.com/en-us/fortify-integrations/gitlab) +- [GrammaTech](https://www.grammatech.com/codesonar-gitlab-integration) +- [Indeni](https://docs.cloudrail.app/#/integrations/gitlab) +- [Jscrambler](https://docs.jscrambler.com/code-integrity/documentation/gitlab-ci-integration) +- [Mend](https://www.mend.io/gitlab/) +- [Semgrep](https://semgrep.dev/for/gitlab) +- [StackHawk](https://docs.stackhawk.com/continuous-integration/gitlab.html) +- [Tenable](https://docs.tenable.com/tenableio/Content/ContainerSecurity/GetStarted.htm) +- [Venafi](https://marketplace.venafi.com/xchange/620d2d6ed419fb06a5c5bd36/solution/6292c2ef7550f2ee553cf223) +- [Veracode](https://community.veracode.com/s/knowledgeitem/gitlab-ci-MCEKSYPRWL35BRTGOVI55SK5RI4A) + +<!-- vale gitlab.Spelling = YES --> ## Continuous integration diff --git a/doc/integration/jira/configure.md b/doc/integration/jira/configure.md index 89afa998431..9660e091798 100644 --- a/doc/integration/jira/configure.md +++ b/doc/integration/jira/configure.md @@ -10,7 +10,7 @@ The Jira issue integration connects one or more GitLab projects to a Jira instan ## Configure the integration -> Authentication with Jira personal access tokens was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/8222) in GitLab 16.0. +> Authentication with Jira personal access tokens [introduced](https://gitlab.com/groups/gitlab-org/-/epics/8222) in GitLab 16.0. Prerequisites: @@ -22,7 +22,7 @@ Prerequisites: - Jira personal access token (GitLab 16.0 and later). You can enable the Jira issue integration by configuring your project settings in GitLab. -You can configure these settings at the [group level](../../user/admin_area/settings/project_integration_management.md#manage-group-level-default-settings-for-a-project-integration) or at the [instance level](../../user/admin_area/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration) for self-managed GitLab. +You can configure these settings at the [group level](../../administration/settings/project_integration_management.md#manage-group-level-default-settings-for-a-project-integration) or at the [instance level](../../administration/settings/project_integration_management.md#manage-instance-level-default-settings-for-a-project-integration) for self-managed GitLab. To configure your project settings in GitLab: @@ -81,7 +81,7 @@ To create a Jira Cloud API token: profile, select **Account Settings > Security > Create and manage API tokens**. 1. Select **Create API token**. -1. In the dialog, enter a label for your token and select **Create**. +1. On the dialog, enter a label for your token and select **Create**. To copy the API token, select **Copy**. diff --git a/doc/integration/jira/connect-app.md b/doc/integration/jira/connect-app.md index 67f51a6f472..005069990c4 100644 --- a/doc/integration/jira/connect-app.md +++ b/doc/integration/jira/connect-app.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab for Jira Cloud app **(FREE)** -With the [GitLab for Jira Cloud](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud) app, you can connect GitLab and Jira Cloud to sync development information in real time. You can view this information in the [Jira development panel](development_panel.md). +With the [GitLab for Jira Cloud](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud?tab=overview&hosting=cloud) app, you can connect GitLab and Jira Cloud to sync development information in real time. You can view this information in the [Jira development panel](development_panel.md). You can use the GitLab for Jira Cloud app to link top-level groups or subgroups. It's not possible to directly link projects or personal namespaces. @@ -33,7 +33,7 @@ To install the GitLab for Jira Cloud app: 1. In Jira, select **Jira Settings > Apps > Find new apps**, and search for GitLab. 1. Select **GitLab for Jira Cloud**, and select **Get it now**. - Alternatively, [get the app directly from the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud). + Alternatively, [get the app directly from the Atlassian Marketplace](https://marketplace.atlassian.com/apps/1221011/gitlab-com-for-jira-cloud?tab=overview&hosting=cloud). 1. To go to the configurations page, select **Get started**. You can always access this page in **Jira Settings > Apps > Manage apps**. @@ -46,9 +46,9 @@ For an overview, see After you add a group, the following data is synced to Jira for all projects in that group: -- New merge requests, branches, and commits -- Existing merge requests (GitLab 13.8 and later) -- Existing branches and commits (GitLab 15.11 and later) +- New and existing merge requests. +- New branches and commits. +- Existing branches and commits (GitLab 15.11 and later). You must delete and add any namespaces that were added to the GitLab for Jira Cloud app in GitLab 15.10 and earlier. ## Update the GitLab for Jira Cloud app diff --git a/doc/integration/jira/development_panel.md b/doc/integration/jira/development_panel.md index 009e620f121..c444ffe8a3b 100644 --- a/doc/integration/jira/development_panel.md +++ b/doc/integration/jira/development_panel.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Jira development panel **(FREE)** -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/233149) from GitLab Premium to GitLab Free in 13.4. - You can use the Jira development panel to view GitLab activity for a Jira issue directly in Jira. To set up the Jira development panel: @@ -51,9 +49,9 @@ depends on where you mention the Jira issue ID in GitLab. | GitLab: where you mention the Jira issue ID | Jira development panel: what information is displayed | |------------------------------------------------|-------------------------------------------------------| -| Merge request title or description | Link to the merge request<br>Link to the branch ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354373) in GitLab 15.11) | -| Branch name | Link to the branch | -| Commit message | Link to the commit | +| Merge request title or description | Link to the merge request<br>Link to the deployment<br>Link to the pipeline by title only and by description ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390888) in GitLab 15.10)<br>Link to the branch ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354373) in GitLab 15.11) | +| Branch name | Link to the branch<br>Link to the deployment | +| Commit message | Link to the commit<br>Link to the deployment from up to 5,000 commits after the last successful deployment to the environment ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300031) in GitLab 16.2 [with a flag](../../administration/feature_flags.md) named `jira_deployment_issue_keys`. Enabled by default) | | [Jira Smart Commit](#jira-smart-commits) | Custom comment, logged time, or workflow transition | ## Jira Smart Commits @@ -93,7 +91,6 @@ For more information about how Smart Commits work and what commands are availabl - [Process issues with Smart Commits](https://support.atlassian.com/jira-software-cloud/docs/process-issues-with-smart-commits/) - [Using Smart Commits](https://confluence.atlassian.com/fisheye/using-smart-commits-960155400.html) -## Troubleshooting +## Related topics -To troubleshoot the Jira development panel on your own server, see the -[Atlassian documentation](https://confluence.atlassian.com/jirakb/troubleshoot-the-development-panel-in-jira-server-574685212.html). +- [Troubleshoot the development panel in Jira Server](https://confluence.atlassian.com/jirakb/troubleshoot-the-development-panel-in-jira-server-574685212.html) diff --git a/doc/integration/jira/index.md b/doc/integration/jira/index.md index 2b6395f437b..dbda2e91dee 100644 --- a/doc/integration/jira/index.md +++ b/doc/integration/jira/index.md @@ -46,7 +46,7 @@ This table shows the capabilities available with the Jira issue integration and | [View a list of Jira issues](issues.md#view-jira-issues) | **{check-circle}** Yes | **{dotted-circle}** No | | [Create a Jira issue for a vulnerability](../../user/application_security/vulnerabilities/index.md#create-a-jira-issue-for-a-vulnerability) | **{check-circle}** Yes | **{dotted-circle}** No | | Create a GitLab branch from a Jira issue | **{dotted-circle}** No | **{check-circle}** Yes, in the issue's development panel | -| Mention a Jira issue ID in a GitLab merge request, and deployments are synced | **{dotted-circle}** No | **{check-circle}** Yes, in the issue's development panel | +| Sync GitLab deployments to Jira issues | **{dotted-circle}** No | **{check-circle}** Yes, in the issue's development panel. Mention a Jira issue ID in a GitLab merge request, branch name, or any of the last 5,000 commits made to the branch after the last successful deployment to the environment | ## Privacy considerations diff --git a/doc/integration/jira/issues.md b/doc/integration/jira/issues.md index 7ed9d3ab329..c1b61e2e587 100644 --- a/doc/integration/jira/issues.md +++ b/doc/integration/jira/issues.md @@ -49,9 +49,6 @@ You can [disable comments](#disable-comments-on-jira-issues) on issues. ### Require associated Jira issue for merge requests to be merged **(ULTIMATE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/280766) in GitLab 13.12 [with a flag](../../administration/feature_flags.md) named `jira_issue_association_on_merge_request`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/335280) in GitLab 14.2. Feature flag `jira_issue_association_on_merge_request` removed. - With this integration, you can prevent merge requests from being merged if they do not refer to a Jira issue. To enable this feature: @@ -90,6 +87,9 @@ For more information, see the [Atlassian documentation](https://confluence.atlas ### Use a prefix +You can define a prefix for GitLab to match Jira issue keys. For example, if your Jira issue ID is `ALPHA-1` +and you've set a `JIRA#` prefix, GitLab matches `JIRA#ALPHA-1` rather than `ALPHA-1`. + To define a prefix for Jira issue keys: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. @@ -131,8 +131,6 @@ Consider this example: ## View Jira issues **(PREMIUM)** -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3622) in GitLab 13.2. - You can view and search issues from a selected Jira project directly in GitLab, provided your GitLab administrator [has configured the integration](configure.md#configure-the-integration). @@ -145,7 +143,7 @@ The issues are sorted by **Created date** by default, with the most recently cre - To display the most recently updated issues first, select **Updated date**. - You can [search and filter the issue list](#search-and-filter-the-issue-list). -- In GitLab 13.10 and later, you can [select an issue from the list to view the issue in GitLab](https://gitlab.com/gitlab-org/gitlab/-/issues/299832). +- You can [select an issue from the list to view the issue in GitLab](https://gitlab.com/gitlab-org/gitlab/-/issues/299832). Issues are grouped into tabs based on their [Jira status](https://confluence.atlassian.com/adminjiraserver070/defining-status-field-values-749382903.html): @@ -175,8 +173,6 @@ Enhancements to use these filters through the user interface ## Automatic issue transitions -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55773) in GitLab 13.11. - When you configure automatic issue transitions, you can transition a referenced Jira issue to the next available status with a category of **Done**. To configure this setting: diff --git a/doc/integration/jira/jira_server_configuration.md b/doc/integration/jira/jira_server_configuration.md index 672df5d615f..373fe137046 100644 --- a/doc/integration/jira/jira_server_configuration.md +++ b/doc/integration/jira/jira_server_configuration.md @@ -65,13 +65,13 @@ To create a permission scheme for the group: 1. On the top bar, in the upper-right corner, select **Administration** (**{settings}**) > **Issues**. 1. On the left sidebar, select **Permission schemes**. 1. Select **Add permission scheme**. -1. In the **Add permission scheme** dialog: +1. On the **Add permission scheme** dialog: - Enter a name for the scheme. - Optional. Enter a description for the scheme. 1. Select **Add**. 1. On the **Permission schemes** page, in the **Actions** column, select **Permissions** for the new scheme. 1. Next to **Administer Projects**, select **Edit**. -1. In the **Grant permission** dialog, for **Granted to**, select **Group**. +1. On the **Grant permission** dialog, for **Granted to**, select **Group**. 1. From the **Group** dropdown list, select `gitlab-developers`, then select **Grant**. You've done it! You can now use your new Jira username and password to configure the diff --git a/doc/integration/jira/troubleshooting.md b/doc/integration/jira/troubleshooting.md index d592455788d..49b5dfba566 100644 --- a/doc/integration/jira/troubleshooting.md +++ b/doc/integration/jira/troubleshooting.md @@ -108,3 +108,12 @@ Check [`production.log`](../../administration/logs/index.md#productionlog) to se ``` If that's the case, ensure the **Due date** field is visible for issues in the integrated Jira project. + +## `An error occurred while requesting data from Jira` when viewing the Jira issues list in GitLab + +You might see a `An error occurred while requesting data from Jira` message when you attempt to view the Jira issues list in GitLab. + +You can see this error when the authentication details in the Jira integration settings are incomplete or incorrect. + +To attempt to resolve this error, try [configuring the integration](configure.md#configure-the-integration) again. Verify that the +authentication details are correct, re-enter your API token or password, and save your changes. diff --git a/doc/integration/kerberos.md b/doc/integration/kerberos.md index 62441f6de8f..5271f60b5dd 100644 --- a/doc/integration/kerberos.md +++ b/doc/integration/kerberos.md @@ -417,7 +417,7 @@ There are a number of potential causes and solutions for this error message. #### Kerberos integration not using a dedicated port -GitLab CI/CD doesn’t work with a Kerberos-enabled GitLab instance unless the Kerberos integration +GitLab CI/CD doesn't work with a Kerberos-enabled GitLab instance unless the Kerberos integration is configured to [use a dedicated port](kerberos.md#http-git-access-with-kerberos-token-passwordless-authentication). #### Lack of connectivity between client machine and Kerberos server @@ -431,6 +431,11 @@ If you're experiencing this error, ensure there is connectivity between the client machine and the Kerberos server - this is a prerequisite! Traffic may be blocked by a firewall, or the DNS records may be incorrect. +#### `GitLab DNS record is a CNAME record` error + +Kerberos fails with this error when GitLab is referenced with a `CNAME` record. +To resolve this issue, ensure the DNS record for GitLab is an `A` record. + #### Mismatched forward and reverse DNS records for GitLab instance hostname Another failure mode occurs when the forward and reverse DNS records for the @@ -442,7 +447,7 @@ above error message. To fix this, ensure that the forward and reverse DNS for your GitLab server match. So for instance, if you access GitLab as `gitlab.example.com`, resolving -to IP address `1.2.3.4`, then `4.3.2.1.in-addr.arpa` must be a `PTR` record for +to IP address `10.0.2.2`, then `2.2.0.10.in-addr.arpa` must be a `PTR` record for `gitlab.example.com`. #### Missing Kerberos libraries on browser or client machine diff --git a/doc/integration/mattermost/index.md b/doc/integration/mattermost/index.md index 0f9192f9a84..7ca4ed8a0e8 100644 --- a/doc/integration/mattermost/index.md +++ b/doc/integration/mattermost/index.md @@ -176,7 +176,7 @@ sudo gitlab-psql -d mattermost_production ## Back up GitLab Mattermost -GitLab Mattermost is not included in the regular [Linux package backup](../../raketasks/backup_restore.md) Rake task. +GitLab Mattermost is not included in the regular [Linux package backup](../../administration/backup_restore/index.md) Rake task. The general Mattermost [backup and disaster recovery](https://docs.mattermost.com/deploy/backup-disaster-recovery.html) documentation can be used as a guide on what needs to be backed up. diff --git a/doc/integration/oauth_provider.md b/doc/integration/oauth_provider.md index 6d08af225db..dffb45d30a1 100644 --- a/doc/integration/oauth_provider.md +++ b/doc/integration/oauth_provider.md @@ -106,6 +106,7 @@ different actions. See the following table for all available scopes. | `openid` | Grants permission to authenticate with GitLab using [OpenID Connect](openid_connect_provider.md). Also gives read-only access to the user's profile and group memberships. | | `profile` | Grants read-only access to the user's profile data using [OpenID Connect](openid_connect_provider.md). | | `email` | Grants read-only access to the user's primary email address using [OpenID Connect](openid_connect_provider.md). | +| `create_runner` | Grants permission to create runners. | At any time you can revoke any access by selecting **Revoke**. @@ -131,7 +132,7 @@ When applications are deleted, all grants and tokens associated with the applica > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/374588) in GitLab 15.9. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `hash_oauth_secrets`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../administration/feature_flags.md) named `hash_oauth_secrets`. On GitLab.com, this feature is available. By default, GitLab stores OAuth application secrets in the database in hashed format. These secrets are only available to users immediately after creating OAuth applications. In diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 2a871b97a28..01ea6408469 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -47,7 +47,7 @@ Linux package, Docker, and self-compiled | Helm chart | Description | Default va ----------------------------|------------|-------------|----------- `allow_single_sign_on` | `allowSingleSignOn` | List of providers that automatically create a GitLab account. The provider names are available in the **OmniAuth provider name** column in the [supported providers table](#supported-providers). | `false`, which means that signing in using your OmniAuth provider account without a pre-existing GitLab account is not allowed. You must create a GitLab account first, and then connect it to your OmniAuth provider account through your profile settings. `auto_link_ldap_user` | `autoLinkLdapUser` | Creates an LDAP identity in GitLab for users that are created through an OmniAuth provider. You can enable this setting if you have [LDAP integration](../administration/auth/ldap/index.md) enabled. Requires the `uid` of the user to be the same in both LDAP and the OmniAuth provider. | `false` -`block_auto_created_users` | `blockAutoCreatedUsers` | Places automatically-created users in a [Pending approval](../user/admin_area/moderate_users.md#users-pending-approval) state (unable to sign in) until they are approved by an administrator. | `true`. If you set the value to `false`, make sure you define providers that you can control, like SAML or Google. Otherwise, any user on the internet can sign in to GitLab without an administrator's approval. +`block_auto_created_users` | `blockAutoCreatedUsers` | Places automatically-created users in a [Pending approval](../administration/moderate_users.md#users-pending-approval) state (unable to sign in) until they are approved by an administrator. | `true`. If you set the value to `false`, make sure you define providers that you can control, like SAML or Google. Otherwise, any user on the internet can sign in to GitLab without an administrator's approval. ### Configure initial settings @@ -309,7 +309,7 @@ To enable automatic linking for SAML, see the [SAML setup instructions](saml.md# ## Create an external providers list You can define a list of external OmniAuth providers. -Users who create accounts or sign in to GitLab through the listed providers do not get access to [internal projects](../user/public_access.md#internal-projects-and-groups) and are marked as [external users](../user/admin_area/external_users.md). +Users who create accounts or sign in to GitLab through the listed providers do not get access to [internal projects](../user/public_access.md#internal-projects-and-groups) and are marked as [external users](../administration/external_users.md). To define the external providers list, use the full name of the provider, for example, `google_oauth2` for Google. For provider names, see the diff --git a/doc/integration/saml.md b/doc/integration/saml.md index f59824c8db6..d31f4ff80b3 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -673,16 +673,16 @@ Prerequisites: 1. Use the following information, and follow the instructions in [Set up your own custom SAML application in Google Workspace](https://support.google.com/a/answer/6087519?hl=en). - | | Typical value | Description | - |------------------|--------------------------------------------------|----------------------------------------------------------| - | Name of SAML App | GitLab | Other names OK. | - | ACS URL | `https://<GITLAB_DOMAIN>/users/auth/saml/callback` | Assertion Consumer Service URL. | - | GITLAB_DOMAIN | `gitlab.example.com` | Your GitLab instance domain. | - | Entity ID | `https://gitlab.example.com` | A value unique to your SAML application. Set it to the `issuer` in your GitLab configuration. | - | Name ID format | EMAIL | Required value. Also known as `name_identifier_format`. | - | Name ID | Primary email address | Your email address. Make sure someone receives content sent to that address. | - | First name | `first_name` | First name. Required value to communicate with GitLab. | - | Last name | `last_name` | Last name. Required value to communicate with GitLab. | + | | Typical value | Description | + |:-----------------|:---------------------------------------------------|:----------------------------------------------------------------------------------------------| + | Name of SAML App | GitLab | Other names OK. | + | ACS URL | `https://<GITLAB_DOMAIN>/users/auth/saml/callback` | Assertion Consumer Service URL. | + | `GITLAB_DOMAIN` | `gitlab.example.com` | Your GitLab instance domain. | + | Entity ID | `https://gitlab.example.com` | A value unique to your SAML application. Set it to the `issuer` in your GitLab configuration. | + | Name ID format | `EMAIL` | Required value. Also known as `name_identifier_format`. | + | Name ID | Primary email address | Your email address. Make sure someone receives content sent to that address. | + | First name | `first_name` | First name. Required value to communicate with GitLab. | + | Last name | `last_name` | Last name. Required value to communicate with GitLab. | 1. Set up the following SAML attribute mappings: @@ -742,7 +742,7 @@ As a result, SAML Group Sync only supports a single SAML provider. For more info You can: - Require users to be members of a certain group. -- Assign users [external](../user/admin_area/external_users.md), administrator or [auditor](../administration/auditor_users.md) roles based on group membership. +- Assign users [external](../administration/external_users.md), administrator or [auditor](../administration/auditor_users.md) roles based on group membership. GitLab checks these groups on each SAML sign in and updates user attributes as necessary. This feature **does not** allow you to automatically add users to GitLab @@ -948,7 +948,7 @@ response, configure GitLab to identify: - Information about a group or user, using a group setting. SAML can automatically identify a user as an -[external user](../user/admin_area/external_users.md), based on the `external_groups` +[external user](../administration/external_users.md), based on the `external_groups` setting. Example configuration: diff --git a/doc/integration/security_partners/index.md b/doc/integration/security_partners/index.md index 5453b3417ab..e89f88becbc 100644 --- a/doc/integration/security_partners/index.md +++ b/doc/integration/security_partners/index.md @@ -1,30 +1,11 @@ --- -stage: Secure -group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index +redirect_to: '../index.md#security-partners' +remove_date: '2023-10-05' --- -# Security partners **(FREE)** +This document was moved to [another location](../index.md#security-partners). -You can integrate GitLab with its security partners. This page has information on how do this with -each security partner: - -<!-- vale gitlab.Spelling = NO --> - -- [Anchore](https://docs.anchore.com/current/docs/configuration/integration/ci_cd/gitlab/) -- [Bridgecrew](https://docs.bridgecrew.io/docs/integrate-with-gitlab-self-managed) -- [Checkmarx](https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1929937052/GitLab+Integration) -- [Deepfactor](https://www.deepfactor.io/docs/integrate-deepfactor-scanner-in-your-ci-cd-pipelines/#gitlab) -- [GrammaTech](https://www.grammatech.com/codesonar-gitlab-integration) -- [Indeni](https://docs.cloudrail.app/#/integrations/gitlab) -- [JScrambler](https://docs.jscrambler.com/code-integrity/documentation/gitlab-ci-integration) -- [Mend](https://www.mend.io/gitlab/) -- [Semgrep](https://semgrep.dev/for/gitlab) -- [StackHawk](https://docs.stackhawk.com/continuous-integration/gitlab.html) -- [Tenable](https://docs.tenable.com/tenableio/Content/ContainerSecurity/GetStarted.htm) -- [Venafi](https://marketplace.venafi.com/xchange/620d2d6ed419fb06a5c5bd36/solution/6292c2ef7550f2ee553cf223) -- [Veracode](https://community.veracode.com/s/knowledgeitem/gitlab-ci-MCEKSYPRWL35BRTGOVI55SK5RI4A) -- [Fortify](https://www.microfocus.com/en-us/fortify-integrations/gitlab) - -<!-- vale gitlab.Spelling = YES --> +<!-- This redirect file can be deleted after <2023-10-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md index db49b30fa21..976f5ca9774 100644 --- a/doc/integration/shibboleth.md +++ b/doc/integration/shibboleth.md @@ -1,6 +1,6 @@ --- -stage: Create -group: Ecosystem +stage: Manage +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md index 02c7debc6dc..e36ee164002 100644 --- a/doc/integration/slash_commands.md +++ b/doc/integration/slash_commands.md @@ -1,51 +1,11 @@ --- -stage: Manage -group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../user/project/integrations/gitlab_slack_application.md#slash-commands' +remove_date: '2023-09-19' --- -# Slash commands in Mattermost and Slack **(FREE)** +This document was moved to [another location](../user/project/integrations/gitlab_slack_application.md#slash-commands). -> [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/24780) from GitLab Ultimate to GitLab Free in 11.9. - -If you want to control and view GitLab content while you're -working in Slack and Mattermost, you can use slash commands. -Type the command as a message in your chat client to activate it. -For Slack, this requires an [integration configuration](../user/project/integrations/slack_slash_commands.md). - -Slash commands are scoped to a project and require -a specified trigger command during configuration. -You should use the project name as the trigger command. - -If you're using the [GitLab for Slack app](../user/project/integrations/gitlab_slack_application.md) for -your GitLab.com projects, [add the `gitlab` keyword at the beginning of the command](../user/project/integrations/gitlab_slack_application.md#slash-commands) -(for example, `/gitlab <project-name> issue show <id>`). - -Assuming `project-name` is the trigger command, the slash commands are: - -| Command | Effect | -| ------- | ------ | -| `/project-name help` | Shows all available slash commands. | -| `/project-name issue new <title> <shift+return> <description>` | Creates a new issue with title `<title>` and description `<description>`. | -| `/project-name issue show <id>` | Shows the issue with ID `<id>`. | -| `/project-name issue close <id>` | Closes the issue with ID `<id>`. | -| `/project-name issue search <query>` | Shows up to 5 issues matching `<query>`. | -| `/project-name issue move <id> to <project>` | Moves the issue with ID `<id>` to `<project>`. | -| `/project-name issue comment <id> <shift+return> <comment>` | Adds a new comment with comment body `<comment>` to the issue with ID `<id>`. | -| `/project-name deploy <from> to <to>` | [Deploys](#deploy-command) from the `<from>` environment to the `<to>` environment. | -| `/project-name run <job name> <arguments>` | Executes the [ChatOps](../ci/chatops/index.md) job `<job name>` on the default branch. | - -## Issue commands - -You can create a new issue, display issue details, and search up to 5 issues. - -## Deploy command - -To deploy to an environment, GitLab tries to find a deployment -manual action in the pipeline. - -If there's only one action for a given environment, it is triggered. -If more than one action is defined, GitLab finds an action -name that equals the environment name to deploy to. - -The command returns an error if no matching action is found. +<!-- This redirect file can be deleted after 2023-09-19. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/integration/sourcegraph.md b/doc/integration/sourcegraph.md index 366a862a9fb..4b0db1ab4b9 100644 --- a/doc/integration/sourcegraph.md +++ b/doc/integration/sourcegraph.md @@ -12,7 +12,7 @@ type: reference, how-to > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73116) in GitLab 14.8. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `sourcegraph`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../administration/feature_flags.md) named `sourcegraph`. On GitLab.com, this feature is available for public projects only. [Sourcegraph](https://sourcegraph.com) provides code intelligence features, natively integrated into the GitLab UI. diff --git a/doc/operations/error_tracking.md b/doc/operations/error_tracking.md index a2d50e43a80..c3902a560c0 100644 --- a/doc/operations/error_tracking.md +++ b/doc/operations/error_tracking.md @@ -1,5 +1,5 @@ --- -stage: Monitor +stage: Analytics group: Observability info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md index edb7176c045..142bd9d898d 100644 --- a/doc/operations/feature_flags.md +++ b/doc/operations/feature_flags.md @@ -14,7 +14,9 @@ Feature flags help reduce risk, allowing you to do controlled testing, and separ delivery from customer launch. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an example of feature flags in action, see [GitLab for deploys, feature flags, and error tracking](https://www.youtube.com/watch?v=5tw2p6lwXxo). +For an example of feature flags in action, see [Feature Flags configuration, instrumentation and use](https://www.youtube.com/watch?v=ViA6suScxkE). + +You can also explore feature flags with a [click-through demo](https://go.gitlab.com/YKuzLt). NOTE: To contribute to the development of the GitLab product, view @@ -85,7 +87,7 @@ and the supported strategies are: - [User List](#user-list) Strategies can be added to feature flags when [creating a feature flag](#create-a-feature-flag), -or by editing an existing feature flag after creation by navigating to **Deployments > Feature flags** +or by editing an existing feature flag after creation by navigating to **Deploy > Feature flags** and selecting **Edit** (**{pencil}**). ### All users diff --git a/doc/operations/incident_management/incident_timeline_events.md b/doc/operations/incident_management/incident_timeline_events.md index 6a52accbfb2..0c47ff46bc9 100644 --- a/doc/operations/incident_management/incident_timeline_events.md +++ b/doc/operations/incident_management/incident_timeline_events.md @@ -88,7 +88,7 @@ of an incident. > [Introduced]([issue-link](https://gitlab.com/gitlab-org/gitlab/-/issues/365489)) in GitLab 15.3 [with a flag](../../administration/feature_flags.md) named `incident_timeline_events_from_labels`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `incident_timeline_events_from_labels`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `incident_timeline_events_from_labels`. On GitLab.com, this feature is not available. This feature is not ready for production use. @@ -139,7 +139,7 @@ Added tags are displayed next to the timestamp. Incident timeline events support the following [GitLab Flavored Markdown](../../user/markdown.md) features. - [Code](../../user/markdown.md#code-spans-and-blocks). -- [Emojis](../../user/markdown.md#emojis). +- [Emoji](../../user/markdown.md#emoji). - [Emphasis](../../user/markdown.md#emphasis). - [GitLab-specific references](../../user/markdown.md#gitlab-specific-references). - [Images](../../user/markdown.md#images), rendered as a link to the uploaded image. diff --git a/doc/operations/incident_management/manage_incidents.md b/doc/operations/incident_management/manage_incidents.md index 3d2688daf6a..bb9dde4b416 100644 --- a/doc/operations/incident_management/manage_incidents.md +++ b/doc/operations/incident_management/manage_incidents.md @@ -222,9 +222,9 @@ When you close an incident that is linked to an [alert](alerts.md), the linked alert's status changes to **Resolved**. You are then credited with the alert's status change. -<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> +<!-- Delete when the `move_close_into_dropdown` feature flag is removed --> If you don't see this action at the top of an incident, your project or instance might have -enabled a feature flag for [moved actions](../../user/project/merge_requests/index.md#move-sidebar-actions) +enabled a feature flag to [moved it in the actions menu](../../user/project/issues/managing_issues.md#move-the-close-button-into-the-actions-menu). ### Automatically close incidents via recovery alerts diff --git a/doc/operations/incident_management/slack.md b/doc/operations/incident_management/slack.md index b007f253fd8..aa0d057bfd8 100644 --- a/doc/operations/incident_management/slack.md +++ b/doc/operations/incident_management/slack.md @@ -37,7 +37,7 @@ Prerequisites: 1. Authorize GitLab to take actions on behalf of your Slack user. Each user must do this before they can use any of the incident slash commands. - To start the authorization flow, try executing a non-incident [Slack slash command](../../integration/slash_commands.md), + To start the authorization flow, try executing a non-incident [Slack slash command](../../user/project/integrations/gitlab_slack_application.md#slash-commands), like `/gitlab <project-alias> issue show <id>`. The `<project-alias>` you select must be a project that has the GitLab for Slack app set up. For more information, see [issue 377548](https://gitlab.com/gitlab-org/gitlab/-/issues/377548). diff --git a/doc/operations/incident_management/status_page.md b/doc/operations/incident_management/status_page.md index a159757842d..a52790b7f70 100644 --- a/doc/operations/incident_management/status_page.md +++ b/doc/operations/incident_management/status_page.md @@ -19,8 +19,8 @@ Selecting an incident displays a detail page with more information about a parti  - Status on the incident, including when the incident was last updated. -- The incident title, including any emojis. -- The description of the incident, including emojis. +- The incident title, including any emoji. +- The description of the incident, including emoji. - Any file attachments provided in the incident description, or comments with a valid image extension. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/205166) in GitLab 13.1. - A chronological ordered list of updates to the incident. @@ -84,7 +84,7 @@ the necessary CI/CD variables to deploy the Status Page to AWS S3: - `AWS_DEFAULT_REGION` - The AWS region. - `AWS_ACCESS_KEY_ID` - The AWS access key ID. - `AWS_SECRET_ACCESS_KEY` - The AWS secret. -1. On the left sidebar, select **CI/CD > Pipelines**. +1. On the left sidebar, select **Build > Pipelines**. 1. To deploy the Status Page to S3, select **Run pipeline**. WARNING: diff --git a/doc/policy/experiment-beta-support.md b/doc/policy/experiment-beta-support.md index 59490ac3d68..b6156f9c20f 100644 --- a/doc/policy/experiment-beta-support.md +++ b/doc/policy/experiment-beta-support.md @@ -12,18 +12,20 @@ All other features are considered to be Generally Available (GA). ## Experiment -Support is not provided for features listed as "Experimental" or "Alpha" or any similar designation. Issues regarding such features should be opened in the GitLab issue tracker. +Support is not provided for features listed as "Experimental" or "Alpha" or any similar designation. Issues regarding such features should be opened in the GitLab issue tracker. Teams should release features as GA from the start unless there are strong reasons to release them as Experiment or Beta versions first. - Not ready for production use. - No support available. -- May be unstable or have performance issues. +- May be unstable. - Can be removed at any time. - Data loss may occur. - Documentation may not exist or just be in a blog format. -- [User interface reflects Experiment status](https://design.gitlab.com/usability/feature-management#highlighting-feature-versions). -- User experience incomplete, might be just quick action access. -- Behind a feature flag that is on by default. -- Behind a toggle that is off by default. +- Offer an easy way to choose to opt-in to experimental features with minimal friction. For example, needing to flip a feature flag is too much friction, but a group or project-level setting that is in the UI is not. +- Link out to the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/) in the opt-in. +- Documentation reflects that the feature is subject to the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). +- [UI reflects experiment status](https://design.gitlab.com/usability/feature-management#highlighting-feature-versions). +- Feedback issue to engage with team. +- UX not finalized, might be just quick action access. - Not announced in a release post. - Can be promoted in the user interface through [discovery moments](https://design.gitlab.com/usability/feature-management#discovery-moments), if needed. - Feedback issue to engage with team. @@ -34,7 +36,7 @@ Commercially-reasonable efforts are made to provide limited support for features - May not be ready for production use. - Support on a commercially-reasonable effort basis. -- May be unstable and can cause performance and stability issues. +- May be unstable. - Configuration and dependencies unlikely to change. - Features and functions unlikely to change. However, breaking changes may occur outside of major releases or with less notice than for Generally Available features. - Data loss not likely. @@ -54,13 +56,13 @@ Generally Available features means that they passed the [Production Readiness Re - Fully documented and supported. - User experience complete and in line with GitLab design standards. -## Never internal +## Provide earlier access -Features are never internal (GitLab team-members) only. +Give users the ability to opt into experimental features when there is enough value. +Where possible, release an experimental feature externally instead of only testing internally or waiting for the feature to be in a Beta state. Our [mission is "everyone can contribute"](https://about.gitlab.com/company/mission/), and that is only possible if people outside the company can try a feature. We get higher quality (more diverse) feedback if people from different organizations try something. -We've also learned that internal only as a state slows us down more than it speeds us up. -Release the experiment instead of testing internally or waiting for the feature to be in a Beta state. +We've learned that keeping features internal only for extended periods of time slows us down unnecessarily. The experimental features are only shown when people/organizations opt-in to experiments, we are allowed to make mistakes here and literally experiment. ## All features are in production diff --git a/doc/policy/maintenance.md b/doc/policy/maintenance.md index 7970f9711e6..a6de6f594fb 100644 --- a/doc/policy/maintenance.md +++ b/doc/policy/maintenance.md @@ -102,7 +102,7 @@ accessible. ### Backporting to older releases -Backporting to more than one stable release is normally reserved for [security releases](#security-releases). +Backporting to more than one stable release is usually reserved for [security releases](#security-releases). In some cases, however, we may need to backport *a bug fix* to more than one stable release, depending on the severity of the bug. diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md index a6635c589aa..231312b3833 100644 --- a/doc/raketasks/backup_gitlab.md +++ b/doc/raketasks/backup_gitlab.md @@ -2,955 +2,13 @@ stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../administration/backup_restore/backup_gitlab.md' +remove_date: '2023-09-26' --- -# Back up GitLab **(FREE SELF)** +This document was moved to [another location](../administration/backup_restore/backup_gitlab.md). -GitLab provides a command line interface to back up your entire instance, -including: - -- Database -- Attachments -- Git repositories data -- CI/CD job output logs -- CI/CD job artifacts -- LFS objects -- Terraform states ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/331806) in GitLab 14.7) -- Container Registry images -- GitLab Pages content -- Packages ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332006) in GitLab 14.7) -- Snippets -- [Group wikis](../user/project/wiki/group.md) -- Project-level Secure Files ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121142) in GitLab 16.1) - -Backups do not include: - -- [Mattermost data](https://docs.mattermost.com/administration/config-settings.html#file-storage) -- Redis (and thus Sidekiq jobs) - -WARNING: -GitLab does not back up any configuration files (`/etc/gitlab`), TLS keys and certificates, or system -files. You are highly advised to read about [storing configuration files](#storing-configuration-files). - -WARNING: -The backup command requires [additional parameters](backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) when -your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. - -WARNING: -Before GitLab 15.5.0, the backup command doesn't verify if another backup is already running, as described in -[issue 362593](https://gitlab.com/gitlab-org/gitlab/-/issues/362593). We strongly recommend -you make sure that all backups are complete before starting a new one. - -Depending on your version of GitLab, use the following command if you installed -GitLab using the Omnibus package: - -- GitLab 12.2 or later: - - ```shell - sudo gitlab-backup create - ``` - -- GitLab 12.1 and earlier: - - ```shell - gitlab-rake gitlab:backup:create - ``` - -If you installed GitLab from source, use the following command: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production -``` - -If you're running GitLab from within a Docker container, run the backup from -the host, based on your installed version of GitLab: - -- GitLab 12.2 or later: - - ```shell - docker exec -t <container name> gitlab-backup create - ``` - -- GitLab 12.1 and earlier: - - ```shell - docker exec -t <container name> gitlab-rake gitlab:backup:create - ``` - -If you're using the [GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab) -on a Kubernetes cluster, you can run the backup task by using `kubectl` to run the `backup-utility` -script on the GitLab toolbox pod. For more details, see the -[charts backup documentation](https://docs.gitlab.com/charts/backup-restore/backup.html). - -Similar to the Kubernetes case, if you have scaled out your GitLab cluster to -use multiple application servers, you should pick a designated node (that isn't -auto-scaled away) for running the backup Rake task. Because the backup Rake -task is tightly coupled to the main Rails application, this is typically a node -on which you're also running Puma or Sidekiq. - -Example output: - -```plaintext -Dumping database tables: -- Dumping table events... [DONE] -- Dumping table issues... [DONE] -- Dumping table keys... [DONE] -- Dumping table merge_requests... [DONE] -- Dumping table milestones... [DONE] -- Dumping table namespaces... [DONE] -- Dumping table notes... [DONE] -- Dumping table projects... [DONE] -- Dumping table protected_branches... [DONE] -- Dumping table schema_migrations... [DONE] -- Dumping table services... [DONE] -- Dumping table snippets... [DONE] -- Dumping table taggings... [DONE] -- Dumping table tags... [DONE] -- Dumping table users... [DONE] -- Dumping table users_projects... [DONE] -- Dumping table web_hooks... [DONE] -- Dumping table wikis... [DONE] -Dumping repositories: -- Dumping repository abcd... [DONE] -Creating backup archive: $TIMESTAMP_gitlab_backup.tar [DONE] -Deleting tmp directories...[DONE] -Deleting old backups... [SKIPPING] -``` - -## Storing configuration files - -The [backup Rake task](#back-up-gitlab) GitLab provides does _not_ store your -configuration files. The primary reason for this is that your database contains -items including encrypted information for two-factor authentication and the -CI/CD _secure variables_. Storing encrypted information in the same location -as its key defeats the purpose of using encryption in the first place. - -WARNING: -The secrets file is essential to preserve your database encryption key. - -At the very **minimum**, you must back up: - -For Omnibus: - -- `/etc/gitlab/gitlab-secrets.json` -- `/etc/gitlab/gitlab.rb` - -For installation from source: - -- `/home/git/gitlab/config/secrets.yml` -- `/home/git/gitlab/config/gitlab.yml` - -For [Docker installations](../install/docker.md), you must -back up the volume where the configuration files are stored. If you created -the GitLab container according to the documentation, it should be in the -`/srv/gitlab/config` directory. - -For [GitLab Helm chart installations](https://gitlab.com/gitlab-org/charts/gitlab) -on a Kubernetes cluster, you must follow the -[Back up the secrets](https://docs.gitlab.com/charts/backup-restore/backup.html#back-up-the-secrets) -instructions. - -You may also want to back up any TLS keys and certificates (`/etc/gitlab/ssl`, `/etc/gitlab/trusted-certs`), and your -[SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079) -to avoid man-in-the-middle attack warnings if you have to perform a full machine restore. - -If you use Omnibus GitLab, review additional information to -[backup your configuration](https://docs.gitlab.com/omnibus/settings/backups.html). - -In the unlikely event that the secrets file is lost, see the -[troubleshooting section](backup_restore.md#when-the-secrets-file-is-lost). - -## Backup options - -The command line tool GitLab provides to backup your instance can accept more -options. - -### Backup strategy option - -The default backup strategy is to essentially stream data from the respective -data locations to the backup using the Linux command `tar` and `gzip`. This works -fine in most cases, but can cause problems when data is rapidly changing. - -When data changes while `tar` is reading it, the error `file changed as we read -it` may occur, and causes the backup process to fail. To combat this, 8.17 -introduces a new backup strategy called `copy`. The strategy copies data files -to a temporary location before calling `tar` and `gzip`, avoiding the error. - -A side-effect is that the backup process takes up to an additional 1X disk -space. The process does its best to clean up the temporary files at each stage -so the problem doesn't compound, but it could be a considerable change for large -installations. This is why the `copy` strategy is not the default in 8.17. - -To use the `copy` strategy instead of the default streaming strategy, specify -`STRATEGY=copy` in the Rake task command. For example: - -```shell -sudo gitlab-backup create STRATEGY=copy -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -### Backup filename - -WARNING: -If you use a custom backup filename, you can't -[limit the lifetime of the backups](#limit-backup-lifetime-for-local-files-prune-old-backups). - -By default, a backup file is created according to the specification in the -previous [Backup timestamp](backup_restore.md#backup-timestamp) section. You can, however, -override the `[TIMESTAMP]` portion of the filename by setting the `BACKUP` -environment variable. For example: - -```shell -sudo gitlab-backup create BACKUP=dump -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -The resulting file is named `dump_gitlab_backup.tar`. This is useful for -systems that make use of rsync and incremental backups, and results in -considerably faster transfer speeds. - -### Confirm archive can be transferred - -To ensure the generated archive is transferable by rsync, you can set the `GZIP_RSYNCABLE=yes` -option. This sets the `--rsyncable` option to `gzip`, which is useful only in -combination with setting [the Backup filename option](#backup-filename). - -The `--rsyncable` option in `gzip` isn't guaranteed to be available -on all distributions. To verify that it's available in your distribution, run -`gzip --help` or consult the man pages. - -```shell -sudo gitlab-backup create BACKUP=dump GZIP_RSYNCABLE=yes -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -### Excluding specific directories from the backup - -You can exclude specific directories from the backup by adding the environment variable `SKIP`, whose values are a comma-separated list of the following options: - -- `db` (database) -- `uploads` (attachments) -- `builds` (CI job output logs) -- `artifacts` (CI job artifacts) -- `lfs` (LFS objects) -- `terraform_state` (Terraform states) -- `registry` (Container Registry images) -- `pages` (Pages content) -- `repositories` (Git repositories data) -- `packages` (Packages) -- `ci_secure_files` (Project-level Secure Files) - -NOTE: -When [backing up and restoring Helm Charts](https://docs.gitlab.com/charts/architecture/backup-restore.html), there is an additional option `packages`, which refers to any packages managed by the GitLab [package registry](../user/packages/package_registry/index.md). -For more information see [command line arguments](https://docs.gitlab.com/charts/architecture/backup-restore.html#command-line-arguments). - -All wikis are backed up as part of the `repositories` group. Non-existent -wikis are skipped during a backup. - -For Omnibus GitLab packages: - -```shell -sudo gitlab-backup create SKIP=db,uploads -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -For installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=production -``` - -`SKIP=` is also used to: - -- [Skip creation of the tar file](#skipping-tar-creation) (`SKIP=tar`). -- [Skip uploading the backup to remote storage](#skip-uploading-backups-to-remote-storage) (`SKIP=remote`). - -### Skipping tar creation - -NOTE: -It is not possible to skip the tar creation when using [object storage](#upload-backups-to-a-remote-cloud-storage) for backups. - -The last part of creating a backup is generation of a `.tar` file containing -all the parts. In some cases (for example, if the backup is picked up by other -backup software) creating a `.tar` file might be wasted effort or even directly -harmful, so you can skip this step by adding `tar` to the `SKIP` environment -variable. - -Adding `tar` to the `SKIP` variable leaves the files and directories containing the -backup in the directory used for the intermediate files. These files are -overwritten when a new backup is created, so you should make sure they are copied -elsewhere, because you can only have one backup on the system. - -For Omnibus GitLab packages: - -```shell -sudo gitlab-backup create SKIP=tar -``` - -For installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create SKIP=tar RAILS_ENV=production -``` - -### Back up Git repositories concurrently - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37158) in GitLab 13.3. -> - [Concurrent restore introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69330) in GitLab 14.3 - -When using [multiple repository storages](../administration/repository_storage_paths.md), -repositories can be backed up or restored concurrently to help fully use CPU time. The -following variables are available to modify the default behavior of the Rake -task: - -- `GITLAB_BACKUP_MAX_CONCURRENCY`: The maximum number of projects to back up at - the same time. Defaults to the number of logical CPUs (in GitLab 14.1 and - earlier, defaults to `1`). -- `GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY`: The maximum number of projects to - back up at the same time on each storage. This allows the repository backups - to be spread across storages. Defaults to `2` (in GitLab 14.1 and earlier, - defaults to `1`). - -For example, for Omnibus GitLab installations with 4 repository storages: - -```shell -sudo gitlab-backup create GITLAB_BACKUP_MAX_CONCURRENCY=4 GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY=1 -``` - -For example, for installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create GITLAB_BACKUP_MAX_CONCURRENCY=4 GITLAB_BACKUP_MAX_STORAGE_CONCURRENCY=1 -``` - -### Incremental repository backups - -> - Introduced in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `incremental_repository_backup`. Disabled by default. -> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/355945) in GitLab 14.10. -> - `PREVIOUS_BACKUP` option [introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/4184) in GitLab 15.0. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `incremental_repository_backup`. -On GitLab.com, this feature is not available. - -NOTE: -Only repositories support incremental backups. Therefore, if you use `INCREMENTAL=yes`, the task -creates a self-contained backup tar archive. This is because all subtasks except repositories are -still creating full backups (they overwrite the existing full backup). -See [issue 19256](https://gitlab.com/gitlab-org/gitlab/-/issues/19256) for a feature request to -support incremental backups for all subtasks. - -Incremental repository backups can be faster than full repository backups because they only pack changes since the last backup into the backup bundle for each repository. -The incremental backup archives are not linked to each other: each archive is a self-contained backup of the instance. There must be an existing backup -to create an incremental backup from: - -- In GitLab 14.9 and 14.10, use the `BACKUP=<timestamp_of_backup>` option to choose the backup to use. The chosen previous backup is overwritten. -- In GitLab 15.0 and later, use the `PREVIOUS_BACKUP=<timestamp_of_backup>` option to choose the backup to use. By default, a backup file is created - as documented in the [Backup timestamp](backup_restore.md#backup-timestamp) section. You can override the `[TIMESTAMP]` portion of the filename by setting the - [`BACKUP` environment variable](#backup-filename). - -To create an incremental backup, run: - -- In GitLab 15.0 or later: - - ```shell - sudo gitlab-backup create INCREMENTAL=yes PREVIOUS_BACKUP=<timestamp_of_backup> - ``` - -- In GitLab 14.9 and 14.10: - - ```shell - sudo gitlab-backup create INCREMENTAL=yes BACKUP=<timestamp_of_backup> - ``` - -To create an [untarred](#skipping-tar-creation) incremental backup from a tarred backup, use `SKIP=tar`: - -```shell -sudo gitlab-backup create INCREMENTAL=yes SKIP=tar -``` - -### Back up specific repository storages - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86896) in GitLab 15.0. - -When using [multiple repository storages](../administration/repository_storage_paths.md), -repositories from specific repository storages can be backed up separately -using the `REPOSITORIES_STORAGES` option. The option accepts a comma-separated list of -storage names. - -For example, for Omnibus GitLab installations: - -```shell -sudo gitlab-backup create REPOSITORIES_STORAGES=storage1,storage2 -``` - -For example, for installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_STORAGES=storage1,storage2 -``` - -### Back up specific repositories - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. - -You can back up specific repositories using the `REPOSITORIES_PATHS` option. -Similarly, you can use `SKIP_REPOSITORIES_PATHS` to skip certain repositories. -Both options accept a comma-separated list of project or group paths. If you -specify a group path, all repositories in all projects in the group and -descendent groups are included or skipped, depending on which option you used. - -For example, to back up all repositories for all projects in **Group A** (`group-a`), the repository for **Project C** in **Group B** (`group-b/project-c`), -and skip the **Project D** in **Group A** (`group-a/project-d`): - -- Omnibus GitLab installations: - - ```shell - sudo gitlab-backup create REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d - ``` - -- Installations from source: - - ```shell - sudo -u git -H bundle exec rake gitlab:backup:create REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d - ``` - -### Upload backups to a remote (cloud) storage - -NOTE: -It is not possible to [skip the tar creation](#skipping-tar-creation) when using object storage for backups. - -You can let the backup script upload (using the [Fog library](https://fog.io/)) -the `.tar` file it creates. In the following example, we use Amazon S3 for -storage, but Fog also lets you use [other storage providers](https://fog.io/storage/). -GitLab also [imports cloud drivers](https://gitlab.com/gitlab-org/gitlab/-/blob/da46c9655962df7d49caef0e2b9f6bbe88462a02/Gemfile#L113) -for AWS, Google, and Aliyun. A local driver is -[also available](#upload-to-locally-mounted-shares). - -[Read more about using object storage with GitLab](../administration/object_storage.md). - -#### Using Amazon S3 - -For Omnibus GitLab packages: - -1. Add the following to `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-west-1', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123' - # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key - # 'use_iam_profile' => true - } - gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' - # Consider using multipart uploads when file size reaches 100MB. Enter a number in bytes. - # gitlab_rails['backup_multipart_chunk_size'] = 104857600 - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect - -#### S3 Encrypted Buckets - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64765) in GitLab 14.3. - -AWS supports these [modes for server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html): - -- Amazon S3-Managed Keys (SSE-S3) -- Customer Master Keys (CMKs) Stored in AWS Key Management Service (SSE-KMS) -- Customer-Provided Keys (SSE-C) - -Use your mode of choice with GitLab. Each mode has similar, but slightly -different, configuration methods. - -##### SSE-S3 - -To enable SSE-S3, in the backup storage options set the `server_side_encryption` -field to `AES256`. For example, in Omnibus GitLab: - -```ruby -gitlab_rails['backup_upload_storage_options'] = { - 'server_side_encryption' => 'AES256' -} -``` - -##### SSE-KMS - -To enable SSE-KMS, you need the -[KMS key via its Amazon Resource Name (ARN) in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). -Under the `backup_upload_storage_options` configuration setting, set: - -- `server_side_encryption` to `aws:kms`. -- `server_side_encryption_kms_key_id` to the ARN of the key. - -For example, in Omnibus GitLab: - -```ruby -gitlab_rails['backup_upload_storage_options'] = { - 'server_side_encryption' => 'aws:kms', - 'server_side_encryption_kms_key_id' => 'arn:aws:<YOUR KMS KEY ID>:' -} -``` - -##### SSE-C - -SSE-C requires you to set these encryption options: - -- `backup_encryption`: AES256. -- `backup_encryption_key`: Unencoded, 32-byte (256 bits) key. The upload fails if this isn't exactly 32 bytes. - -For example, in Omnibus GitLab: - -```ruby -gitlab_rails['backup_encryption'] = 'AES256' -gitlab_rails['backup_encryption_key'] = '<YOUR 32-BYTE KEY HERE>' -``` - -If the key contains binary characters and cannot be encoded in UTF-8, -instead, specify the key with the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable. -For example: - -```ruby -gitlab_rails['env'] = { 'GITLAB_BACKUP_ENCRYPTION_KEY' => "\xDE\xAD\xBE\xEF" * 8 } -``` - -#### Digital Ocean Spaces - -This example can be used for a bucket in Amsterdam (AMS3): - -1. Add the following to `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'ams3', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123', - 'endpoint' => 'https://ams3.digitaloceanspaces.com' - } - gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect - -If you see a `400 Bad Request` error message when using Digital Ocean Spaces, -the cause may be the use of backup encryption. Because Digital Ocean Spaces -doesn't support encryption, remove or comment the line that contains -`gitlab_rails['backup_encryption']`. - -#### Other S3 Providers - -Not all S3 providers are fully compatible with the Fog library. For example, -if you see a `411 Length Required` error message after attempting to upload, -you may need to downgrade the `aws_signature_version` value from the default -value to `2`, [due to this issue](https://github.com/fog/fog-aws/issues/428). - -For installations from source: - -1. Edit `home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - # snip - upload: - # Fog storage connection settings, see https://fog.io/storage/ . - connection: - provider: AWS - region: eu-west-1 - aws_access_key_id: AKIAKIAKI - aws_secret_access_key: 'secret123' - # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty - # ie. aws_access_key_id: '' - # use_iam_profile: 'true' - # The remote 'directory' to store your backups. For S3, this would be the bucket name. - remote_directory: 'my.s3.bucket' - # Specifies Amazon S3 storage class to use for backups, this is optional - # storage_class: 'STANDARD' - # - # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional - # 'encryption' must be set in order for this to have any effect. - # 'encryption_key' should be set to the 256-bit encryption key for Amazon S3 to use to encrypt or decrypt. - # To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` your data. - # encryption: 'AES256' - # encryption_key: '<key>' - # - # - # Turns on AWS Server-Side Encryption with Amazon S3-Managed keys (optional) - # https://docs.aws.amazon.com/AmazonS3/latest/userguide/serv-side-encryption.html - # For SSE-S3, set 'server_side_encryption' to 'AES256'. - # For SS3-KMS, set 'server_side_encryption' to 'aws:kms'. Set - # 'server_side_encryption_kms_key_id' to the ARN of customer master key. - # storage_options: - # server_side_encryption: 'aws:kms' - # server_side_encryption_kms_key_id: 'arn:aws:kms:YOUR-KEY-ID-HERE' - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect - -If you're uploading your backups to S3, you should create a new -IAM user with restricted access rights. To give the upload user access only for -uploading backups create the following IAM profile, replacing `my.s3.bucket` -with the name of your bucket: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "Stmt1412062044000", - "Effect": "Allow", - "Action": [ - "s3:AbortMultipartUpload", - "s3:GetBucketAcl", - "s3:GetBucketLocation", - "s3:GetObject", - "s3:GetObjectAcl", - "s3:ListBucketMultipartUploads", - "s3:PutObject", - "s3:PutObjectAcl" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket/*" - ] - }, - { - "Sid": "Stmt1412062097000", - "Effect": "Allow", - "Action": [ - "s3:GetBucketLocation", - "s3:ListAllMyBuckets" - ], - "Resource": [ - "*" - ] - }, - { - "Sid": "Stmt1412062128000", - "Effect": "Allow", - "Action": [ - "s3:ListBucket" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket" - ] - } - ] -} -``` - -#### Using Google Cloud Storage - -To use Google Cloud Storage to save backups, you must first create an -access key from the Google console: - -1. Go to the [Google storage settings page](https://console.cloud.google.com/storage/settings). -1. Select **Interoperability**, and then create an access key. -1. Make note of the **Access Key** and **Secret** and replace them in the - following configurations. -1. In the buckets advanced settings ensure the Access Control option - **Set object-level and bucket-level permissions** is selected. -1. Ensure you have already created a bucket. - -For Omnibus GitLab packages: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_upload_connection'] = { - 'provider' => 'Google', - 'google_storage_access_key_id' => 'Access Key', - 'google_storage_secret_access_key' => 'Secret', - - ## If you have CNAME buckets (foo.example.com), you might run into SSL issues - ## when uploading backups ("hostname foo.example.com.storage.googleapis.com - ## does not match the server certificate"). In that case, uncomnent the following - ## setting. See: https://github.com/fog/fog/issues/2834 - #'path_style' => true - } - gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket' - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect - -For installations from source: - -1. Edit `home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - upload: - connection: - provider: 'Google' - google_storage_access_key_id: 'Access Key' - google_storage_secret_access_key: 'Secret' - remote_directory: 'my.google.bucket' - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect - -#### Using Azure Blob storage - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25877) in GitLab 13.4. - -For Omnibus GitLab packages: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AzureRM', - 'azure_storage_account_name' => '<AZURE STORAGE ACCOUNT NAME>', - 'azure_storage_access_key' => '<AZURE STORAGE ACCESS KEY>', - 'azure_storage_domain' => 'blob.core.windows.net', # Optional - } - gitlab_rails['backup_upload_remote_directory'] = '<AZURE BLOB CONTAINER>' - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect - -For installations from source: - -1. Edit `home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - upload: - connection: - provider: 'AzureRM' - azure_storage_account_name: '<AZURE STORAGE ACCOUNT NAME>' - azure_storage_access_key: '<AZURE STORAGE ACCESS KEY>' - remote_directory: '<AZURE BLOB CONTAINER>' - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect - -For more details, see the [table of Azure parameters](../administration/object_storage.md#azure-blob-storage). - -#### Specifying a custom directory for backups - -This option works only for remote storage. If you want to group your backups, -you can pass a `DIRECTORY` environment variable: - -```shell -sudo gitlab-backup create DIRECTORY=daily -sudo gitlab-backup create DIRECTORY=weekly -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -### Skip uploading backups to remote storage - -If you have configured GitLab to [upload backups in a remote storage](#upload-backups-to-a-remote-cloud-storage), -you can use the `SKIP=remote` option to skip uploading your backups to the remote storage. - -For Omnibus GitLab packages: - -```shell -sudo gitlab-backup create SKIP=remote -``` - -For installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:create SKIP=remote RAILS_ENV=production -``` - -### Upload to locally-mounted shares - -You can send backups to a locally-mounted share (for example, `NFS`,`CIFS`, or `SMB`) using the Fog -[`Local`](https://github.com/fog/fog-local#usage) storage provider. - -To do this, you must set the following configuration keys: - -- `backup_upload_connection.local_root`: mounted directory that backups are copied to. -- `backup_upload_remote_directory`: subdirectory of the `backup_upload_connection.local_root` directory. It is created if it doesn't exist. - If you want to copy the tarballs to the root of your mounted directory, use `.`. - -When mounted, the directory set in the `local_root` key must be owned by either: - -- The `git` user. So, mounting with the `uid=` of the `git` user for `CIFS` and `SMB`. -- The user that you are executing the backup tasks as. For Omnibus GitLab, this is the `git` user. - -Because file system performance may affect overall GitLab performance, -[we don't recommend using cloud-based file systems for storage](../administration/nfs.md#avoid-using-cloud-based-file-systems). - -#### Avoid conflicting configuration - -Don't set the following configuration keys to the same path: - -- `gitlab_rails['backup_path']` (`backup.path` for source installations). -- `gitlab_rails['backup_upload_connection'].local_root` (`backup.upload.connection.local_root` for source installations). - -The `backup_path` configuration key sets the local location of the backup file. The `upload` configuration key is -intended for use when the backup file is uploaded to a separate server, perhaps for archival purposes. - -If these configuration keys are set to the same location, the upload feature fails because a backup already exists at -the upload location. This failure causes the upload feature to delete the backup because it assumes it's a residual file -remaining after the failed upload attempt. - -#### Configure uploads to locally-mounted shares - -For Omnibus GitLab packages: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_upload_connection'] = { - :provider => 'Local', - :local_root => '/mnt/backups' - } - - # The directory inside the mounted folder to copy backups to - # Use '.' to store them in the root directory - gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups' - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect. - -For installations from source: - -1. Edit `home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - upload: - # Fog storage connection settings, see https://fog.io/storage/ . - connection: - provider: Local - local_root: '/mnt/backups' - # The directory inside the mounted folder to copy backups to - # Use '.' to store them in the root directory - remote_directory: 'gitlab_backups' - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect. - -### Backup archive permissions - -The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`) -have the owner/group `git`/`git` and 0600 permissions by default. This is -meant to avoid other system users reading GitLab data. If you need the backup -archives to have different permissions, you can use the `archive_permissions` -setting. - -For Omnibus GitLab packages: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect. - -For installations from source: - -1. Edit `/home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - archive_permissions: 0644 # Makes the backup archives world-readable - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect. - -### Configuring cron to make daily backups - -WARNING: -The following cron jobs do not [back up your GitLab configuration files](#storing-configuration-files) -or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). - -You can schedule a cron job that backs up your repositories and GitLab metadata. - -For Omnibus GitLab packages: - -1. Edit the crontab for the `root` user: - - ```shell - sudo su - - crontab -e - ``` - -1. There, add the following line to schedule the backup for everyday at 2 AM: - - ```plaintext - 0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1 - ``` - - Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:create` instead. - -For installations from source: - -1. Edit the crontab for the `git` user: - - ```shell - sudo -u git crontab -e - ``` - -1. Add the following lines at the bottom: - - ```plaintext - # Create a full backup of the GitLab repositories and SQL database every day at 2am - 0 2 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 - ``` - -The `CRON=1` environment setting directs the backup script to hide all progress -output if there aren't any errors. This is recommended to reduce cron spam. -When troubleshooting backup problems, however, replace `CRON=1` with `--trace` to log verbosely. - -## Limit backup lifetime for local files (prune old backups) - -WARNING: -The process described in this section doesn't work if you used a [custom filename](#backup-filename) -for your backups. - -To prevent regular backups from using all your disk space, you may want to set a limited lifetime -for backups. The next time the backup task runs, backups older than the `backup_keep_time` are -pruned. - -This configuration option manages only local files. GitLab doesn't prune old -files stored in a third-party [object storage](#upload-backups-to-a-remote-cloud-storage) -because the user may not have permission to list and delete files. It's -recommended that you configure the appropriate retention policy for your object -storage (for example, [AWS S3](https://docs.aws.amazon.com/AmazonS3/latest/user-guide/create-lifecycle.html)). - -For Omnibus GitLab packages: - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - ## Limit backup lifetime to 7 days - 604800 seconds - gitlab_rails['backup_keep_time'] = 604800 - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect. - -For installations from source: - -1. Edit `/home/git/gitlab/config/gitlab.yml`: - - ```yaml - backup: - ## Limit backup lifetime to 7 days - 604800 seconds - keep_time: 604800 - ``` - -1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) - for the changes to take effect. +<!-- This redirect file can be deleted after <2023-09-26>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 1fd772c06da..97cafefb45d 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -2,1093 +2,13 @@ stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../administration/backup_restore/index.md' +remove_date: '2023-09-26' --- -# Back up and restore GitLab **(FREE SELF)** +This document was moved to [another location](../administration/backup_restore/index.md). -GitLab provides Rake tasks for backing up and restoring GitLab instances. - -An application data backup creates an archive file that contains the database, -all repositories and all attachments. - -You can only restore a backup to **exactly the same version and type (CE/EE)** -of GitLab on which it was created. The best way to -[migrate your projects from one server to another](#migrate-to-a-new-server) is through a backup and restore. - -WARNING: -GitLab doesn't back up items that aren't stored on the file system. If you're -using [object storage](../administration/object_storage.md), be sure to enable -backups with your object storage provider, if desired. - -## Requirements - -To be able to back up and restore, ensure that Rsync is installed on your -system. If you installed GitLab: - -- _Using the Omnibus package_, Rsync is already installed. -- _From source_, check if `rsync` is installed. If Rsync is not installed, install it. For example: - - ```shell - # Debian/Ubuntu - sudo apt-get install rsync - - # RHEL/CentOS - sudo yum install rsync - ``` - -### `gitaly-backup` for repository backup and restore - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333034) in GitLab 14.2. -> - [Deployed behind a feature flag](../user/feature_flags.md), enabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/333034) in GitLab 14.10. [Feature flag `gitaly_backup`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83254) removed. - -The `gitaly-backup` binary is used by the backup Rake task to create and restore repository backups from Gitaly. -`gitaly-backup` replaces the previous backup method that directly calls RPCs on Gitaly from GitLab. - -The backup Rake task must be able to find this executable. In most cases, you don't need to change -the path to the binary as it should work fine with the default path `/opt/gitlab/embedded/bin/gitaly-backup`. -If you have a specific reason to change the path, it can be configured in Omnibus GitLab packages: - -1. Add the following to `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['backup_gitaly_backup_path'] = '/path/to/gitaly-backup' - ``` - -1. [Reconfigure GitLab](../administration/restart_gitlab.md#reconfigure-a-linux-package-installation) - for the changes to take effect. - -## Backup timestamp - -The backup archive is saved in `backup_path`, which is specified in the -`config/gitlab.yml` file. The filename is `[TIMESTAMP]_gitlab_backup.tar`, -where `TIMESTAMP` identifies the time at which each backup was created, plus -the GitLab version. The timestamp is needed if you need to restore GitLab and -multiple backups are available. - -For example, if the backup name is `1493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar`, -the timestamp is `1493107454_2018_04_25_10.6.4-ce`. - -## Back up GitLab - -For detailed information on backing up GitLab, see [Backup GitLab](backup_gitlab.md). - -## Restore GitLab - -For detailed information on restoring GitLab, see [Restore GitLab](restore_gitlab.md). - -## Alternative backup strategies - -In the following cases, consider using file system data transfer or snapshots as part of your backup strategy: - -- Your GitLab instance contains a lot of Git repository data and the GitLab backup script is too slow. -- Your GitLab instance has a lot of forked projects and the regular backup task duplicates the Git data for all of them. -- Your GitLab instance has a problem and using the regular backup and import Rake tasks isn't possible. - -WARNING: -Gitaly Cluster [does not support snapshot backups](../administration/gitaly/index.md#snapshot-backup-and-recovery-limitations). - -When considering using file system data transfer or snapshots: - -- Don't use these methods to migrate from one operating system to another. The operating systems of the source and destination should be as similar as possible. For example, - don't use these methods to migrate from Ubuntu to Fedora. -- Data consistency is very important. We recommend stopping GitLab with `sudo gitlab-ctl stop` before taking doing a file system transfer (with rsync, for example) or taking a - snapshot. - -Example: Amazon Elastic Block Store (EBS) - -> A GitLab server using Omnibus GitLab hosted on Amazon AWS. -> An EBS drive containing an ext4 file system is mounted at `/var/opt/gitlab`. -> In this case you could make an application backup by taking an EBS snapshot. -> The backup includes all repositories, uploads and PostgreSQL data. - -Example: Logical Volume Manager (LVM) snapshots + rsync - -> A GitLab server using Omnibus GitLab, with an LVM logical volume mounted at `/var/opt/gitlab`. -> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. -> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only file system at `/mnt/gitlab_backup`. -> Now we can have a longer running rsync job which creates a consistent replica on the remote server. -> The replica includes all repositories, uploads and PostgreSQL data. - -If you're running GitLab on a virtualized server, you can possibly also create -VM snapshots of the entire GitLab server. It's not uncommon however for a VM -snapshot to require you to power down the server, which limits this solution's -practical use. - -### Back up repository data separately - -First, ensure you back up existing GitLab data while [skipping repositories](backup_gitlab.md#excluding-specific-directories-from-the-backup): - -```shell -# for Omnibus GitLab package installations -sudo gitlab-backup create SKIP=repositories - -# for installations from source: -sudo -u git -H bundle exec rake gitlab:backup:create SKIP=repositories RAILS_ENV=production -``` - -For manually backing up the Git repository data on disk, there are multiple possible strategies: - -- Use snapshots, such as the previous examples of Amazon EBS drive snapshots, or LVM snapshots + rsync. -- Use [GitLab Geo](../administration/geo/index.md) and rely on the repository data on a Geo secondary site. -- [Prevent writes and copy the Git repository data](#prevent-writes-and-copy-the-git-repository-data). -- [Create an online backup by marking repositories as read-only (experimental)](#online-backup-through-marking-repositories-as-read-only-experimental). - -#### Prevent writes and copy the Git repository data - -Git repositories must be copied in a consistent way. They should not be copied during concurrent write -operations, as this can lead to inconsistencies or corruption issues. For more details, -[issue #270422](https://gitlab.com/gitlab-org/gitlab/-/issues/270422 "Provide documentation on preferred method of migrating Gitaly servers") -has a longer discussion explaining the potential problems. - -To prevent writes to the Git repository data, there are two possible approaches: - -- Use [maintenance mode](../administration/maintenance_mode/index.md) to place GitLab in a read-only state. -- Create explicit downtime by stopping all Gitaly services before backing up the repositories: - - ```shell - sudo gitlab-ctl stop gitaly - # execute git data copy step - sudo gitlab-ctl start gitaly - ``` - -You can copy Git repository data using any method, as long as writes are prevented on the data being copied -(to prevent inconsistencies and corruption issues). In order of preference and safety, the recommended methods are: - -1. Use `rsync` with archive-mode, delete, and checksum options, for example: - - ```shell - rsync -aR --delete --checksum source destination # be extra safe with the order as it will delete existing data if inverted - ``` - -1. Use a [`tar` pipe to copy the entire repository's directory to another server or location](../administration/operations/moving_repositories.md#tar-pipe-to-another-server). - -1. Use `sftp`, `scp`, `cp`, or any other copying method. - -#### Online backup through marking repositories as read-only (experimental) - -One way of backing up repositories without requiring instance-wide downtime -is to programmatically mark projects as read-only while copying the underlying data. - -There are a few possible downsides to this: - -- Repositories are read-only for a period of time that scales with the size of the repository. -- Backups take a longer time to complete due to marking each project as read-only, potentially leading to inconsistencies. For example, - a possible date discrepancy between the last data available for the first project that gets backed up compared to - the last project that gets backed up. -- Fork networks should be entirely read-only while the projects inside get backed up to prevent potential changes to the pool repository. - -There is an **experimental** script that attempts to automate this process in -[the Geo team Runbooks project](https://gitlab.com/gitlab-org/geo-team/runbooks/-/tree/main/experimental-online-backup-through-rsync). - -## Back up and restore for installations using PgBouncer - -Do not back up or restore GitLab through a PgBouncer connection. These -tasks must [bypass PgBouncer and connect directly to the PostgreSQL primary database node](#bypassing-pgbouncer), -or they cause a GitLab outage. - -When the GitLab backup or restore task is used with PgBouncer, the -following error message is shown: - -```ruby -ActiveRecord::StatementInvalid: PG::UndefinedTable -``` - -Each time the GitLab backup runs, GitLab starts generating 500 errors and errors about missing -tables will [be logged by PostgreSQL](../administration/logs/index.md#postgresql-logs): - -```plaintext -ERROR: relation "tablename" does not exist at character 123 -``` - -This happens because the task uses `pg_dump`, which -[sets a null search path and explicitly includes the schema in every SQL query](https://gitlab.com/gitlab-org/gitlab/-/issues/23211) -to address [CVE-2018-1058](https://www.postgresql.org/about/news/postgresql-103-968-9512-9417-and-9322-released-1834/). - -Since connections are reused with PgBouncer in transaction pooling mode, -PostgreSQL fails to search the default `public` schema. As a result, -this clearing of the search path causes tables and columns to appear -missing. - -### Bypassing PgBouncer - -There are two ways to fix this: - -1. [Use environment variables to override the database settings](#environment-variable-overrides) for the backup task. -1. Reconfigure a node to [connect directly to the PostgreSQL primary database node](../administration/postgresql/pgbouncer.md#procedure-for-bypassing-pgbouncer). - -#### Environment variable overrides - -By default, GitLab uses the database configuration stored in a -configuration file (`database.yml`). However, you can override the database settings -for the backup and restore task by setting environment -variables that are prefixed with `GITLAB_BACKUP_`: - -- `GITLAB_BACKUP_PGHOST` -- `GITLAB_BACKUP_PGUSER` -- `GITLAB_BACKUP_PGPORT` -- `GITLAB_BACKUP_PGPASSWORD` -- `GITLAB_BACKUP_PGSSLMODE` -- `GITLAB_BACKUP_PGSSLKEY` -- `GITLAB_BACKUP_PGSSLCERT` -- `GITLAB_BACKUP_PGSSLROOTCERT` -- `GITLAB_BACKUP_PGSSLCRL` -- `GITLAB_BACKUP_PGSSLCOMPRESSION` - -For example, to override the database host and port to use 192.168.1.10 -and port 5432 with the Omnibus package: - -```shell -sudo GITLAB_BACKUP_PGHOST=192.168.1.10 GITLAB_BACKUP_PGPORT=5432 /opt/gitlab/bin/gitlab-backup create -``` - -See the [PostgreSQL documentation](https://www.postgresql.org/docs/12/libpq-envars.html) -for more details on what these parameters do. - -## Migrate to a new server - -<!-- some details borrowed from GitLab.com move from Azure to GCP detailed at https://gitlab.com/gitlab-com/migration/-/blob/master/.gitlab/issue_templates/failover.md --> - -You can use GitLab backup and restore to migrate your instance to a new server. This section outlines a typical procedure for a GitLab deployment running on a single server. -If you're running GitLab Geo, an alternative option is [Geo disaster recovery for planned failover](../administration/geo/disaster_recovery/planned_failover.md). - -WARNING: -Avoid uncoordinated data processing by both the new and old servers, where multiple -servers could connect concurrently and process the same data. For example, when using -[incoming email](../administration/incoming_email.md), if both GitLab instances are -processing email at the same time, then both instances miss some data. -This type of problem can occur with other services as well, such as a -[non-packaged database](https://docs.gitlab.com/omnibus/settings/database.html#using-a-non-packaged-postgresql-database-management-server), -a non-packaged Redis instance, or non-packaged Sidekiq. - -Prerequisites: - -- Some time before your migration, consider notifying your users of upcoming - scheduled maintenance with a [broadcast message banner](../user/admin_area/broadcast_messages.md). -- Ensure your backups are complete and current. Create a complete system-level backup, or - take a snapshot of all servers involved in the migration, in case destructive commands - (like `rm`) are run incorrectly. - -### Prepare the new server - -To prepare the new server: - -1. Copy the - [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079) - from the old server to avoid man-in-the-middle attack warnings. - See [Manually replicate the primary site's SSH host keys](../administration/geo/replication/configuration.md#step-2-manually-replicate-the-primary-sites-ssh-host-keys) for example steps. -1. [Install and configure GitLab](https://about.gitlab.com/install/) except - [incoming email](../administration/incoming_email.md): - 1. Install GitLab. - 1. Configure by copying `/etc/gitlab` files from the old server to the new server, and update as necessary. - Read the - [Omnibus configuration backup and restore instructions](https://docs.gitlab.com/omnibus/settings/backups.html) for more detail. - 1. If applicable, disable [incoming email](../administration/incoming_email.md). - 1. Block new CI/CD jobs from starting upon initial startup after the backup and restore. - Edit `/etc/gitlab/gitlab.rb` and set the following: - - ```ruby - nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" - ``` - - 1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -1. Stop GitLab to avoid any potential unnecessary and unintentional data processing: - - ```shell - sudo gitlab-ctl stop - ``` - -1. Configure the new server to allow receiving the Redis database and GitLab backup files: - - ```shell - sudo rm -f /var/opt/gitlab/redis/dump.rdb - sudo chown <your-linux-username> /var/opt/gitlab/redis /var/opt/gitlab/backups - ``` - -### Prepare and transfer content from the old server - -1. Ensure you have an up-to-date system-level backup or snapshot of the old server. -1. Enable [maintenance mode](../administration/maintenance_mode/index.md), - if supported by your GitLab edition. -1. Block new CI/CD jobs from starting: - 1. Edit `/etc/gitlab/gitlab.rb`, and set the following: - - ```ruby - nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" - ``` - - 1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -1. Disable periodic background jobs: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. On the left sidebar, select **Monitoring > Background Jobs**. - 1. Under the Sidekiq dashboard, select **Cron** tab and then - **Disable All**. -1. Wait for the currently running CI/CD jobs to finish, or accept that jobs that have not completed may be lost. - To view jobs currently running, on the left sidebar, select **Overviews > Jobs**, - and then select **Running**. -1. Wait for Sidekiq jobs to finish: - 1. On the left sidebar, select **Monitoring > Background Jobs**. - 1. Under the Sidekiq dashboard, select **Queues** and then **Live Poll**. - Wait for **Busy** and **Enqueued** to drop to 0. - These queues contain work that has been submitted by your users; - shutting down before these jobs complete may cause the work to be lost. - Make note of the numbers shown in the Sidekiq dashboard for post-migration verification. -1. Flush the Redis database to disk, and stop GitLab other than the services needed for migration: - - ```shell - sudo /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket save && sudo gitlab-ctl stop && sudo gitlab-ctl start postgresql && sudo gitlab-ctl start gitaly - ``` - -1. Create a GitLab backup: - - ```shell - sudo gitlab-backup create - ``` - -1. Disable the following GitLab services and prevent unintentional restarts by adding the following to the bottom of `/etc/gitlab/gitlab.rb`: - - ```ruby - alertmanager['enable'] = false - gitlab_exporter['enable'] = false - gitlab_pages['enable'] = false - gitlab_workhorse['enable'] = false - grafana['enable'] = false - logrotate['enable'] = false - gitlab_rails['incoming_email_enabled'] = false - nginx['enable'] = false - node_exporter['enable'] = false - postgres_exporter['enable'] = false - postgresql['enable'] = false - prometheus['enable'] = false - puma['enable'] = false - redis['enable'] = false - redis_exporter['enable'] = false - registry['enable'] = false - sidekiq['enable'] = false - ``` - -1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -1. Verify everything is stopped, and confirm no services are running: - - ```shell - sudo gitlab-ctl status - ``` - -1. Transfer the Redis database and GitLab backups to the new server: - - ```shell - sudo scp /var/opt/gitlab/redis/dump.rdb <your-linux-username>@new-server:/var/opt/gitlab/redis - sudo scp /var/opt/gitlab/backups/your-backup.tar <your-linux-username>@new-server:/var/opt/gitlab/backups - ``` - -### Restore data on the new server - -1. Restore appropriate file system permissions: - - ```shell - sudo chown gitlab-redis /var/opt/gitlab/redis - sudo chown gitlab-redis:gitlab-redis /var/opt/gitlab/redis/dump.rdb - sudo chown git:root /var/opt/gitlab/backups - sudo chown git:git /var/opt/gitlab/backups/your-backup.tar - ``` - -1. [Restore the GitLab backup](#restore-gitlab). -1. Verify that the Redis database restored correctly: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. On the left sidebar, select **Monitoring > Background Jobs**. - 1. Under the Sidekiq dashboard, verify that the numbers - match with what was shown on the old server. - 1. While still under the Sidekiq dashboard, select **Cron** and then **Enable All** - to re-enable periodic background jobs. -1. Test that read-only operations on the GitLab instance work as expected. For example, browse through project repository files, merge requests, and issues. -1. Disable [Maintenance Mode](../administration/maintenance_mode/index.md), if previously enabled. -1. Test that the GitLab instance is working as expected. -1. If applicable, re-enable [incoming email](../administration/incoming_email.md) and test it is working as expected. -1. Update your DNS or load balancer to point at the new server. -1. Unblock new CI/CD jobs from starting by removing the custom NGINX configuration - you added previously: - - ```ruby - # The following line must be removed - nginx['custom_gitlab_server_config'] = "location = /api/v4/jobs/request {\n deny all;\n return 503;\n }\n" - ``` - -1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -1. Remove the scheduled maintenance [broadcast message banner](../user/admin_area/broadcast_messages.md). - -## Additional notes - -This documentation is for GitLab Community and Enterprise Edition. We back up -GitLab.com and ensure your data is secure. You can't, however, use these -methods to export or back up your data yourself from GitLab.com. - -Issues are stored in the database, and can't be stored in Git itself. - -To migrate your repositories from one server to another with an up-to-date -version of GitLab, use the [import Rake task](import.md) to do a mass import of -the repository. If you do an import Rake task rather than a backup restore, -you get all of your repositories, but no other data. - -## Troubleshooting - -The following are possible problems you might encounter, along with potential -solutions. - -### Restoring database backup using Omnibus packages outputs warnings - -If you're using backup restore procedures, you may encounter the following -warning messages: - -```plaintext -ERROR: must be owner of extension pg_trgm -ERROR: must be owner of extension btree_gist -ERROR: must be owner of extension plpgsql -WARNING: no privileges could be revoked for "public" (two occurrences) -WARNING: no privileges were granted for "public" (two occurrences) -``` - -Be advised that the backup is successfully restored in spite of these warning -messages. - -The Rake task runs this as the `gitlab` user, which doesn't have superuser -access to the database. When restore is initiated, it also runs as the `gitlab` -user, but it also tries to alter the objects it doesn't have access to. -Those objects have no influence on the database backup or restore, but display -a warning message. - -For more information, see: - -- PostgreSQL issue tracker: - - [Not being a superuser](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com). - - [Having different owners](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us). - -- Stack Overflow: [Resulting errors](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). - -### When the secrets file is lost - -If you didn't [back up the secrets file](backup_gitlab.md#storing-configuration-files), you -must complete several steps to get GitLab working properly again. - -The secrets file is responsible for storing the encryption key for the columns -that contain required, sensitive information. If the key is lost, GitLab can't -decrypt those columns, preventing access to the following items: - -- [CI/CD variables](../ci/variables/index.md) -- [Kubernetes / GCP integration](../user/infrastructure/clusters/index.md) -- [Custom Pages domains](../user/project/pages/custom_domains_ssl_tls_certification/index.md) -- [Project error tracking](../operations/error_tracking.md) -- [Runner authentication](../ci/runners/index.md) -- [Project mirroring](../user/project/repository/mirror/index.md) -- [Integrations](../user/project/integrations/index.md) -- [Web hooks](../user/project/integrations/webhooks.md) - -In cases like CI/CD variables and runner authentication, you can experience -unexpected behaviors, such as: - -- Stuck jobs. -- 500 errors. - -In this case, you must reset all the tokens for CI/CD variables and -runner authentication, which is described in more detail in the following -sections. After resetting the tokens, you should be able to visit your project -and the jobs begin running again. - -Use the information in the following sections at your own risk. - -#### Verify that all values can be decrypted - -You can determine if your database contains values that can't be decrypted by using a -[Rake task](../administration/raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). - -#### Take a backup - -You must directly modify GitLab data to work around your lost secrets file. - -WARNING: -Be sure to create a full database backup before attempting any changes. - -#### Disable user two-factor authentication (2FA) - -Users with 2FA enabled can't sign in to GitLab. In that case, you must -[disable 2FA for everyone](../security/two_factor_authentication.md#for-all-users), -after which users must reactivate 2FA. - -#### Reset CI/CD variables - -1. Enter the database console: - - For Omnibus GitLab 14.1 and earlier: - - ```shell - sudo gitlab-rails dbconsole - ``` - - For Omnibus GitLab 14.2 and later: - - ```shell - sudo gitlab-rails dbconsole --database main - ``` - - For installations from source, GitLab 14.1 and earlier: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production - ``` - - For installations from source, GitLab 14.2 and later: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production --database main - ``` - -1. Examine the `ci_group_variables` and `ci_variables` tables: - - ```sql - SELECT * FROM public."ci_group_variables"; - SELECT * FROM public."ci_variables"; - ``` - - These are the variables that you need to delete. - -1. Delete all variables: - - ```sql - DELETE FROM ci_group_variables; - DELETE FROM ci_variables; - ``` - -1. If you know the specific group or project from which you wish to delete variables, you can include a `WHERE` statement to specify that in your `DELETE`: - - ```sql - DELETE FROM ci_group_variables WHERE group_id = <GROUPID>; - DELETE FROM ci_variables WHERE project_id = <PROJECTID>; - ``` - -You may need to reconfigure or restart GitLab for the changes to take effect. - -#### Reset runner registration tokens - -1. Enter the database console: - - For Omnibus GitLab 14.1 and earlier: - - ```shell - sudo gitlab-rails dbconsole - ``` - - For Omnibus GitLab 14.2 and later: - - ```shell - sudo gitlab-rails dbconsole --database main - ``` - - For installations from source, GitLab 14.1 and earlier: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production - ``` - - For installations from source, GitLab 14.2 and later: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production --database main - ``` - -1. Clear all tokens for projects, groups, and the entire instance: - - WARNING: - The final `UPDATE` operation stops the runners from being able to pick - up new jobs. You must register new runners. - - ```sql - -- Clear project tokens - UPDATE projects SET runners_token = null, runners_token_encrypted = null; - -- Clear group tokens - UPDATE namespaces SET runners_token = null, runners_token_encrypted = null; - -- Clear instance tokens - UPDATE application_settings SET runners_registration_token_encrypted = null; - -- Clear key used for JWT authentication - -- This may break the $CI_JWT_TOKEN job variable: - -- https://gitlab.com/gitlab-org/gitlab/-/issues/325965 - UPDATE application_settings SET encrypted_ci_jwt_signing_key = null; - -- Clear runner tokens - UPDATE ci_runners SET token = null, token_encrypted = null; - ``` - -#### Reset pending pipeline jobs - -1. Enter the database console: - - For Omnibus GitLab 14.1 and earlier: - - ```shell - sudo gitlab-rails dbconsole - ``` - - For Omnibus GitLab 14.2 and later: - - ```shell - sudo gitlab-rails dbconsole --database main - ``` - - For installations from source, GitLab 14.1 and earlier: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production - ``` - - For installations from source, GitLab 14.2 and later: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production --database main - ``` - -1. Clear all the tokens for pending jobs: - - For GitLab 15.3 and earlier: - - ```sql - -- Clear build tokens - UPDATE ci_builds SET token = null, token_encrypted = null; - ``` - - For GitLab 15.4 and later: - - ```sql - -- Clear build tokens - UPDATE ci_builds SET token_encrypted = null; - ``` - -A similar strategy can be employed for the remaining features. By removing the -data that can't be decrypted, GitLab can be returned to operation, and the -lost data can be manually replaced. - -#### Fix integrations and webhooks - -If you've lost your secrets, the [integrations settings pages](../user/project/integrations/index.md) -and [webhooks settings pages](../user/project/integrations/webhooks.md) are probably displaying `500` error messages. - -The fix is to truncate the affected tables (those containing encrypted columns). -This deletes all your configured integrations, webhooks, and related metadata. -You should verify that the secrets are the root cause before deleting any data. - -1. Enter the database console: - - For Omnibus GitLab 14.1 and earlier: - - ```shell - sudo gitlab-rails dbconsole - ``` - - For Omnibus GitLab 14.2 and later: - - ```shell - sudo gitlab-rails dbconsole --database main - ``` - - For installations from source, GitLab 14.1 and earlier: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production - ``` - - For installations from source, GitLab 14.2 and later: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production --database main - ``` - -1. Truncate the following tables: - - ```sql - -- truncate web_hooks table - TRUNCATE integrations, chat_names, issue_tracker_data, jira_tracker_data, slack_integrations, web_hooks, zentao_tracker_data, web_hook_logs; - ``` - -### Container Registry push failures after restoring from a backup - -If you use the [Container Registry](../user/packages/container_registry/index.md), -pushes to the registry may fail after restoring your backup on an Omnibus GitLab -instance after restoring the registry data. - -These failures mention permission issues in the registry logs, similar to: - -```plaintext -level=error -msg="response completed with error" -err.code=unknown -err.detail="filesystem: mkdir /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/...: permission denied" -err.message="unknown error" -``` - -This issue is caused by the restore running as the unprivileged user `git`, -which is unable to assign the correct ownership to the registry files during -the restore process ([issue #62759](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62759 "Incorrect permissions on registry filesystem after restore")). - -To get your registry working again: - -```shell -sudo chown -R registry:registry /var/opt/gitlab/gitlab-rails/shared/registry/docker -``` - -If you changed the default file system location for the registry, run `chown` -against your custom location, instead of `/var/opt/gitlab/gitlab-rails/shared/registry/docker`. - -### Backup fails to complete with Gzip error - -When running the backup, you may receive a Gzip error message: - -```shell -sudo /opt/gitlab/bin/gitlab-backup create -... -Dumping ... -... -gzip: stdout: Input/output error - -Backup failed -``` - -If this happens, examine the following: - -- Confirm there is sufficient disk space for the Gzip operation. It's not uncommon for backups that - use the [default strategy](backup_gitlab.md#backup-strategy-option) to require half the instance size - in free disk space during backup creation. -- If NFS is being used, check if the mount option `timeout` is set. The - default is `600`, and changing this to smaller values results in this error. - -### Backup fails with `File name too long` error - -During backup, you can get the `File name too long` error ([issue #354984](https://gitlab.com/gitlab-org/gitlab/-/issues/354984)). For example: - -```plaintext -Problem: <class 'OSError: [Errno 36] File name too long: -``` - -This problem stops the backup script from completing. To fix this problem, you must truncate the filenames causing the problem. A maximum of 246 characters, including the file extension, is permitted. - -WARNING: -The steps in this section can potentially lead to **data loss**. All steps must be followed strictly in the order given. - -Truncating filenames to resolve the error involves: - -- Cleaning up remote uploaded files that aren't tracked in the database. -- Truncating the filenames in the database. -- Rerunning the backup task. - -#### Clean up remote uploaded files - -A [known issue](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45425) caused object store uploads to remain after a parent resource was deleted. This issue was [resolved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18698). - -To fix these files, you must clean up all remote uploaded files that are in the storage but not tracked in the `uploads` database table. - -1. List all the object store upload files that can be moved to a lost and found directory if they don't exist in the GitLab database: - - ```shell - bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production - ``` - -1. If you are sure you want to delete these files and remove all non-referenced uploaded files, run: - - WARNING: - The following action is **irreversible**. - - ```shell - bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production DRY_RUN=false - ``` - -#### Truncate the filenames referenced by the database - -You must truncate the files referenced by the database that are causing the problem. The filenames referenced by the database are stored: - -- In the `uploads` table. -- In the references found. Any reference found from other database tables and columns. -- On the file system. - -Truncate the filenames in the `uploads` table: - -1. Enter the database console: - - For Omnibus GitLab 14.2 and later: - - ```shell - sudo gitlab-rails dbconsole --database main - ``` - - For Omnibus GitLab 14.1 and earlier: - - ```shell - sudo gitlab-rails dbconsole - ``` - - For installations from source, GitLab 14.2 and later: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production --database main - ``` - - For installations from source, GitLab 14.1 and earlier: - - ```shell - sudo -u git -H bundle exec rails dbconsole -e production - ``` - -1. Search the `uploads` table for filenames longer than 246 characters: - - The following query selects the `uploads` records with filenames longer than 246 characters in batches of 0 to 10000. This improves the performance on large GitLab instances with tables having thousand of records. - - ```sql - CREATE TEMP TABLE uploads_with_long_filenames AS - SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, id, path - FROM uploads AS u - WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; - - CREATE INDEX ON uploads_with_long_filenames(row_id); - - SELECT - u.id, - u.path, - -- Current filename - (regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] AS current_filename, - -- New filename - CONCAT( - LEFT(SPLIT_PART((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), - COALESCE(SUBSTRING((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) - ) AS new_filename, - -- New path - CONCAT( - COALESCE((regexp_match(u.path, '(.*\/).*'))[1], ''), - CONCAT( - LEFT(SPLIT_PART((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), - COALESCE(SUBSTRING((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) - ) - ) AS new_path - FROM uploads_with_long_filenames AS u - WHERE u.row_id > 0 AND u.row_id <= 10000; - ``` - - Output example: - - ```postgresql - -[ RECORD 1 ]----+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- - id | 34 - path | public/@hashed/loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisit.txt - current_filename | loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisit.txt - new_filename | loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelits.txt - new_path | public/@hashed/loremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelitsedvulputatemisitloremipsumdolorsitametconsecteturadipiscingelitseddoeiusmodtemporincididuntutlaboreetdoloremagnaaliquaauctorelits.txt - ``` - - Where: - - - `current_filename`: a filename that is currently more than 246 characters long. - - `new_filename`: a filename that has been truncated to 246 characters maximum. - - `new_path`: new path considering the `new_filename` (truncated). - - Once you validate the batch results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. - -1. Rename the files found in the `uploads` table from long filenames to new truncated filenames. The following query rolls back the update so you can check the results safely within a transaction wrapper: - - ```sql - CREATE TEMP TABLE uploads_with_long_filenames AS - SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, path, id - FROM uploads AS u - WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; - - CREATE INDEX ON uploads_with_long_filenames(row_id); - - BEGIN; - WITH updated_uploads AS ( - UPDATE uploads - SET - path = - CONCAT( - COALESCE((regexp_match(updatable_uploads.path, '(.*\/).*'))[1], ''), - CONCAT( - LEFT(SPLIT_PART((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), - COALESCE(SUBSTRING((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) - ) - ) - FROM - uploads_with_long_filenames AS updatable_uploads - WHERE - uploads.id = updatable_uploads.id - AND updatable_uploads.row_id > 0 AND updatable_uploads.row_id <= 10000 - RETURNING uploads.* - ) - SELECT id, path FROM updated_uploads; - ROLLBACK; - ``` - - Once you validate the batch update results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. - -1. Validate that the new filenames from the previous query are the expected ones. If you are sure you want to truncate the records found in the previous step to 246 characters, run the following: - - WARNING: - The following action is **irreversible**. - - ```sql - CREATE TEMP TABLE uploads_with_long_filenames AS - SELECT ROW_NUMBER() OVER(ORDER BY id) row_id, path, id - FROM uploads AS u - WHERE LENGTH((regexp_match(u.path, '[^\\/:*?"<>|\r\n]+$'))[1]) > 246; - - CREATE INDEX ON uploads_with_long_filenames(row_id); - - UPDATE uploads - SET - path = - CONCAT( - COALESCE((regexp_match(updatable_uploads.path, '(.*\/).*'))[1], ''), - CONCAT( - LEFT(SPLIT_PART((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1], '.', 1), 242), - COALESCE(SUBSTRING((regexp_match(updatable_uploads.path, '[^\\/:*?"<>|\r\n]+$'))[1] FROM '\.(?:.(?!\.))+$')) - ) - ) - FROM - uploads_with_long_filenames AS updatable_uploads - WHERE - uploads.id = updatable_uploads.id - AND updatable_uploads.row_id > 0 AND updatable_uploads.row_id <= 10000; - ``` - - Once you finish the batch update, you must change the batch size (`updatable_uploads.row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. - -Truncate the filenames in the references found: - -1. Check if those records are referenced somewhere. One way to do this is to dump the database and search for the parent directory name and filename: - - 1. To dump your database, you can use the following command as an example: - - ```shell - pg_dump -h /var/opt/gitlab/postgresql/ -d gitlabhq_production > gitlab-dump.tmp - ``` - - 1. Then you can search for the references using the `grep` command. Combining the parent directory and the filename can be a good idea. For example: - - ```shell - grep public/alongfilenamehere.txt gitlab-dump.tmp - ``` - -1. Replace those long filenames using the new filenames obtained from querying the `uploads` table. - -Truncate the filenames on the file system. You must manually rename the files in your file system to the new filenames obtained from querying the `uploads` table. - -#### Re-run the backup task - -After following all the previous steps, re-run the backup task. - -### Restoring database backup fails when `pg_stat_statements` was previously enabled - -The GitLab backup of the PostgreSQL database includes all SQL statements required to enable extensions that were -previously enabled in the database. - -The `pg_stat_statements` extension can only be enabled or disabled by a PostgreSQL user with `superuser` role. -As the restore process uses a database user with limited permissions, it can't execute the following SQL statements: - -```sql -DROP EXTENSION IF EXISTS pg_stat_statements; -CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; -``` - -When trying to restore the backup in a PostgreSQL instance that doesn't have the `pg_stats_statements` extension, -the following error message is displayed: - -```plaintext -ERROR: permission denied to create extension "pg_stat_statements" -HINT: Must be superuser to create this extension. -ERROR: extension "pg_stat_statements" does not exist -``` - -When trying to restore in an instance that has the `pg_stats_statements` extension enabled, the cleaning up step -fails with an error message similar to the following: - -```plaintext -rake aborted! -ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' -/opt/gitlab/embedded/bin/bundle:23:in `load' -/opt/gitlab/embedded/bin/bundle:23:in `<main>' -Caused by: -PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' -/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' -/opt/gitlab/embedded/bin/bundle:23:in `load' -/opt/gitlab/embedded/bin/bundle:23:in `<main>' -Tasks: TOP => gitlab:db:drop_tables -(See full trace by running task with --trace) -``` - -#### Prevent the dump file to include `pg_stat_statements` - -To prevent the inclusion of the extension in the PostgreSQL dump file that is part of the backup bundle, -enable the extension in any schema except the `public` schema: - -```sql -CREATE SCHEMA adm; -CREATE EXTENSION pg_stat_statements SCHEMA adm; -``` - -If the extension was previously enabled in the `public` schema, move it to a new one: - -```sql -CREATE SCHEMA adm; -ALTER EXTENSION pg_stat_statements SET SCHEMA adm; -``` - -To query the `pg_stat_statements` data after changing the schema, prefix the view name with the new schema: - -```sql -SELECT * FROM adm.pg_stat_statements limit 0; -``` - -To make it compatible with third-party monitoring solutions that expect it to be enabled in the `public` schema, -you need to include it in the `search_path`: - -```sql -set search_path to public,adm; -``` - -#### Fix an existing dump file to remove references to `pg_stat_statements` - -To fix an existing backup file, do the following changes: - -1. Extract from the backup the following file: `db/database.sql.gz`. -1. Decompress the file or use an editor that is capable of handling it compressed. -1. Remove the following lines, or similar ones: - - ```sql - CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; - ``` - - ```sql - COMMENT ON EXTENSION pg_stat_statements IS 'track planning and execution statistics of all SQL statements executed'; - ``` - -1. Save the changes and recompress the file. -1. Update the backup file with the modified `db/database.sql.gz`. +<!-- This redirect file can be deleted after <2023-09-26>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/generate_sample_prometheus_data.md b/doc/raketasks/generate_sample_prometheus_data.md index 73c49bd2599..77c7d2c2c1a 100644 --- a/doc/raketasks/generate_sample_prometheus_data.md +++ b/doc/raketasks/generate_sample_prometheus_data.md @@ -2,31 +2,11 @@ stage: Monitor group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +remove_date: '2023-08-22' +redirect_to: 'index.md' --- -# Sample Prometheus data Rake task **(FREE SELF)** +# Sample Prometheus data Rake task (removed) **(FREE SELF)** -The Rake task runs Prometheus queries for each of the metrics of a specific environment -for a series of time intervals to now: - -- 30 minutes -- 3 hours -- 8 hours -- 24 hours -- 72 hours -- 7 days - -The results of each query are stored under a `sample_metrics` directory as a YAML -file named by the metric's `identifier`. When the environmental variable `USE_SAMPLE_METRICS` -is set, the Prometheus API query is re-routed to `Projects::Environments::SampleMetricsController`, -which loads the appropriate data set if it's present in the `sample_metrics` directory. - -The Rake task requires an ID from an environment with an available Prometheus installation. - -## Example - -The following example demonstrates how to run the Rake task: - -```shell -bundle exec rake gitlab:generate_sample_prometheus_data[21] -``` +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/346541) in GitLab 14.7 +and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/399231) in 16.0. diff --git a/doc/raketasks/index.md b/doc/raketasks/index.md index 47fa7e855a1..e998c97a4d7 100644 --- a/doc/raketasks/index.md +++ b/doc/raketasks/index.md @@ -20,7 +20,7 @@ The following Rake tasks are available for use with GitLab: | Tasks | Description | |:------------------------------------------------------|:------------| -| [Back up and restore](backup_restore.md) | Back up, restore, and migrate GitLab instances between servers. | +| [Back up and restore](../administration/backup_restore/index.md) | Back up, restore, and migrate GitLab instances between servers. | | [Clean up](cleanup.md) | Clean up unneeded items from GitLab instances. | | [Development](../development/rake_tasks.md) | Tasks for GitLab contributors. | | [Elasticsearch](../integration/advanced_search/elasticsearch.md#gitlab-advanced-search-rake-tasks) | Maintain Elasticsearch in a GitLab instance. | @@ -32,10 +32,8 @@ The following Rake tasks are available for use with GitLab: | [Integrity checks](../administration/raketasks/check.md) | Check the integrity of repositories, files, LDAP, and more. | | [LDAP maintenance](../administration/raketasks/ldap.md) | [LDAP](../administration/auth/ldap/index.md)-related tasks. | | [List repositories](list_repos.md) | List all GitLab-managed Git repositories on disk. | -| [Migrate snippets to Git](migrate_snippets.md) | Migrate GitLab Snippets to Git repositories, and show the migration status. | | [Praefect Rake tasks](../administration/raketasks/praefect.md) | [Praefect](../administration/gitaly/praefect.md)-related tasks. | | [Project import/export](../administration/raketasks/project_import_export.md) | Prepare for [project exports and imports](../user/project/settings/import_export.md). | -| [Sample Prometheus data](generate_sample_prometheus_data.md) | Generate sample Prometheus data. | | [Sidekiq job migration](../administration/sidekiq/sidekiq_job_migration.md) | Migrate Sidekiq jobs scheduled for future dates to a new queue. | | [Service Desk email](../administration/raketasks/service_desk_email.md) | Service Desk email-related tasks. | | [SMTP maintenance](../administration/raketasks/smtp.md) | SMTP-related tasks. | diff --git a/doc/raketasks/migrate_snippets.md b/doc/raketasks/migrate_snippets.md index 16f78d7fc71..c845d9bed73 100644 --- a/doc/raketasks/migrate_snippets.md +++ b/doc/raketasks/migrate_snippets.md @@ -1,101 +1,11 @@ --- -stage: Create -group: IDE -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2023-10-13' --- -# Migrate to versioned snippets Rake tasks **(FREE SELF)** +This document was moved to [another location](index.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215861) in GitLab 13.0. - -In GitLab 13.0, [GitLab Snippets are backed by Git repositories](../user/snippets.md#versioned-snippets). -Snippet content is stored in the repository, and users can update it directly through Git. - -Nevertheless, existing GitLab Snippets must be migrated to this new feature. -For each snippet: - -- A new repository is created. -- A file is created in the repository, using the snippet filename. -- The snippet is committed to the repository. - -GitLab performs this migration through a [Background Migration](../development/database/background_migrations.md) -when the GitLab instance is upgraded to 13.0 or a higher version. -However, if the migration fails for any of the snippets, they must be migrated individually. -The following Rake tasks help with that process. - -## Migrate specific snippets to Git - -In case you want to migrate a range of snippets, run the tasks as described below. - -For Omnibus installations, run: - -```shell -sudo gitlab-rake gitlab:snippets:migrate SNIPPET_IDS=1,2,3,4 -``` - -For installations from source code, run: - -```shell -bundle exec rake gitlab:snippets:migrate SNIPPET_IDS=1,2,3,4 -``` - -There is a default limit (100) to the number of ids supported in the migration -process. You can modify this limit by using the environment variable `LIMIT`. - -```shell -sudo gitlab-rake gitlab:snippets:migrate SNIPPET_IDS=1,2,3,4 LIMIT=50 -``` - -For installations from source code, run: - -```shell -bundle exec rake gitlab:snippets:migrate SNIPPET_IDS=1,2,3,4 LIMIT=50 -``` - -## Show whether the snippet background migration is running - -In case you want to check the status of the snippet background migration, -whether it is running or not, you can use the following task. - -For Omnibus installations, run: - -```shell -sudo gitlab-rake gitlab:snippets:migration_status -``` - -For installations from source code, run: - -```shell -bundle exec rake gitlab:snippets:migration_status RAILS_ENV=production -``` - -## List non-migrated snippets - -With the following task, you can get the ids of all of the snippets -that haven't been migrated yet or failed to migrate. - -For Omnibus installations, run: - -```shell -sudo gitlab-rake gitlab:snippets:list_non_migrated -``` - -For installations from source code, run: - -```shell -bundle exec rake gitlab:snippets:list_non_migrated RAILS_ENV=production -``` - -As the number of non-migrated snippets can be large, we limit -by default the size of the number of ids returned to 100. You can -modify this limit by using the environment variable `LIMIT`. - -```shell -sudo gitlab-rake gitlab:snippets:list_non_migrated LIMIT=200 -``` - -For installations from source code, run: - -```shell -bundle exec rake gitlab:snippets:list_non_migrated RAILS_ENV=production LIMIT=200 -``` +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/restore_gitlab.md b/doc/raketasks/restore_gitlab.md index bbb2f2aa648..434e256de35 100644 --- a/doc/raketasks/restore_gitlab.md +++ b/doc/raketasks/restore_gitlab.md @@ -2,404 +2,13 @@ stage: Systems group: Geo info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../administration/backup_restore/restore_gitlab.md' +remove_date: '2023-09-26' --- -# Restore GitLab **(FREE SELF)** +This document was moved to [another location](../administration/backup_restore/restore_gitlab.md). -GitLab provides a command line interface to restore your entire installation, -and is flexible enough to fit your needs. - -The [restore prerequisites section](#restore-prerequisites) includes crucial -information. Be sure to read and test the complete restore process at least -once before attempting to perform it in a production environment. - -You can restore a backup only to _the exact same version and type (CE/EE)_ of -GitLab that you created it on (for example CE 15.1.4). - -If your backup is a different version than the current installation, you must -[downgrade](../update/package/downgrade.md) or [upgrade](../update/package/index.md#upgrade-to-a-specific-version-using-the-official-repositories) your GitLab installation -before restoring the backup. - -Each backup archive contains a full self-contained backup, including those created through the [incremental repository backup procedure](backup_gitlab.md#incremental-repository-backups). To restore an incremental repository backup, use the same instructions as restoring any other regular backup archive. - -## Restore prerequisites - -You need to have a working GitLab installation before you can perform a -restore. This is because the system user performing the restore actions (`git`) -is usually not allowed to create or delete the SQL database needed to import -data into (`gitlabhq_production`). All existing data is either erased -(SQL) or moved to a separate directory (such as repositories and uploads). -Restoring SQL data skips views owned by PostgreSQL extensions. - -To restore a backup, **you must also restore the GitLab secrets**. -These include the database encryption key, [CI/CD variables](../ci/variables/index.md), and -variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). -Without the keys, [multiple issues occur](backup_restore.md#when-the-secrets-file-is-lost), -including loss of access by users with [two-factor authentication enabled](../user/profile/account/two_factor_authentication.md), -and GitLab Runners cannot log in. - -Restore: - -- `/etc/gitlab/gitlab-secrets.json` (Linux package) -- `/home/git/gitlab/.secret` (self-compiled installations) -- Rails secret (cloud-native GitLab) - - [This can be converted to the Linux package format](https://docs.gitlab.com/charts/installation/migration/helm_to_package.html), if required. - -You may also want to restore your previous `/etc/gitlab/gitlab.rb` (for Omnibus packages) -or `/home/git/gitlab/config/gitlab.yml` (for installations from source) and -any TLS keys, certificates (`/etc/gitlab/ssl`, `/etc/gitlab/trusted-certs`), or -[SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). - -Depending on your case, you might want to run the restore command with one or -more of the following options: - -- `BACKUP=timestamp_of_backup`: Required if more than one backup exists. - Read what the [backup timestamp is about](backup_restore.md#backup-timestamp). -- `force=yes`: Doesn't ask if the `authorized_keys` file should get regenerated, - and assumes 'yes' for warning about database tables being removed, - enabling the `Write to authorized_keys file` setting, and updating LDAP - providers. - -If you're restoring into directories that are mount points, you must ensure these directories are -empty before attempting a restore. Otherwise, GitLab attempts to move these directories before -restoring the new data, which causes an error. - -Read more about [configuring NFS mounts](../administration/nfs.md) - -Restoring a backup from an instance using local storage restores to local storage even if the target instance uses object storage. -Migrations to object storage must be done before or after restoration. - -## Restore for Omnibus GitLab installations - -This procedure assumes that: - -- You have installed the **exact same version and type (CE/EE)** of GitLab - Omnibus with which the backup was created. -- You have run `sudo gitlab-ctl reconfigure` at least once. -- GitLab is running. If not, start it using `sudo gitlab-ctl start`. - -First ensure your backup tar file is in the backup directory described in the -`gitlab.rb` configuration `gitlab_rails['backup_path']`. The default is -`/var/opt/gitlab/backups`. The backup file needs to be owned by the `git` user. - -```shell -sudo cp 11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar /var/opt/gitlab/backups/ -sudo chown git:git /var/opt/gitlab/backups/11493107454_2018_04_25_10.6.4-ce_gitlab_backup.tar -``` - -Stop the processes that are connected to the database. Leave the rest of GitLab -running: - -```shell -sudo gitlab-ctl stop puma -sudo gitlab-ctl stop sidekiq -# Verify -sudo gitlab-ctl status -``` - -Next, ensure you have completed the [restore prerequisites](#restore-prerequisites) steps and have run `gitlab-ctl reconfigure` -after copying over the GitLab secrets file from the original installation. - -Next, restore the backup, specifying the timestamp of the backup you wish to -restore: - -```shell -# This command will overwrite the contents of your GitLab database! -# NOTE: "_gitlab_backup.tar" is omitted from the name -sudo gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce -``` - -Users of GitLab 12.1 and earlier should use the command `gitlab-rake gitlab:backup:restore` instead. -Some [known non-blocking error messages may appear](backup_restore.md#restoring-database-backup-using-omnibus-packages-outputs-warnings). - -WARNING: -`gitlab-rake gitlab:backup:restore` doesn't set the correct file system -permissions on your Registry directory. This is a [known issue](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/62759). -In GitLab 12.2 or later, you can use `gitlab-backup restore` to avoid this -issue. - -If there's a GitLab version mismatch between your backup tar file and the -installed version of GitLab, the restore command aborts with an error -message. Install the [correct GitLab version](https://packages.gitlab.com/gitlab/), -and then try again. - -WARNING: -The restore command requires [additional parameters](backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) when -your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. - -Next, restart and [check](../administration/raketasks/maintenance.md#check-gitlab-configuration) GitLab: - -```shell -sudo gitlab-ctl restart -sudo gitlab-rake gitlab:check SANITIZE=true -``` - -In GitLab 13.1 and later, check [database values can be decrypted](../administration/raketasks/check.md#verify-database-values-can-be-decrypted-using-the-current-secrets) -especially if `/etc/gitlab/gitlab-secrets.json` was restored, or if a different server is -the target for the restore. - -```shell -sudo gitlab-rake gitlab:doctor:secrets -``` - -For added assurance, you can perform [an integrity check on the uploaded files](../administration/raketasks/check.md#uploaded-files-integrity): - -```shell -sudo gitlab-rake gitlab:artifacts:check -sudo gitlab-rake gitlab:lfs:check -sudo gitlab-rake gitlab:uploads:check -``` - -## Restore for Docker image and GitLab Helm chart installations - -For GitLab installations using the Docker image or the GitLab Helm chart on a -Kubernetes cluster, the restore task expects the restore directories to be -empty. However, with Docker and Kubernetes volume mounts, some system level -directories may be created at the volume roots, such as the `lost+found` -directory found in Linux operating systems. These directories are usually owned -by `root`, which can cause access permission errors since the restore Rake task -runs as the `git` user. To restore a GitLab installation, users have to confirm -the restore target directories are empty. - -For both these installation types, the backup tarball has to be available in -the backup location (default location is `/var/opt/gitlab/backups`). - -### Restore for Helm chart installations - -The GitLab Helm chart uses the process documented in -[restoring a GitLab Helm chart installation](https://docs.gitlab.com/charts/backup-restore/restore.html#restoring-a-gitlab-installation) - -### Restore for Docker image installations - -If you're using [Docker Swarm](../install/docker.md#install-gitlab-using-docker-swarm-mode), -the container might restart during the restore process because Puma is shut down, -and so the container health check fails. To work around this problem, -temporarily disable the health check mechanism. - -1. Edit `docker-compose.yml`: - - ```yaml - healthcheck: - disable: true - ``` - -1. Deploy the stack: - - ```shell - docker stack deploy --compose-file docker-compose.yml mystack - ``` - -For more information, see [issue 6846](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6846 "GitLab restore can fail owing to `gitlab-healthcheck`"). - -The restore task can be run from the host: - -```shell -# Stop the processes that are connected to the database -docker exec -it <name of container> gitlab-ctl stop puma -docker exec -it <name of container> gitlab-ctl stop sidekiq - -# Verify that the processes are all down before continuing -docker exec -it <name of container> gitlab-ctl status - -# Run the restore. NOTE: "_gitlab_backup.tar" is omitted from the name -docker exec -it <name of container> gitlab-backup restore BACKUP=11493107454_2018_04_25_10.6.4-ce - -# Restart the GitLab container -docker restart <name of container> - -# Check GitLab -docker exec -it <name of container> gitlab-rake gitlab:check SANITIZE=true -``` - -## Restore for installation from source - -First, ensure your backup tar file is in the backup directory described in the -`gitlab.yml` configuration: - -```yaml -## Backup settings -backup: - path: "tmp/backups" # Relative paths are relative to Rails.root (default: tmp/backups/) -``` - -The default is `/home/git/gitlab/tmp/backups`, and it needs to be owned by the `git` user. Now, you can begin the backup procedure: - -```shell -# Stop processes that are connected to the database -sudo service gitlab stop - -sudo -u git -H bundle exec rake gitlab:backup:restore RAILS_ENV=production -``` - -Example output: - -```plaintext -Unpacking backup... [DONE] -Restoring database tables: --- create_table("events", {:force=>true}) - -> 0.2231s -[...] -- Loading fixture events...[DONE] -- Loading fixture issues...[DONE] -- Loading fixture keys...[SKIPPING] -- Loading fixture merge_requests...[DONE] -- Loading fixture milestones...[DONE] -- Loading fixture namespaces...[DONE] -- Loading fixture notes...[DONE] -- Loading fixture projects...[DONE] -- Loading fixture protected_branches...[SKIPPING] -- Loading fixture schema_migrations...[DONE] -- Loading fixture services...[SKIPPING] -- Loading fixture snippets...[SKIPPING] -- Loading fixture taggings...[SKIPPING] -- Loading fixture tags...[SKIPPING] -- Loading fixture users...[DONE] -- Loading fixture users_projects...[DONE] -- Loading fixture web_hooks...[SKIPPING] -- Loading fixture wikis...[SKIPPING] -Restoring repositories: -- Restoring repository abcd... [DONE] -- Object pool 1 ... -Deleting tmp directories...[DONE] -``` - -Next, restore `/home/git/gitlab/.secret` if necessary, [as previously mentioned](#restore-prerequisites). - -Restart GitLab: - -```shell -sudo service gitlab restart -``` - -## Restoring only one or a few projects or groups from a backup - -Although the Rake task used to restore a GitLab instance doesn't support -restoring a single project or group, you can use a workaround by restoring -your backup to a separate, temporary GitLab instance, and then export your -project or group from there: - -1. [Install a new GitLab](../install/index.md) instance at the same version as - the backed-up instance from which you want to restore. -1. [Restore the backup](#restore-gitlab) into this new instance, then - export your [project](../user/project/settings/import_export.md) - or [group](../user/group/import/index.md#migrate-groups-by-uploading-an-export-file-deprecated). For more information about what is and isn't exported, see the export feature's documentation. -1. After the export is complete, go to the old instance and then import it. -1. After importing the projects or groups that you wanted is complete, you may - delete the new, temporary GitLab instance. - -A feature request to provide direct restore of individual projects or groups -is being discussed in [issue #17517](https://gitlab.com/gitlab-org/gitlab/-/issues/17517). - -## Restore options - -The command line tool GitLab provides to restore from backup can accept more -options. - -### Disabling prompts during restore - -During a restore from backup, the restore script may ask for confirmation before -proceeding. If you wish to disable these prompts, you can set the `GITLAB_ASSUME_YES` -environment variable to `1`. - -For Omnibus GitLab packages: - -```shell -sudo GITLAB_ASSUME_YES=1 gitlab-backup restore -``` - -For installations from source: - -```shell -sudo -u git -H GITLAB_ASSUME_YES=1 bundle exec rake gitlab:backup:restore RAILS_ENV=production -``` - -### Excluding tasks on restore - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/19347) in GitLab 14.10. - -You can exclude specific tasks on restore by adding the environment variable `SKIP`, whose values are a comma-separated list of the following options: - -- `db` (database) -- `uploads` (attachments) -- `builds` (CI job output logs) -- `artifacts` (CI job artifacts) -- `lfs` (LFS objects) -- `terraform_state` (Terraform states) -- `registry` (Container Registry images) -- `pages` (Pages content) -- `repositories` (Git repositories data) -- `packages` (Packages) - -For Omnibus GitLab packages: - -```shell -sudo gitlab-backup restore BACKUP=timestamp_of_backup SKIP=db,uploads -``` - -For installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup SKIP=db,uploads RAILS_ENV=production -``` - -### Restore specific repository storages - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86896) in GitLab 15.0. - -When using [multiple repository storages](../administration/repository_storage_paths.md), -repositories from specific repository storages can be restored separately -using the `REPOSITORIES_STORAGES` option. The option accepts a comma-separated list of -storage names. - -For example, for Omnibus GitLab installations: - -```shell -sudo gitlab-backup restore BACKUP=timestamp_of_backup REPOSITORIES_STORAGES=storage1,storage2 -``` - -For example, for installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup REPOSITORIES_STORAGES=storage1,storage2 -``` - -### Restore specific repositories - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88094) in GitLab 15.1. - -You can restore specific repositories using the `REPOSITORIES_PATHS` and the `SKIP_REPOSITORIES_PATHS` options. -Both options accept a comma-separated list of project and group paths. If you -specify a group path, all repositories in all projects in the group and -descendent groups are included or skipped, depending on which option you used. The project and group repositories must exist within the specified backup. - -For example, to restore all repositories for all projects in **Group A** (`group-a`), the repository for **Project C** in **Group B** (`group-b/project-c`), -and skip the **Project D** in **Group A** (`group-a/project-d`): - -- Omnibus GitLab installations: - - ```shell - sudo gitlab-backup restore BACKUP=timestamp_of_backup REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d - ``` - -- Installations from source: - - ```shell - sudo -u git -H bundle exec rake gitlab:backup:restore BACKUP=timestamp_of_backup REPOSITORIES_PATHS=group-a,group-b/project-c SKIP_REPOSITORIES_PATHS=group-a/project-d - ``` - -### Restore untarred backups - -If an [untarred backup](backup_gitlab.md#skipping-tar-creation) (made with `SKIP=tar`) is found, -and no backup is chosen with `BACKUP=<timestamp>`, the untarred backup is used. - -For example, for Omnibus GitLab installations: - -```shell -sudo gitlab-backup restore -``` - -For example, for installations from source: - -```shell -sudo -u git -H bundle exec rake gitlab:backup:restore -``` +<!-- This redirect file can be deleted after <2023-09-26>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index bb3f1f404b4..fa7d0813001 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # User management Rake tasks **(FREE SELF)** GitLab provides Rake tasks for managing users. Administrators can also use the Admin Area to -[manage users](../user/admin_area/index.md#administering-users). +[manage users](../administration/admin_area.md#administering-users). ## Add user as a developer to all projects diff --git a/doc/security/email_verification.md b/doc/security/email_verification.md index b87fd28dbed..4d7dd1ab36f 100644 --- a/doc/security/email_verification.md +++ b/doc/security/email_verification.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86352) in GitLab 15.2 [with a flag](../administration/feature_flags.md) named `require_email_verification`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `require_email_verification`. On GitLab.com, this feature is not available. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `require_email_verification`. On GitLab.com, this feature is not available. Account email verification provides an additional layer of GitLab account security. When certain conditions are met, an account is locked. If your account is locked, diff --git a/doc/security/hardening_application_recommendations.md b/doc/security/hardening_application_recommendations.md index 9d4d2d562fd..e9c09abdea1 100644 --- a/doc/security/hardening_application_recommendations.md +++ b/doc/security/hardening_application_recommendations.md @@ -121,7 +121,7 @@ select the **Disabled feed token** checkbox. If all of your users are coming from specific IP addresses, use **Global-allowed IP ranges** to specifically allow only those addresses. -For more details on **Visibility and access control**, see [visibility and access controls](../user/admin_area/settings/visibility_and_access_controls.md). +For more details on **Visibility and access control**, see [visibility and access controls](../administration/settings/visibility_and_access_controls.md). For information on SSH settings, see [SSH keys restrictions](../security/ssh_keys_restrictions.md). @@ -134,7 +134,7 @@ restricted. Account avatars can be manually uploaded by users. The settings in this section are intended to help enforce a custom implementation of your own specific standards on your users. As the various scenarios are too many and too varied, you should review the -[account and limit settings documentation](../user/admin_area/settings/account_and_limit_settings.md) +[account and limit settings documentation](../administration/settings/account_and_limit_settings.md) and apply changes to enforce your own policies. ### Sign-up restrictions @@ -158,7 +158,7 @@ email addresses, then list that domain in **Allowed domains for sign-ups**. This prevents those with email addresses in other domains from signing up. For more detailed information, see -[sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md). +[sign-up restrictions](../administration/settings/sign_up_restrictions.md). ### Sign-in restrictions @@ -176,7 +176,7 @@ In **Email notification for unknown sign-ins**, ensure that **Enable email notif is selected. This sends an email to users when a sign-in occurs from an unrecognized location. For more detailed information, see -[sign-in restrictions](../user/admin_area/settings/sign_in_restrictions.md). +[sign-in restrictions](../administration/settings/sign_in_restrictions.md). ## Integrations diff --git a/doc/security/hardening_operating_system_recommendations.md b/doc/security/hardening_operating_system_recommendations.md index 8b4b706815c..33f88d43d22 100644 --- a/doc/security/hardening_operating_system_recommendations.md +++ b/doc/security/hardening_operating_system_recommendations.md @@ -116,7 +116,7 @@ vm.mmap_min_addr=4096 # Default is 0, randomize virtual address space in memory, makes vuln exploitation # harder kernel.randomize_va_space=2 -# Restrict kernel pointer access (e.g. cat /proc/kallsyms) for exploit assistance +# Restrict kernel pointer access (for example, cat /proc/kallsyms) for exploit assistance kernel.kptr_restrict=2 # Restrict verbose kernel errors in dmesg kernel.dmesg_restrict=1 diff --git a/doc/security/img/ssh_keys_restrictions_settings.png b/doc/security/img/ssh_keys_restrictions_settings.png Binary files differindex 94258af3bf9..1f37a25d19f 100644 --- a/doc/security/img/ssh_keys_restrictions_settings.png +++ b/doc/security/img/ssh_keys_restrictions_settings.png diff --git a/doc/security/index.md b/doc/security/index.md index a62d7171112..1b486ab5feb 100644 --- a/doc/security/index.md +++ b/doc/security/index.md @@ -27,6 +27,6 @@ type: index - [Project Import decompressed archive size limits](project_import_decompressed_archive_size_limits.md) - [Responding to security incidents](responding_to_security_incidents.md) -To harden your GitLab instance and minimize the risk of unwanted user account creation, consider access control features like [Sign up restrictions](../user/admin_area/settings/sign_up_restrictions.md) and [Authentication options](../topics/authentication/index.md). For more detailed information, refer to [Hardening](hardening.md). +To harden your GitLab instance and minimize the risk of unwanted user account creation, consider access control features like [Sign up restrictions](../administration/settings/sign_up_restrictions.md) and [Authentication options](../topics/authentication/index.md). For more detailed information, refer to [Hardening](hardening.md). Self-managed GitLab customers and administrators are responsible for the security of their underlying hosts, and for keeping GitLab itself up to date. It is important to [regularly patch GitLab](../policy/maintenance.md), patch your operating system and its software, and harden your hosts in accordance with vendor guidance. diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md index d835d8eb08c..5353c11d2f1 100644 --- a/doc/security/rate_limits.md +++ b/doc/security/rate_limits.md @@ -31,20 +31,21 @@ similarly mitigated by a rate limit. You can set these rate limits in the Admin Area of your instance: -- [Import/Export rate limits](../user/admin_area/settings/import_export_rate_limits.md) -- [Issue rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md) -- [Note rate limits](../user/admin_area/settings/rate_limit_on_notes_creation.md) -- [Protected paths](../user/admin_area/settings/protected_paths.md) -- [Raw endpoints rate limits](../user/admin_area/settings/rate_limits_on_raw_endpoints.md) -- [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) -- [Package registry rate limits](../user/admin_area/settings/package_registry_rate_limits.md) -- [Git LFS rate limits](../user/admin_area/settings/git_lfs_rate_limits.md) -- [Files API rate limits](../user/admin_area/settings/files_api_rate_limits.md) -- [Deprecated API rate limits](../user/admin_area/settings/deprecated_api_rate_limits.md) +- [Import/Export rate limits](../administration/settings/import_export_rate_limits.md) +- [Issue rate limits](../administration/settings/rate_limit_on_issues_creation.md) +- [Note rate limits](../administration/settings/rate_limit_on_notes_creation.md) +- [Protected paths](../administration/settings/protected_paths.md) +- [Raw endpoints rate limits](../administration/settings/rate_limits_on_raw_endpoints.md) +- [User and IP rate limits](../administration/settings/user_and_ip_rate_limits.md) +- [Package registry rate limits](../administration/settings/package_registry_rate_limits.md) +- [Git LFS rate limits](../administration/settings/git_lfs_rate_limits.md) +- [Rate limits on Git SSH operations](../administration/settings/rate_limits_on_git_ssh_operations.md) +- [Files API rate limits](../administration/settings/files_api_rate_limits.md) +- [Deprecated API rate limits](../administration/settings/deprecated_api_rate_limits.md) - [GitLab Pages rate limits](../administration/pages/index.md#rate-limits) -- [Pipeline rate limits](../user/admin_area/settings/rate_limit_on_pipelines_creation.md) -- [Incident management rate limits](../user/admin_area/settings/incident_management_rate_limits.md) -- [Unauthenticated access to Projects List API rate limits](../user/admin_area/settings/rate_limit_on_projects_api.md) +- [Pipeline rate limits](../administration/settings/rate_limit_on_pipelines_creation.md) +- [Incident management rate limits](../administration/settings/incident_management_rate_limits.md) +- [Unauthenticated access to Projects List API rate limits](../administration/settings/rate_limit_on_projects_api.md) You can set these rate limits using the Rails console: @@ -73,24 +74,6 @@ For configuration information, see ## Non-configurable limits -### Git operations using SSH - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78373) in GitLab 14.7 [with a flag](../administration/feature_flags.md) named `rate_limit_gitlab_shell`. Available by default without a feature flag from 15.8. - -GitLab applies rate limits to Git operations that use SSH by user account and project. When the rate limit is exceeded, GitLab rejects -further connection requests from that user for the project. - -The rate limit applies at the Git command ([plumbing](https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain)) level. -Each command has a rate limit of 600 per minute. For example: - -- `git push` has a rate limit of 600 per minute. -- `git pull` has its own rate limit of 600 per minute. - -Because the same commands are shared by `git-upload-pack`, `git pull`, and `git clone`, they share a rate limit. - -The requests per minute threshold for this rate limit is not configurable. Self-managed customers can disable this -rate limit. - ### Repository archives > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25750) in GitLab 12.9. diff --git a/doc/security/responding_to_security_incidents.md b/doc/security/responding_to_security_incidents.md index 5c00c53c5bf..0cd7170d35b 100644 --- a/doc/security/responding_to_security_incidents.md +++ b/doc/security/responding_to_security_incidents.md @@ -14,7 +14,7 @@ additional steps. These suggestions are intended to supplement existing security If you suspect that a user account or bot account has been compromised, consider taking the following steps: -- [Block the user](../user/admin_area/moderate_users.md#block-a-user) to mitigate any current risk. +- [Block the user](../administration/moderate_users.md#block-a-user) to mitigate any current risk. - [Review the audit events](../administration/audit_events.md) available to you to identify any suspicious account behavior. For example: - Suspicious sign-in events. @@ -44,8 +44,8 @@ hosts in accordance with vendor guidance. If you suspect that your GitLab instance has been compromised, consider taking the following steps: - [Review the audit events](../administration/audit_events.md) available to you for suspicious account behavior. -- [Review all users](../user/admin_area/moderate_users.md) (including the Administrative root user), and follow the steps in [Suspected compromised user account](#suspected-compromised-user-account) if necessary. -- Review the [Credentials Inventory](../user/admin_area/credentials_inventory.md), if available to you. +- [Review all users](../administration/moderate_users.md) (including the Administrative root user), and follow the steps in [Suspected compromised user account](#suspected-compromised-user-account) if necessary. +- Review the [Credentials Inventory](../administration/credentials_inventory.md), if available to you. - Change any sensitive credentials, variables, tokens, and secrets. For example, those located in instance configuration, database, CI/CD pipelines, or elsewhere. - Upgrade to the latest version of GitLab and adopt a plan to upgrade after every security patch release. diff --git a/doc/security/token_overview.md b/doc/security/token_overview.md index 5a772106562..fb4fb71356a 100644 --- a/doc/security/token_overview.md +++ b/doc/security/token_overview.md @@ -139,7 +139,10 @@ Each user has a long-lived feed token that does not expire. This token allows au - RSS readers to load a personalized RSS feed. - Calendar applications to load a personalized calendar. -This token is visible in those feed URLs. You cannot use this token to access any other data. +You cannot use this token to access any other data. + +The user-scoped feed token can be used for all feeds, however feed and calendar URLs are generated +with a different token that is only valid for one feed. Anyone who has your token can read activity and issue RSS feeds or your calendar feed as if they were you, including confidential issues. If that happens, [reset the token](../user/profile/contributions_calendar.md#reset-the-user-activity-feed-token). @@ -171,16 +174,25 @@ This table shows available scopes per token. Scopes can be limited further on to ## Security considerations -- Access tokens should be treated like passwords and kept secure. -- Adding access tokens to URLs is a security risk, especially when cloning or adding a remote because Git then writes the URL to its `.git/config` file in plain text. URLs are +1. Treat access tokens like passwords and keep them secure. +1. When creating a scoped token, consider using the most limited scope possible to reduce the impact of accidentally leaking the token. +1. When creating a token, consider setting a token that expires when your task is complete. For example, if performing a one-off import, set the + token to expire after a few hours or a day. This reduces the impact of a token that is accidentally leaked because it is useless when it expires. +1. If you are recording a video that might contain a sensitive secret like a personal access token (PAT), feed token, or trigger token, you must mask that secret before uploading the video to GitLab Unfiltered or any other video hosting service. As an additional defense-in-depth security measure, you must revoke those secrets before you share the video publicly. For more information, see [revoking a PAT](../user/profile/personal_access_tokens.md#revoke-a-personal-access-token). +1. Adding access tokens to URLs is a security risk, especially when cloning or adding a remote because Git then writes the URL to its `.git/config` file in plain text. URLs are also generally logged by proxies and application servers, which makes those credentials visible to system administrators. Instead, pass API calls an access token using headers like [the `Private-Token` header](../api/rest/index.md#personalprojectgroup-access-tokens). -- Tokens can also be stored using a [Git credential storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). -- Tokens must not be committed to your source code. Instead, consider an approach such as [using external secrets in CI](../ci/secrets/index.md). -- When creating a scoped token, consider using the most limited scope possible to reduce the impact of accidentally leaking the token. -- When creating a token, consider setting a token that expires when your task is complete. For example, if performing a one-off import, set the - token to expire after a few hours or a day. This reduces the impact of a token that is accidentally leaked because it is useless when it expires. -- Be careful not to include tokens when pasting code, console commands, or log outputs into an issue or MR description or comment. -- Don’t log credentials in the console logs. Consider [protecting](../ci/variables/index.md#protect-a-cicd-variable) and +1. You can also store token using a [Git credential storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). +1. Do not: + - Store tokens in plain text in your projects. + - Include tokens when pasting code, console commands, or log outputs into an issue, MR description, or comment. + Consider an approach such as [using external secrets in CI](../ci/secrets/index.md). +1. Do not log credentials in the console logs or artifacts. Consider [protecting](../ci/variables/index.md#protect-a-cicd-variable) and [masking](../ci/variables/index.md#mask-a-cicd-variable) your credentials. -- Review all currently active access tokens of all types on a regular basis and revoke any that are no longer needed. +1. If you have set up a demo environment to showcase a project you have been working on and you are recording a video or writing a blog post describing that project, make sure you are not leaking sensitive secrets during that process. If you are done with the demo, you must revoke all the secrets created during that demo. +1. Review all active access tokens of all types on a regular basis and revoke any that are no longer needed. This includes: + - Personal, project, and group access tokens. + - Feed tokens. + - Trigger tokens. + - Runner registration tokens. + - Any other sensitive secrets etc. diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index 25937993c16..5dd8da72e83 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -140,7 +140,7 @@ sudo -u git -H bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_EN > - Push notification support [introduced](https://gitlab.com/gitlab-org/gitlab-shell/-/issues/506) in GitLab 15.3. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. The feature is not ready for production use. This feature flag also affects [session duration for Git Operations when 2FA is enabled](../user/admin_area/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. The feature is not ready for production use. This feature flag also affects [session duration for Git Operations when 2FA is enabled](../administration/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). You can enforce 2FA for [Git over SSH operations](../development/gitlab_shell/features.md#git-operations). However, you should use [ED25519_SK](../user/ssh.md#ed25519_sk-ssh-keys) or [ECDSA_SK](../user/ssh.md#ecdsa_sk-ssh-keys) SSH keys instead. 2FA is enforced for Git operations only, and internal commands such as [`personal_access_token`](../development/gitlab_shell/features.md#personal-access-token) are excluded. @@ -165,7 +165,7 @@ SSH key. 2FA does not protect users with compromised *private* SSH keys. Once an OTP is verified, anyone can run Git over SSH with that private SSH key for -the configured [session duration](../user/admin_area/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). +the configured [session duration](../administration/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). <!-- ## Troubleshooting diff --git a/doc/security/unlock_user.md b/doc/security/unlock_user.md index e8215616e1b..5e21cad8f3e 100644 --- a/doc/security/unlock_user.md +++ b/doc/security/unlock_user.md @@ -12,7 +12,7 @@ type: howto Users are locked after ten failed sign-in attempts. These users remain locked: - For 10 minutes, after which time they are automatically unlocked. -- Until an administrator unlocks them from the [Admin Area](../user/admin_area/index.md) or the command line in under 10 minutes. +- Until an administrator unlocks them from the [Admin Area](../administration/admin_area.md) or the command line in under 10 minutes. ## GitLab.com users diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md index 13b34d28614..78c32341bf6 100644 --- a/doc/security/webhooks.md +++ b/doc/security/webhooks.md @@ -155,7 +155,7 @@ Most GitLab instances have their `public_runner_releases_url` set to `https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-runner/releases`, which can prevent you from [filtering requests](#filter-requests). -To resolve this issue, [configure GitLab to no longer fetch runner release version data from GitLab.com](../user/admin_area/settings/continuous_integration.md#disable-runner-version-management). +To resolve this issue, [configure GitLab to no longer fetch runner release version data from GitLab.com](../administration/settings/continuous_integration.md#disable-runner-version-management). ### GitLab subscription management is blocked diff --git a/doc/subscriptions/bronze_starter.md b/doc/subscriptions/bronze_starter.md index b981a63a8b7..1f84c27d27a 100644 --- a/doc/subscriptions/bronze_starter.md +++ b/doc/subscriptions/bronze_starter.md @@ -15,13 +15,13 @@ New paid features will not be released in Bronze and Starter tiers after GitLab The following features remain available to Bronze and Starter customers, even though the tiers are no longer mentioned in GitLab documentation: -- [Activate GitLab EE with a license](../user/admin_area/license.md) -- [Add a help message to the sign-in page](../user/admin_area/settings/help_page.md#add-a-help-message-to-the-sign-in-page) +- [Activate GitLab EE with a license](../administration/license.md) +- [Add a help message to the sign-in page](../administration/settings/help_page.md#add-a-help-message-to-the-sign-in-page) - [Burndown and burnup charts](../user/project/milestones/burndown_and_burnup_charts.md) in the [Milestone View](../user/project/milestones/index.md#burndown-charts), - [Code owners](../user/project/codeowners/index.md) - Description templates: - [Setting a default template for merge requests and issues](../user/project/description_templates.md#set-a-default-template-for-merge-requests-and-issues) -- [Email from GitLab](../user/admin_area/email_from_gitlab.md) +- [Email from GitLab](../administration/email_from_gitlab.md) - Groups: - [Creating group memberships via CN](../user/group/access_and_permissions.md#create-group-links-via-cn) - [Group push rules](../user/group/access_and_permissions.md#group-push-rules) @@ -84,7 +84,7 @@ the tiers are no longer mentioned in GitLab documentation: - Reference Architecture information: - [Zero downtime upgrades](../administration/reference_architectures/index.md#zero-downtime-upgrades) - Repositories: - - [Repository size limit](../user/admin_area/settings/account_and_limit_settings.md#repository-size-limit) + - [Repository size limit](../administration/settings/account_and_limit_settings.md#repository-size-limit) - Repository mirroring: - [Pull mirroring](../user/project/repository/mirror/pull.md) outside repositories in a GitLab repository - [Overwrite diverged branches](../user/project/repository/mirror/pull.md#overwrite-diverged-branches) diff --git a/doc/subscriptions/choosing_subscription.md b/doc/subscriptions/choosing_subscription.md index 07b04c83bd7..ccb68792531 100644 --- a/doc/subscriptions/choosing_subscription.md +++ b/doc/subscriptions/choosing_subscription.md @@ -30,8 +30,10 @@ A new subscription must be purchased and applied as needed. ## Choose a GitLab tier Pricing is [tier-based](https://about.gitlab.com/pricing/), allowing you to choose -the features which fit your budget. For information on what features are available -at each tier for each product, see the [GitLab self-managed feature comparison](https://about.gitlab.com/pricing/feature-comparison/). +the features that fit your budget. + +For more details, see +[a comparison of self-managed features available in each tier](https://about.gitlab.com/pricing/feature-comparison/). ## Find your subscription diff --git a/doc/subscriptions/community_programs.md b/doc/subscriptions/community_programs.md index 5e11292a084..26d221cf62d 100644 --- a/doc/subscriptions/community_programs.md +++ b/doc/subscriptions/community_programs.md @@ -10,11 +10,11 @@ GitLab provides the following community program subscriptions. ## GitLab for Education -For qualifying non-profit educational institutions, the [GitLab for Education Program](https://about.gitlab.com/solutions/education/) provides GitLab Ultimate, plus 50,000 units of compute per month. The subscription granted under GitLab for Education can only be used for instructional use or non-commercial academic research. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Education Program page](https://about.gitlab.com/solutions/education/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/education-program/). +For qualifying non-profit educational institutions, the [GitLab for Education Program](https://about.gitlab.com/solutions/education/) provides GitLab Ultimate, plus 50,000 compute minutes per month. The subscription granted under GitLab for Education can only be used for instructional use or non-commercial academic research. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Education Program page](https://about.gitlab.com/solutions/education/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/education-program/). ## GitLab for Open Source -For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 units of compute per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/opensource-program/). +For qualifying open source projects, the [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/) provides GitLab Ultimate, plus 50,000 compute minutes per month. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Open Source Program page](https://about.gitlab.com/solutions/open-source/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/opensource-program/). ### Meeting GitLab for Open Source Program requirements @@ -77,4 +77,4 @@ Exceptions to this public visibility requirement apply in select circumstances ( ## GitLab for Startups -For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 units of compute per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/startups-program/). +For qualifying startups, the [GitLab for Startups](https://about.gitlab.com/solutions/startups/) program provides GitLab Ultimate, plus 50,000 compute minutes per month for 12 months. For more information—including instructions for applying to the program and renewing program membership—see the [GitLab for Startups Program page](https://about.gitlab.com/solutions/startups/) and the [GitLab handbook](https://about.gitlab.com/handbook/marketing/community-relations/community-programs/startups-program/). diff --git a/doc/subscriptions/customers_portal.md b/doc/subscriptions/customers_portal.md index 07e6e952419..25f36432f4b 100644 --- a/doc/subscriptions/customers_portal.md +++ b/doc/subscriptions/customers_portal.md @@ -6,22 +6,39 @@ info: To determine the technical writer assigned to the Stage/Group associated w # The Customers Portal -For some management tasks for your subscription and account, you use the Customers Portal. +For some management tasks for your subscription and account, such as purchasing additional seats or storage and viewing invoices, you use the Customers Portal. See the following pages for specific instructions on managing your subscription: + +- [GitLab SaaS subscription](gitlab_com/index.md) +- [Self-managed subscription](self_managed/index.md) If you made your purchase through an authorized reseller, you must contact them directly to make changes to your subscription (your subscriptions are read-only). -You can also specifically manage your [GitLab SaaS subscription](gitlab_com/index.md) -or [self-managed subscription](self_managed/index.md). +## Sign in to Customers Portal -## Change account owner information +You can sign in to Customers Portal either with your GitLab.com account or your email and password (if you have not yet [linked your Customers Portal account to your GitLab.com account](#link-a-gitlabcom-account)). + +NOTE: +If you registered for Customers Portal with your GitLab.com account, sign in with this account. + +To sign in to Customers Portal using your GitLab.com account: -Account owner personal details are used on invoices. The account owner email address is used for the Customers Portal legacy login and license-related email. +1. Navigate to [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **Continue with your GitLab.com account**. + +To sign in to Customers Portal using your email and password: + +1. Navigate to [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Select **Sign in with your email and password**. +1. Provide the **Email** and **Password** for your Customers Portal account. +1. Select **Sign in**. + +## Change account owner information -If you have registered a Customers Portal account through a GitLab.com account, the GitLab.com account is used for login. +The account owner's personal details are used on invoices. The account owner's email address is used for the [Customers Portal legacy sign-in](#sign-in-to-customers-portal) and license-related email. To change account owner information, including name, billing address, and email address: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. Select **My account > Account details**. 1. Expand the **Personal details** section. 1. Edit the personal details. @@ -37,7 +54,7 @@ to another person, after you enter that person's personal details, you must also To change your company details, including company name and VAT number: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. Select **My account > Account details**. 1. Expand the **Company details** section. 1. Edit the company details. @@ -54,7 +71,7 @@ If you would like to use an alternative method to pay, please To change your payment method: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. Select **My account > Payment methods**. 1. **Edit** an existing payment method's information or **Add new payment method**. 1. Select **Save Changes**. @@ -64,7 +81,7 @@ To change your payment method: Automatic renewal of a subscription is charged to your default payment method. To mark a payment method as the default: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. Select **My account > Payment methods**. 1. **Edit** the selected payment method and check the **Make default payment method** checkbox. 1. Select **Save Changes**. @@ -75,10 +92,10 @@ Follow this guideline if you have a legacy Customers Portal account and use an e To link a GitLab.com account to your Customers Portal account: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in?legacy=true) using email and password. +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in?legacy=true) using email and password. 1. On the Customers Portal page, select **My account > Account details**. 1. Under **Your GitLab.com account**, select **Link account**. -1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. +1. Sign in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. ## Change the linked account @@ -88,17 +105,38 @@ If you have a legacy Customers Portal account that is not linked to a GitLab.com To change the GitLab.com account linked to your Customers Portal account: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. In a separate browser tab, go to [GitLab.com](https://gitlab.com/users/sign_in) and ensure you are not logged in. 1. On the Customers Portal page, select **My account > Account details**. 1. Under **Your GitLab.com account**, select **Change linked account**. -1. Log in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. +1. Sign in to the [GitLab.com](https://gitlab.com/users/sign_in) account you want to link to the Customers Portal account. ## Change Customers Portal account password To change the password for this customers portal account: -1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). +1. Sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in). 1. Select the **My account** dropdown list and select **Account details**. 1. Make the required changes to the **Your password** section. 1. Select **Save changes**. + +## Customers that purchased through a reseller + +If you purchased a subscription through an authorized reseller (including GCP and AWS marketplaces), you have access to the Customers Portal to: + +- View your subscription. +- Associate your subscription with the relevant group (GitLab SaaS) or download the license (GitLab self-managed). +- Manage contact information. + +Other changes and requests must be done through the reseller, including: + +- Changes to the subscription. +- Purchase of additional seats, Storage, or Compute. +- Requests for invoices, because those are issued by the reseller, not GitLab. + +Resellers do not have access to the Customers Portal, or their customers' accounts. + +After your subscription order is processed, you will receive several emails: + +- A "Welcome to the Customers Portal" email, including instructions on how to log in. +- A purchase confirmation email with instructions on how to provision access. diff --git a/doc/subscriptions/gitlab_com/index.md b/doc/subscriptions/gitlab_com/index.md index 28eff9032f0..8cd5777e4cb 100644 --- a/doc/subscriptions/gitlab_com/index.md +++ b/doc/subscriptions/gitlab_com/index.md @@ -16,7 +16,7 @@ You don't need to install anything to use GitLab SaaS, you only need to The subscription determines which features are available for your private projects. Organizations with public open source projects can actively apply to our [GitLab for Open Source Program](https://about.gitlab.com/solutions/open-source/join/). -Qualifying open source projects also get 50,000 units of compute and free access to the **Ultimate** tier +Qualifying open source projects also get 50,000 compute minutes and free access to the **Ultimate** tier through the [GitLab for Open Source program](https://about.gitlab.com/solutions/open-source/). ## Obtain a GitLab SaaS subscription @@ -103,6 +103,11 @@ To view a list of seats being used: 1. Select **Settings > Usage Quotas**. 1. On the **Seats** tab, view usage information. +For each user, a list shows groups and projects where the user is a direct member. + +- **Group invite** indicates the user is a member of a [group shared with a group](../../user/group/manage.md#share-a-group-with-another-group). +- **Project invite** indicates the user is a member of a [group shared with a project](../../user/project/members/share_project_with_groups.md#share-a-project-with-a-group). + The data in seat usage listing, **Seats in use**, and **Seats in subscription** are updated live. The counts for **Max seats used** and **Seats owed** are updated once per day. @@ -347,12 +352,13 @@ If you have difficulty during the renewal process, contact the Contacts can renew a subscription, cancel a subscription, or transfer the subscription to a different namespace. -To change the contacts: +For information about how to transfer ownership of the Customers Portal account to another person, see +[Change account owner information](../customers_portal.md#change-account-owner-information). + +To add a secondary contact for your subscription: 1. Ensure an account exists in the [Customers Portal](https://customers.gitlab.com/customers/sign_in) for the user you want to add. -1. Verify you have access to at least one of - [these requirements](https://about.gitlab.com/handbook/support/license-and-renewals/workflows/customersdot/associating_purchases.html). 1. [Create a ticket with the Support team](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293). Include any relevant material in your request. ## Compute @@ -363,10 +369,10 @@ on GitLab shared runners. Refer to [Compute usage](../../ci/pipelines/cicd_minutes.md) for more information. -### Purchase additional units of compute +### Purchase additional compute minutes -You can [purchase additional units of compute](../../ci/pipelines/cicd_minutes.md#purchase-additional-units-of-compute) -for your personal or group namespace. Units of compute are a **one-time purchase**, so they do not renew. +You can [purchase additional compute minutes](../../ci/pipelines/cicd_minutes.md#purchase-additional-compute-minutes) +for your personal or group namespace. Compute minutes are a **one-time purchase**, so they do not renew. ## Add-on subscription for additional Storage and Transfer diff --git a/doc/subscriptions/gitlab_dedicated/index.md b/doc/subscriptions/gitlab_dedicated/index.md index f14f38f4717..75fcdb70161 100644 --- a/doc/subscriptions/gitlab_dedicated/index.md +++ b/doc/subscriptions/gitlab_dedicated/index.md @@ -43,7 +43,7 @@ GitLab Dedicated supports instance-level [SAML OmniAuth](../../integration/saml. GitLab Dedicated offers public connectivity by default with support for IP allowlists. You can [optionally specify a list of IP addresses](../../administration/dedicated/index.md#ip-allowlist) that can access your GitLab Dedicated instance. Subsequently, when an IP not on the allowlist tries to access your instance the connection is refused. -Private connectivity via [AWS PrivateLink](https://aws.amazon.com/privatelink/) is also offered as an option. Both [inbound](../../administration/dedicated/index.md#inbound-private-link) and [outbound](../../administration/dedicated/index.md#outbound-private-link) PrivateLinks are supported. When connecting to an internal service running in your VPC over HTTPS via PrivateLink, GitLab Dedicated supports the ability to use a private SSL certificate, which can be provided when [updating your instance configuration](../../administration/dedicated/index.md#custom-certificates). +Private connectivity via [AWS PrivateLink](https://aws.amazon.com/privatelink/) is also offered as an option. Both [inbound](../../administration/dedicated/index.md#inbound-private-link) and [outbound](../../administration/dedicated/index.md#outbound-private-link) PrivateLinks are supported. When connecting to internal resources over an outbound PrivateLink with non public certificates, you can specify a list of certificates that are trusted by GitLab. These certificates can be provided when [updating your instance configuration](../../administration/dedicated/index.md#custom-certificates). #### Encryption @@ -92,7 +92,7 @@ GitLab may conduct unscheduled maintenance to address high-severity issues affec ### Application -GitLab Dedicated comes with the self-managed [Ultimate feature set](https://about.gitlab.com/pricing/feature-comparison/) with the exception of the unsupported features [listed below](#features-that-are-not-available). +GitLab Dedicated comes with the self-managed [Ultimate feature set](https://about.gitlab.com/pricing/feature-comparison/) with the exception of the [unsupported features](#features-that-are-not-available) listed below. #### GitLab Runners diff --git a/doc/subscriptions/self_managed/index.md b/doc/subscriptions/self_managed/index.md index d6e21c3683b..e754c2a06a2 100644 --- a/doc/subscriptions/self_managed/index.md +++ b/doc/subscriptions/self_managed/index.md @@ -15,12 +15,12 @@ To subscribe to GitLab for a GitLab self-managed installation: 1. Go to the [Customers Portal](https://customers.gitlab.com/) and purchase a GitLab self-managed plan. 1. After purchase, an activation code is sent to the email address associated with the Customers Portal account. - You must [add this code to your GitLab instance](../../user/admin_area/license.md). + You must [add this code to your GitLab instance](../../administration/license.md). NOTE: If you're purchasing a subscription for an existing **Free** GitLab self-managed instance, ensure you're purchasing enough seats to -[cover your users](../../user/admin_area/index.md#administering-users). +[cover your users](../../administration/admin_area.md#administering-users). ### Subscription seats @@ -47,10 +47,10 @@ The lists of users are displayed. A _billable user_ counts against the number of subscription seats. Every user is considered a billable user, with the following exceptions: -- [Deactivated users](../../user/admin_area/moderate_users.md#deactivate-a-user) and - [blocked users](../../user/admin_area/moderate_users.md#block-a-user) don't count as billable users in the current subscription. When they are either deactivated or blocked they release a _billable user_ seat. However, they may +- [Deactivated users](../../administration/moderate_users.md#deactivate-a-user) and + [blocked users](../../administration/moderate_users.md#block-a-user) don't count as billable users in the current subscription. When they are either deactivated or blocked they release a _billable user_ seat. However, they may count toward overages in the subscribed seat count. -- Users who are [pending approval](../../user/admin_area/moderate_users.md#users-pending-approval). +- Users who are [pending approval](../../administration/moderate_users.md#users-pending-approval). - Users with only the [Minimal Access role](../../user/permissions.md#users-with-minimal-access) on self-managed Ultimate subscriptions or any GitLab.com subscriptions. - Users with only the [Guest or Minimal Access roles on an Ultimate subscription](#free-guest-users). - Users without project or group memberships on an Ultimate subscription. @@ -101,7 +101,7 @@ The user must not be assigned any other role, anywhere in the instance. NOTE: If a user creates a project, they are assigned the Maintainer or Owner role. To prevent a user from creating projects, as an administrator, you can mark the user -as [external](../../user/admin_area/external_users.md). +as [external](../../administration/external_users.md). ## Tips for managing users and subscription seats @@ -114,14 +114,14 @@ Managing the number of users against the number of subscription seats can be a c GitLab has several features which can help you manage the number of users: -- Enable the [**Require administrator approval for new sign ups**](../../user/admin_area/settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) +- Enable the [**Require administrator approval for new sign ups**](../../administration/settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) option. - Enable `block_auto_created_users` for new sign-ups via [LDAP](../../administration/auth/ldap/index.md#basic-configuration-settings) or [OmniAuth](../../integration/omniauth.md#configure-common-settings). -- Enable the [User cap](../../user/admin_area/settings/sign_up_restrictions.md#user-cap) +- Enable the [User cap](../../administration/settings/sign_up_restrictions.md#user-cap) option. **Available in GitLab 13.7 and later**. -- [Disable new sign-ups](../../user/admin_area/settings/sign_up_restrictions.md), and instead manage new +- [Disable new sign-ups](../../administration/settings/sign_up_restrictions.md), and instead manage new users manually. -- View a breakdown of users by role in the [Users statistics](../../user/admin_area/index.md#users-statistics) page. +- View a breakdown of users by role in the [Users statistics](../../administration/admin_area.md#users-statistics) page. ## Subscription data synchronization @@ -132,7 +132,7 @@ To enable subscription data synchronization you must have: - GitLab Enterprise Edition (EE), version 14.1 or later. - Connection to the internet, and must not have an offline environment. -- [Activated](../../user/admin_area/license.md) your instance with an activation code. +- [Activated](../../administration/license.md) your instance with an activation code. When your instance is activated, and data is synchronized, the following processes are automated: @@ -345,7 +345,7 @@ A payment receipt is emailed to you, which you can also access in the Customers If your subscription was activated with an activation code, the additional seats are reflected in your instance immediately. If you're using a license file, you receive an updated file. -To add the seats, [add the license file](../../user/admin_area/license_file.md) +To add the seats, [add the license file](../../administration/license_file.md) to your instance. ### Renew subscription manually @@ -354,7 +354,7 @@ Starting 30 days before a subscription expires, a banner with the expiry date di You should follow these steps during renewal: -1. Prior to the renewal date, prune any inactive or unwanted users by [blocking them](../../user/admin_area/moderate_users.md#block-a-user). +1. Prior to the renewal date, prune any inactive or unwanted users by [blocking them](../../administration/moderate_users.md#block-a-user). 1. Determine if you have a need for user growth in the upcoming subscription. 1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in) and select the **Renew** button beneath your existing subscription. The **Renew** button remains disabled (grayed-out) until 15 days before a subscription expires. @@ -367,7 +367,7 @@ You can hover your mouse on the **Renew** button to see the date when it will be 1. Enter the number of [users over subscription](#users-over-subscription) in the second box for the user overage incurred in your previous subscription term. 1. Review your renewal details and complete the payment process. 1. An activation code for the renewal term is available on the [Manage Purchases](https://customers.gitlab.com/subscriptions) page on the relevant subscription card. Select **Copy activation code** to get a copy. -1. [Add the activation code](../../user/admin_area/license.md) to your instance. +1. [Add the activation code](../../administration/license.md) to your instance. An invoice is generated for the renewal and available for viewing or download on the [View invoices](https://customers.gitlab.com/receipts) page. If you have difficulty during the renewal process, contact our [support team](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293) for assistance. @@ -421,19 +421,20 @@ The following is emailed to you: [**View invoices**](https://customers.gitlab.com/receipts). - A new activation code for your license. -[Add the activation code](../../user/admin_area/license.md) to your instance. +[Add the activation code](../../administration/license.md) to your instance. The new tier takes effect when the new license is activated. ## Add or change the contacts for your subscription Contacts can renew a subscription, cancel a subscription, or transfer the subscription to a different namespace. -To change the contacts: +For information about how to transfer ownership of the Customers Portal account to another person, see +[Change account owner information](../customers_portal.md#change-account-owner-information). + +To add a secondary contact for your subscription: 1. Ensure an account exists in the [Customers Portal](https://customers.gitlab.com/customers/sign_in) for the user you want to add. -1. Verify you have access to at least one of - [these requirements](https://about.gitlab.com/handbook/support/license-and-renewals/workflows/customersdot/associating_purchases.html). 1. [Create a ticket with the Support team](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293). Include any relevant material in your request. ## Subscription expiry @@ -450,7 +451,7 @@ before this occurs. ## Activate a license file or key -If you have a license file or key, you can activate it [in the Admin Area](../../user/admin_area/license_file.md#activate-gitlab-ee-with-a-license-file-or-key). +If you have a license file or key, you can activate it [in the Admin Area](../../administration/license_file.md#activate-gitlab-ee-with-a-license-file-or-key). ## Contact Support diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md index 6348e3a5fa0..b85b99d1874 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_eks.md @@ -200,7 +200,7 @@ to monitor it. After successfully deploying your application, you can view its website and check on its health on the **Environments** page by navigating to -**Deployments > Environments**. This page displays details about +**Operate > Environments**. This page displays details about the deployed applications, and the right-hand column displays icons that link you to common environment tasks: @@ -297,4 +297,3 @@ and customized to fit your workflow. Here are some helpful resources for further 1. [Incremental rollout to production](../cicd_variables.md#incremental-rollout-to-production) 1. [Disable jobs you don't need with CI/CD variables](../cicd_variables.md) 1. [Use your own buildpacks to build your application](../customize.md#custom-buildpacks) -1. [Prometheus monitoring](../../../user/project/integrations/prometheus.md) diff --git a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md index dac85926ff5..f6a6c16e010 100644 --- a/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md +++ b/doc/topics/autodevops/cloud_deployments/auto_devops_with_gke.md @@ -204,7 +204,7 @@ to monitor it. After successfully deploying your application, you can view its website and check on its health on the **Environments** page by navigating to -**Deployments > Environments**. This page displays details about +**Operate > Environments**. This page displays details about the deployed applications, and the right-hand column displays icons that link you to common environment tasks: @@ -302,4 +302,3 @@ and customized to fit your workflow. Here are some helpful resources for further 1. [Incremental rollout to production](../cicd_variables.md#incremental-rollout-to-production) 1. [Disable jobs you don't need with CI/CD variables](../cicd_variables.md) 1. [Use your own buildpacks to build your application](../customize.md#custom-buildpacks) -1. [Prometheus monitoring](../../../user/project/integrations/prometheus.md) diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 10979f0bb21..d5ebd4e9fcb 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -174,7 +174,7 @@ To enable Auto DevOps for your instance: When enabled, Auto DevOps attempts to run pipelines in every project. If the pipeline fails in a particular project, it disables itself. -GitLab administrators can change this in the [Auto DevOps settings](../../user/admin_area/settings/continuous_integration.md#auto-devops). +GitLab administrators can change this in the [Auto DevOps settings](../../administration/settings/continuous_integration.md#auto-devops). If a [CI/CD configuration file](../../ci/yaml/index.md) is present, it remains unchanged and Auto DevOps does not affect it. diff --git a/doc/topics/autodevops/multiple_clusters_auto_devops.md b/doc/topics/autodevops/multiple_clusters_auto_devops.md index 0b1699d8230..1b6b5bc33c0 100644 --- a/doc/topics/autodevops/multiple_clusters_auto_devops.md +++ b/doc/topics/autodevops/multiple_clusters_auto_devops.md @@ -32,7 +32,7 @@ To deploy your environments to different Kubernetes clusters: For deprecated, [certificate-based clusters](../../user/infrastructure/clusters/index.md#certificate-based-kubernetes-integration-deprecated): -1. Go to the project and select **Infrastructure > Kubernetes clusters** from the left sidebar. +1. Go to the project and select **Operate > Kubernetes clusters** from the left sidebar. 1. [Set the environment scope of each cluster](../../user/project/clusters/multiple_kubernetes_clusters.md#setting-the-environment-scope). 1. For each cluster, [add a domain based on its Ingress IP address](../../user/project/clusters/gitlab_managed_clusters.md#base-domain). diff --git a/doc/topics/autodevops/prepare_deployment.md b/doc/topics/autodevops/prepare_deployment.md index b71beee708e..e69c0ca0909 100644 --- a/doc/topics/autodevops/prepare_deployment.md +++ b/doc/topics/autodevops/prepare_deployment.md @@ -54,17 +54,17 @@ Auto DevOps requires a wildcard DNS `A` record matching the base domains. For a base domain of `example.com`, you'd need a DNS entry like: ```plaintext -*.example.com 3600 A 1.2.3.4 +*.example.com 3600 A 10.0.2.2 ``` -In this case, the deployed applications are served from `example.com`, and `1.2.3.4` +In this case, the deployed applications are served from `example.com`, and `10.0.2.2` is the IP address of your load balancer, generally NGINX ([see requirements](requirements.md)). Setting up the DNS record is beyond the scope of this document; check with your DNS provider for information. Alternatively, you can use free public services like [nip.io](https://nip.io) which provide automatic wildcard DNS without any configuration. For [nip.io](https://nip.io), -set the Auto DevOps base domain to `1.2.3.4.nip.io`. +set the Auto DevOps base domain to `10.0.2.2.nip.io`. After completing setup, all requests hit the load balancer, which routes requests to the Kubernetes pods running your application. diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md index dd73a2056e5..bfb88cf0dc4 100644 --- a/doc/topics/autodevops/requirements.md +++ b/doc/topics/autodevops/requirements.md @@ -71,17 +71,17 @@ Auto DevOps requires a wildcard DNS `A` record that matches the base domains. Fo a base domain of `example.com`, you'd need a DNS entry like: ```plaintext -*.example.com 3600 A 1.2.3.4 +*.example.com 3600 A 10.0.2.2 ``` -In this case, the deployed applications are served from `example.com`, and `1.2.3.4` +In this case, the deployed applications are served from `example.com`, and `10.0.2.2` is the IP address of your load balancer, generally NGINX ([see requirements](requirements.md)). Setting up the DNS record is beyond the scope of this document; check with your DNS provider for information. Alternatively, you can use free public services like [nip.io](https://nip.io) which provide automatic wildcard DNS without any configuration. For [nip.io](https://nip.io), -set the Auto DevOps base domain to `1.2.3.4.nip.io`. +set the Auto DevOps base domain to `10.0.2.2.nip.io`. After completing setup, all requests hit the load balancer, which routes requests to the Kubernetes pods running your application. diff --git a/doc/topics/build_your_application.md b/doc/topics/build_your_application.md index 340cb7a1db8..48e81e26b02 100644 --- a/doc/topics/build_your_application.md +++ b/doc/topics/build_your_application.md @@ -4,14 +4,10 @@ group: unassigned info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Build your application **(FREE)** +# Use CI/CD to build your application **(FREE)** Add your source code to a repository, create merge requests to check in code, and use CI/CD to generate your application. Include packages in your app and output it to a variety of environments. -- [Repositories](../user/project/repository/index.md) -- [Merge requests](../user/project/merge_requests/index.md) - [CI/CD](../ci/index.md) - [Runners](https://docs.gitlab.com/runner/) -- [GitLab Pages](../user/project/pages/index.md) -- [Packages and registries](../user/packages/index.md) diff --git a/doc/topics/cron/index.md b/doc/topics/cron/index.md index b437541e0ea..ca1bc7f40f2 100644 --- a/doc/topics/cron/index.md +++ b/doc/topics/cron/index.md @@ -42,6 +42,7 @@ are valid: - Run once a month on the 2nd Monday: `0 0 * * 1#2` - Run once a year at midnight of 1 January: `0 0 1 1 *` - Run every other Sunday at 0900 hours: `0 9 * * sun%2` + - This syntax is from the [fugit modulo extension](https://github.com/floraison/fugit#the-modulo-extension) For complete cron documentation, refer to the [crontab(5) — Linux manual page](https://man7.org/linux/man-pages/man5/crontab.5.html). diff --git a/doc/topics/git/git_rebase.md b/doc/topics/git/git_rebase.md index bc9337481d4..dd2260b04dc 100644 --- a/doc/topics/git/git_rebase.md +++ b/doc/topics/git/git_rebase.md @@ -44,10 +44,9 @@ branch, such as `release-15-3`. You can also specify a different remote reposito To back up a branch before taking any destructive action, like a rebase or force push: 1. Open your feature branch in the terminal: `git checkout my-feature` -1. Check out a new branch from it: `git checkout -b my-feature-backup` +1. Create a backup branch: `git branch my-feature-backup` Any changes added to `my-feature` after this point are lost if you restore from the backup branch. -1. Change back to your original branch: `git checkout my-feature` Your branch is backed up, and you can try a rebase or a force push. If anything goes wrong, restore your branch from its backup: diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 5f3458821dd..97e729aee55 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -33,7 +33,7 @@ Prerequisites: To do this: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > CI/CD**. +1. Select **Settings > General**. 1. Expand the **Visibility, project features, permissions** section. 1. Turn on the **Git Large File Storage (LFS)** toggle. 1. Select **Save changes**. diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index 22548be2e8b..e4c106e3e33 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -59,6 +59,38 @@ git revert <commit-sha> git commit --amend ``` +### Create a new message for older commits + +WARNING: +Changing commit history can disrupt others' work if they have cloned, forked, or have active branches. +Only amend pushed commits if you're sure it's safe. +To learn more, see [Git rebase and force push](git_rebase.md). + +```shell +git rebase -i HEAD~n +``` + +Replace `n` with the number of commits you want to go back. + +This opens your text editor with a list of commits. +In the editor, replace `pick` with `reword` for each commit you want to change the message: + +```shell +reword 1fc6c95 original commit message +pick 6b2481b another commit message +pick 5c1291b another commit message +``` + +After saving and closing the file, you can update each message in a new editor window. + +After updating your commits, you must push them to the repository. +As this rewrites history, a force push is required. +To prevent unintentional overwrites, use `--force-with-lease`: + +```shell +git push --force-with-lease +``` + ### Add a file to the last commit ```shell @@ -213,4 +245,4 @@ questions that you know someone might ask. Each scenario can be a third-level heading, for example `### Getting error message X`. If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/topics/manage_code.md b/doc/topics/manage_code.md new file mode 100644 index 00000000000..136e471d14a --- /dev/null +++ b/doc/topics/manage_code.md @@ -0,0 +1,12 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Manage your code **(FREE)** + +Store your source files in a repository, and make changes to them by using merge requests. + +- [Repositories](../user/project/repository/index.md) +- [Merge requests](../user/project/merge_requests/index.md) diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md index e51e015dddf..dd739fdaf77 100644 --- a/doc/topics/offline/quick_start_guide.md +++ b/doc/topics/offline/quick_start_guide.md @@ -147,7 +147,7 @@ Updating CA certificates... Runtime platform arch=amd64 os=linux pid=7 revision=1b659122 version=12.8.0 Running in system-mode. -Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): +Please enter the gitlab-ci coordinator URL (for example, https://gitlab.com/): https://my-host.internal Please enter the gitlab-ci token for this runner: XXXXXXXXXXX @@ -158,7 +158,7 @@ Please enter the gitlab-ci tags for this runner (comma separated): Registering runner... succeeded runner=FSMwkvLZ Please enter the executor: custom, docker, virtualbox, kubernetes, docker+machine, docker-ssh+machine, docker-ssh, parallels, shell, ssh: docker -Please enter the default Docker image (e.g. ruby:2.6): +Please enter the default Docker image (for example, ruby:2.6): ruby:2.6 Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! ``` @@ -204,7 +204,7 @@ Version Check and Service Ping improve the GitLab user experience and ensure tha users are on the most up-to-date instances of GitLab. These two services can be turned off for offline environments so that they do not attempt and fail to reach out to GitLab services. -For more information, see [Enable or disable usage statistics](../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics). +For more information, see [Enable or disable usage statistics](../../administration/settings/usage_statistics.md#enable-or-disable-usage-statistics). ### Configure NTP @@ -262,7 +262,7 @@ PKG_METADATA_MANIFEST_OUTPUT_FILE="/tmp/license_db_export_manifest.json" PKG_METADATA_DOWNLOADS_OUTPUT_FILE="/tmp/license_db_object_links.tsv" # Download the contents of the bucket -curl --silent --show-error --request GET "https://storage.googleapis.com/storage/v1/b/prod-export-license-bucket-1a6c642fc4de57d4/o" > "$PKG_METADATA_MANIFEST_OUTPUT_FILE" +curl --silent --show-error --request GET "https://storage.googleapis.com/storage/v1/b/prod-export-license-bucket-1a6c642fc4de57d4/o?maxResults=7500" > "$PKG_METADATA_MANIFEST_OUTPUT_FILE" # Parse the links and names for the bucket objects and output them into a tsv file jq -r '.items[] | [.name, .mediaLink] | @tsv' "$PKG_METADATA_MANIFEST_OUTPUT_FILE" > "$PKG_METADATA_DOWNLOADS_OUTPUT_FILE" diff --git a/doc/topics/release_your_application.md b/doc/topics/release_your_application.md index e8dba553cab..3cc5e9a66b3 100644 --- a/doc/topics/release_your_application.md +++ b/doc/topics/release_your_application.md @@ -15,8 +15,10 @@ release features incrementally. - [Environments and deployments](../ci/environments/index.md) - [Releases](../user/project/releases/index.md) +- [Packages and registries](../user/packages/index.md) - [Review Apps](../ci/review_apps/index.md) - [Feature flags](../operations/feature_flags.md) +- [GitLab Pages](../user/project/pages/index.md) ## Related topics diff --git a/doc/tutorials/configure_gitlab_runner_to_use_gke/index.md b/doc/tutorials/configure_gitlab_runner_to_use_gke/index.md index e10c5d37ba0..05340994edf 100644 --- a/doc/tutorials/configure_gitlab_runner_to_use_gke/index.md +++ b/doc/tutorials/configure_gitlab_runner_to_use_gke/index.md @@ -26,7 +26,7 @@ To configure GitLab Runner to use the GKE: Before you can configure GitLab Runner to use the GKE you must: - Have a project where you have the Maintainer or Owner role. If you don't have a project, you can [create it](../../user/project/index.md). -- [Obtain the project runner registration token](../../ci/runners/register_runner.md#generate-a-registration-token-deprecated). +- [Obtain the project runner registration token](../../ci/runners/runners_scope.md#create-a-project-runner-with-a-registration-token-deprecated). - Install GitLab Runner. ## Set up your environment diff --git a/doc/tutorials/dependency_scanning.md b/doc/tutorials/dependency_scanning.md index 51424c3319e..90bc2ec96a2 100644 --- a/doc/tutorials/dependency_scanning.md +++ b/doc/tutorials/dependency_scanning.md @@ -104,7 +104,7 @@ scanned for vulnerabilities. Use the content shown in the [Yarn lockfile](#yarn-lock-file-content) section. -1. Go to **CI/CD > Pipelines** and confirm that the latest pipeline completed successfully. +1. Go to **Build > Pipelines** and confirm that the latest pipeline completed successfully. In the pipeline, dependency scanning runs and the vulnerabilities are detected automatically. @@ -116,7 +116,7 @@ medium severity vulnerabilities and confirm only the high severity vulnerability To triage the vulnerabilities: -1. Go to **Security and Compliance > Vulnerability report**. +1. Go to **Secure > Vulnerability report**. 1. Select each of the medium severity vulnerabilities by selecting the checkbox in each row. 1. From the **Set status** dropdown list select **Dismiss**. From the **Dismissal reason** dropdown list select **Used in tests**, add the comment "Used in tests", then select **Change status**. @@ -161,12 +161,12 @@ To fix the vulnerability: ``` 1. Switch to the GitLab browser tab. -1. Go to **Merge requests**, then select **Create merge request**. +1. Go to **Code > Merge requests**, then select **Create merge request**. 1. On the **New merge request** page, scroll to the bottom and select **Create merge request**. Wait for the merge request pipeline to complete. 1. Refresh the page, then select **Merge**. 1. Wait for the pipeline to complete successfully. -1. Go to **Security and Compliance > Vulnerability report**. +1. Go to **Secure > Vulnerability report**. 1. Select the **High** vulnerability's description. A banner confirms that the vulnerability has been resolved in the `main` branch. You would @@ -174,7 +174,7 @@ To fix the vulnerability: `yarn.lock` file. For this tutorial, you can skip the verification step. 1. In the **Status** dropdown list, select **Resolve**, then select **Change status**. -1. Go to **Security and Compliance > Vulnerability report**. +1. Go to **Secure > Vulnerability report**. You should now see _no_ vulnerabilities listed in the vulnerability report. @@ -224,7 +224,7 @@ To add a new vulnerability: ``` 1. Switch to the GitLab browser tab. -1. Go to **Merge requests**, then select **Create merge request**. +1. Go to **Code > Merge requests**, then select **Create merge request**. 1. On the **New merge request** page, scroll to the bottom and select **Create merge request**. Wait for the merge request pipeline to complete, then refresh the page. The merge diff --git a/doc/tutorials/fuzz_testing/index.md b/doc/tutorials/fuzz_testing/index.md index 41b5068ab72..f51f7cdb427 100644 --- a/doc/tutorials/fuzz_testing/index.md +++ b/doc/tutorials/fuzz_testing/index.md @@ -165,7 +165,7 @@ fuzz test using the pipeline you've just created. To run the fuzz test: -1. On the left sidebar, select **Merge requests**. +1. On the left sidebar, select **Code > Merge requests**. 1. Select **New merge request**. 1. In the **Source branch** section, select the `add-fuzz-test` branch. 1. In the **Target branch** section, make sure that your namespace and the `main` branch are selected. diff --git a/doc/tutorials/hugo/index.md b/doc/tutorials/hugo/index.md index 0e66d096fe2..97c79e77392 100644 --- a/doc/tutorials/hugo/index.md +++ b/doc/tutorials/hugo/index.md @@ -88,7 +88,7 @@ To build a Hugo site with GitLab, you first need to create a `.gitlab-ci.yml` fi You specify your configuration options in a special file called `.gitlab-ci.yml`. To create a `.gitlab-ci.yml` file: -1. On the left sidebar, select **Repository > Files**. +1. On the left sidebar, select **Code > Repository**. 1. Above the file list, select the plus icon ( + ), then select **New file** from the dropdown list. 1. For the filename, enter `.gitlab-ci.yml`. Don't omit the period at the beginning. 1. Select the **Apply a template** dropdown list, then enter "Hugo" in the filter box. diff --git a/doc/tutorials/left_sidebar/index.md b/doc/tutorials/left_sidebar/index.md index f26849eac45..80fbbf2032e 100644 --- a/doc/tutorials/left_sidebar/index.md +++ b/doc/tutorials/left_sidebar/index.md @@ -63,7 +63,7 @@ To start, we will find the project we want to work on.  -## Customize the sidebar +## Pin frequently used items You can pin menu items if you tend to use them frequently. @@ -76,6 +76,9 @@ The item is displayed in the **Pinned** section:  +NOTE: +The items you pin while you're viewing a project are different than the items you pin while viewing a group. + ## Use a more focused view On the left sidebar, you can also choose a more focused view into the areas you have access to. diff --git a/doc/tutorials/make_first_git_commit/index.md b/doc/tutorials/make_first_git_commit/index.md index b78fcd237dd..8099fc65f14 100644 --- a/doc/tutorials/make_first_git_commit/index.md +++ b/doc/tutorials/make_first_git_commit/index.md @@ -6,7 +6,7 @@ info: For assistance with this tutorial, see https://about.gitlab.com/handbook/p # Tutorial: Make your first Git commit -This tutorial is going to teach you a little bit about how Git works. It walks +This tutorial will teach you a little bit about how Git works. It walks you through the steps of creating your own project, editing a file, and committing changes to a Git repository from the command line. diff --git a/doc/tutorials/manage_user/index.md b/doc/tutorials/manage_user/index.md index b21454fa5d5..91309d7cb42 100644 --- a/doc/tutorials/manage_user/index.md +++ b/doc/tutorials/manage_user/index.md @@ -389,7 +389,7 @@ add different users to that project. You have now created a project in the parent group. -In this project, go to **Project information > Members**. +In this project, go to **Manage > Members**. The existing members of the parent group (you and Alex) are already members of this project because when your project belongs to a group, project members inherit diff --git a/doc/tutorials/move_personal_project_to_group/index.md b/doc/tutorials/move_personal_project_to_group/index.md index 65d7e8d07e5..29849752f5f 100644 --- a/doc/tutorials/move_personal_project_to_group/index.md +++ b/doc/tutorials/move_personal_project_to_group/index.md @@ -40,7 +40,7 @@ Maintainer role for the group. If you don't have a group, create one: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. 1. In **Group name**, enter a name for the group. 1. In **Group URL**, enter a path for the group, which is used as the namespace. 1. Choose the [visibility level](../../user/public_access.md). diff --git a/doc/tutorials/protected_workflow/img/approval_rules_v16_2.png b/doc/tutorials/protected_workflow/img/approval_rules_v16_2.png Binary files differnew file mode 100644 index 00000000000..a7db9488bad --- /dev/null +++ b/doc/tutorials/protected_workflow/img/approval_rules_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/branch_is_protected_v16_2.png b/doc/tutorials/protected_workflow/img/branch_is_protected_v16_2.png Binary files differnew file mode 100644 index 00000000000..c4ea048a41e --- /dev/null +++ b/doc/tutorials/protected_workflow/img/branch_is_protected_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/branch_list_v16_1.png b/doc/tutorials/protected_workflow/img/branch_list_v16_1.png Binary files differnew file mode 100644 index 00000000000..d557c4a9350 --- /dev/null +++ b/doc/tutorials/protected_workflow/img/branch_list_v16_1.png diff --git a/doc/tutorials/protected_workflow/img/new_file_v16_2.png b/doc/tutorials/protected_workflow/img/new_file_v16_2.png Binary files differnew file mode 100644 index 00000000000..d42b461f9c5 --- /dev/null +++ b/doc/tutorials/protected_workflow/img/new_file_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/new_project_v16_2.png b/doc/tutorials/protected_workflow/img/new_project_v16_2.png Binary files differnew file mode 100644 index 00000000000..00effb87856 --- /dev/null +++ b/doc/tutorials/protected_workflow/img/new_project_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/protections_in_place_v16_2.png b/doc/tutorials/protected_workflow/img/protections_in_place_v16_2.png Binary files differnew file mode 100644 index 00000000000..b21a714562e --- /dev/null +++ b/doc/tutorials/protected_workflow/img/protections_in_place_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/search_engineering_v16_2.png b/doc/tutorials/protected_workflow/img/search_engineering_v16_2.png Binary files differnew file mode 100644 index 00000000000..ae69b41a665 --- /dev/null +++ b/doc/tutorials/protected_workflow/img/search_engineering_v16_2.png diff --git a/doc/tutorials/protected_workflow/img/subgroup_structure_v16_1.png b/doc/tutorials/protected_workflow/img/subgroup_structure_v16_1.png Binary files differnew file mode 100644 index 00000000000..3c4ee02d0f4 --- /dev/null +++ b/doc/tutorials/protected_workflow/img/subgroup_structure_v16_1.png diff --git a/doc/tutorials/protected_workflow/index.md b/doc/tutorials/protected_workflow/index.md new file mode 100644 index 00000000000..5245bdc5ba9 --- /dev/null +++ b/doc/tutorials/protected_workflow/index.md @@ -0,0 +1,290 @@ +--- +stage: Create +group: Code Review +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +--- + +<!-- vale gitlab.FutureTense = NO --> + +# Tutorial: Build a protected workflow for your project **(FREE)** + +When your team starts a new project, they need a workflow that balances efficiency +with appropriate reviews. In GitLab, you can create user groups, combine those +groups with branch protections, and then enforce those protections with approval rules. + +This tutorial sets up protections for Excelsior Project's `1.x` and `1.x.x` +release branches, and creates a minimal approval workflow for the project: + +1. [Create the `engineering` group](#create-the-engineering-group) +1. [Create subgroups in `engineering`](#create-subgroups-in-engineering) +1. [Add users to the subgroups](#add-users-to-the-subgroups) +1. [Create the Excelsior project](#create-the-excelsior-project) +1. [Add a basic CODEOWNERS file](#add-a-basic-codeowners-file) +1. [Configure approval rules](#configure-approval-rules) +1. [Enforce CODEOWNER approval on branches](#enforce-codeowner-approval-on-branches) +1. [Create the release branches](#create-the-release-branches) + +## Prerequisites + +- You must have the Maintainer or Owner role. +- You need a list of managers and their email addresses. +- You need a list of your backend and frontend engineers, and their email addresses. +- You understand [semantic versioning](https://semver.org/) for branch names. + +## Create the `engineering` group + +Before setting up Excelsior Project, you should create a group to own +the project. Here, you'll set up the Engineering group: + +1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. +1. Select **Create group**. +1. For **Group name**, enter `Engineering`. +1. For the **Group URL**, enter `engineering`. +1. Set the **Visibility level** to **Private**. +1. Personalize your experience so GitLab shows the most helpful information to you: + - For **Role**, select **System administrator**. + - For **Who will be using this group?** select **My company or team**. + - For **What will you use this group for?** select **I want to store my code**. +1. Skip inviting members to the group. You'll add users in a later section of this tutorial. +1. Select **Create group**. + +Next, you'll add subgroups to this `engineering` group for more granular control. + +## Create subgroups in `engineering` + +The `engineering` group is a good start, but Excelsior Project's +backend engineers, frontend engineers, and managers +have different tasks, and different areas of specialty. + +Here, you'll create three more granular subgroups in the Engineering group to +segment users by the type of work they do: `managers`, `frontend`, and `backend`. +Then you'll add these new groups as members of the `engineering` group. + +First, create the new subgroup: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) + and search for `engineering`. Select the group named `Engineering`: + +  + +1. On the overview page for the `engineering` group, in the upper-right corner, + select **New subgroup**. +1. For the **Subgroup name**, enter `Managers`. +1. Set the **Visibility level** to **Private**. +1. Select **Create subgroup**. + +Next, add the subgroup as a member of the `engineering` group: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) + and search for `engineering`. Select the group named `Engineering`. +1. On the left sidebar, select **Manage > Members**. +1. On the top right, select **Invite a group**. +1. For **Select a group to invite**, select `Engineering / Managers`. +1. When adding the subgroups select the role **Maintainer**. + This configures the highest role a member of the subgroup can inherit when accessing the `engineering` group and its projects. +1. Optional. Select an expiration date. +1. Select **Invite**. + +Repeat this process to create subgroups for `backend` and `frontend`. When you're done, +search for the `engineering` group one more time. Its overview page should show +three subgroups, like this: + + + +## Add users to the subgroups + +In the previous step, when you added your subgroups to the parent group (`engineering`), you limited +members of the subgroups to the Maintainer role. This is the highest role they can inherit +for projects owned by `engineering`. As a result: + +- User 1 is added to the `manager` subgroup with the Guest role, and receives + the Guest role on `engineering` projects. +- User 2 is added to the `manager` group with the Owner role. This role is higher + than the maximum role (Maintainer) you set, so User 2 receives the Maintainer + role instead of Owner. + +To add a user to the `frontend` subgroup: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) + and search for `frontend`. Select the `Frontend` group. +1. Select **Manage > Members**. +1. Select **Invite members**. +1. Fill in the fields. Select the **Developer** role by default, increasing it + to **Maintainer** if this user reviews the work of others. +1. Select **Invite**. +1. Repeat these steps until you've added all of the frontend engineers into the + `frontend` subgroup. + +Now do the same with the `backend` and `managers` groups. The same user can be a +member of multiple subgroups. + +## Create the Excelsior project + +Now that your group structure is in place, create the `excelsior` project +for the teams to work in. Because both frontend and backend engineers +are involved, `excelsior` should belong to `engineering` instead of any of the +smaller subgroups you just created. + +To create the new `excelsior` project: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and + search for `engineering`. Select the group named `Engineering`. +1. On the overview page for the `engineering` group, on the left sidebar, at the top, + select **Create new...** (**{plus}**) and **In this group > New project/repository**. +1. Select **Create blank project**. +1. Enter the project details: + - In the **Project name** field, enter `Excelsior`. The **Project slug** should + auto-populate with `excelsior`. + - For **Visibility Level**, select **Public**. + - Select **Initialize repository with a README** to add an initial file to the repository. +1. Select **Create project**. + +GitLab creates the `excelsior` project for you, and redirects you to its homepage. +It should look like this: + + + +You'll use a feature on this page in the next step. + +## Add a basic CODEOWNERS file + +Add a CODEOWNERS file to the root directory of your project to route reviews to +the right subgroup. This example sets up four rules: + +- All changes should be reviewed by someone in the `engineering` group. +- A manager should review any change to the CODEOWNERS file itself. +- Frontend engineers should review changes to frontend files. +- Backend engineers should review changes to backend files. + +NOTE: +GitLab Free supports only optional reviews. To make reviews required, you need +GitLab Premium or Ultimate. + +To add a CODEOWNERS file to your `excelsior` project: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and + search for `Excelsior`. Select the project named `Excelsior`. +1. Next to the branch name, select the plus icon (**{plus}**), then **New file**: +  +1. For **Filename**, enter `CODEOWNERS`. This will create a file named `CODEOWNERS` + in the root directory of your project. +1. Paste this example into the editing area, changing `@engineering/` if it + does not match your group structure: + + ```plaintext + # All changes should be reviewed by someone in the engineering group + * @engineering + + # A manager should review any changes to this file + CODEOWNERS @engineering/managers + + # Frontend files should be reviewed by FE engineers + [Frontend] @engineering/frontend + *.scss + *.js + + # Backend files should be reviewed by BE engineers + [Backend] @engineering/backend + *.rb + ``` + +1. For a **Commit message**, paste in: + + ```plaintext + Adds a new CODEOWNERS file + + Creates a small CODEOWNERS file to: + - Route backend and frontend changes to the right teams + - Route CODEOWNERS file changes to managers + - Request all changes be reviewed + ``` + +1. Select **Commit changes**. + +The CODEOWNERS file is now in place in the `main` branch of your project, and +available for all future branches created in this project. + +## Configure approval rules + +The CODEOWNERS file describes the appropriate reviewers for directories and +file types. Approval rules direct merge requests to those reviewers. +Here, you will set up an approval rule that uses the information in your new CODEOWNERS +file and adds protection for release branches: + +1. On the left sidebar, select **Settings > Merge requests**. +1. In the **Merge request approvals** section, scroll to **Approval rules**. +1. Select **Add approval rule**. +1. Create a rule named `Enforce CODEOWNERS`. +1. Select **All protected branches**. +1. To make the rule required in GitLab Premium and GitLab Ultimate, + set the **Approvals required** to `1`. +1. Add the `managers` group as approvers. +1. Select **Add approval rule**. +1. Scroll to **Approval settings** and make sure + **Prevent editing approval rules in merge requests** is selected. +1. Select **Save changes**. + +When added, the `Enforce CODEOWNERS` rule looks like this: + + + +## Enforce CODEOWNER approval on branches + +You've configured several protections for your project, and you're now ready to +combine those protections together to safeguard your project's important branches: + +- Your users are sorted into logical groups and subgroups. +- Your CODEOWNERS file describes the subject matter experts for file types and directories. +- Your approval rule encourages (in GitLab Free) or requires (in GitLab Premium and GitLab Ultimate) + the subject matter experts to review changes. + +Your `excelsior` project uses [semantic versioning](https://semver.org/) for +release branch names, so you know the release branches follow the pattern `1.x` +and `1.x.x`. You want all code added to these branches to be reviewed by subject +matter experts, and for managers to make the final decision on what work is merged +into the release branch. + +Rather than create protections for a branch at a time, configure wildcard branch rules +to protect multiple branches: + +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Protected branches**. +1. From the **Branch** dropdown list, type `1.*`, and then select **Create wildcard `1.*`**. +1. To require everyone to submit merge requests, rather than pushing commits directly: + 1. Set **Allowed to merge** to **Maintainers**. + 1. Set **Allowed to push and merge** to **No one**. + 1. Leave **Allowed to force push** disabled. +1. In GitLab Premium and GitLab Ultimate, to require Code Owners to review changes + to files they work on, toggle **Require approval from code owners**. +1. Select **Protect**. +1. In the table of branches, find the rule marked as `Default`. (Depending on + your version of GitLab, this branch may be named `main` or `master`.) Set the + values for this branch to match the settings you used for the `1.*` rule. + +Your rules are now in place, even though no `1.*` branches exist yet: + + + +## Create the release branches + +Now that all branch protections in place, you're ready to create your 1.0.0 release branch: + +1. On the left sidebar, select **Code > Branches**. +1. On the top right, select **New branch**. Name it `1.0.0`. +1. Select **Create branch**. + +The branch protections are now visible in the UI: + +- On the left sidebar, select **Code > Branches**. In the list of branches, + branch `1.0.0` should be show that it is protected: + +  + +- On the left sidebar, select **Settings > Repository**, then expand **Branch rules** + to see details about all protected branches: + +  + +Congratulations! Your engineers can work independently in their feature branches, +and all code submitted for consideration for the 1.0.0 release branch will +be reviewed by subject matter experts. diff --git a/doc/update/background_migrations.md b/doc/update/background_migrations.md index bf9f2df9e87..cbdcfef3bae 100644 --- a/doc/update/background_migrations.md +++ b/doc/update/background_migrations.md @@ -4,132 +4,119 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Check for background migrations before upgrading +# Background migrations and upgrades **(FREE SELF)** -Certain releases may require different migrations to be -finished before you update to the newer version. +> - Batched background migrations [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/51332) in GitLab 13.11 [with a flag](../user/feature_flags.md) named `execute_batched_migrations_on_schedule`. Disabled by default. +> - Feature flag `execute_batched_migrations_on_schedule` [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/329511) in GitLab 13.12. +> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](../development/database/batched_background_migrations.md#enable-or-disable-background-migrations). -There are two kinds of migrations: +Certain releases may require different migrations to be finished before you +update to the newer version. Two kinds of migrations exist. They differ, and you +should check that both are complete before upgrading GitLab: -- [Background migrations](#background-migrations) -- [Batched background migrations](#batched-background-migrations) (available in GitLab 14.0 and later) +- [Batched background migrations](#batched-background-migrations), most + commonly used in GitLab 14.0 and later. +- [Background migrations](#background-migrations) that are not batched. + Most commonly used in GitLab 13.11 and earlier. -Background migrations and batched migrations are not the same, so you should check that both are -complete before updating. - -Decrease the time required to complete these migrations by increasing the number of +To decrease the time required to complete these migrations, increase the number of [Sidekiq workers](../administration/sidekiq/extra_sidekiq_processes.md) that can process jobs in the `background_migration` queue. -## Background migrations - -### Pending migrations - -**For Omnibus installations:** - -```shell -sudo gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' -sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.queued.count' -``` - -**For installations from source:** - -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' -sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.queued.count' -``` - -### Failed migrations - -**For Omnibus installations:** +## Batched background migrations -For GitLab 14.0-14.9: - -```shell -sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.failed.count' -``` - -For GitLab 14.10 and later: +To update database tables in batches, GitLab can use batched background migrations. These migrations +are created by GitLab developers and run automatically on upgrade. However, such migrations are +limited in scope to help with migrating some `integer` database columns to `bigint`. This is needed to +prevent integer overflow for some tables. -```shell -sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.with_status(:failed).count' -``` +Some installations [may need to run GitLab 14.0 for at least a day](index.md#1400) +to complete the database changes introduced by that upgrade. -**For installations from source:** +Batched background migrations are handled by Sidekiq and +[run in isolation](../development/database/batched_background_migrations.md#isolation), +so an instance can remain operational while the migrations are processed. However, +performance might degrade on larger instances that are heavily used while +batched background migrations are run. You should +[Actively monitor the Sidekiq status](../administration/admin_area.md#background-jobs) +until all migrations are completed. -For GitLab 14.0-14.9: +### Check the status of batched background migrations -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.failed.count' -``` +You can check the status of batched background migrations in the GitLab UI, or +by querying the database directly. Before you upgrade GitLab, all migrations must +have a `Finished` status. -For GitLab 14.10 and later: +If the migrations are not finished and you try to upgrade GitLab, you might +see this error: -```shell -cd /home/git/gitlab -sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.with_status(:failed).count' +```plaintext +Expected batched background migration for the given configuration to be marked +as 'finished', but it is 'active': ``` -## Batched background migrations **(FREE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/51332) in GitLab 13.11, [behind a feature flag](../user/feature_flags.md), disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/329511) in GitLab 13.12. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-background-migrations). - -There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). -Refer to this feature's version history for more details. - -To update database tables in batches, GitLab can use batched background migrations. These migrations -are created by GitLab developers and run automatically on upgrade. However, such migrations are -limited in scope to help with migrating some `integer` database columns to `bigint`. This is needed to -prevent integer overflow for some tables. +If you get this error, +[review the options](#database-migrations-failing-because-of-batched-background-migration-not-finished) for +how to complete the batched background migrations needed for the GitLab upgrade. -Some installations [may need to run GitLab 14.0 for at least a day](index.md#1400) to complete the database changes introduced by that upgrade. +#### From the GitLab UI -Batched background migrations are handled by Sidekiq and [run in isolation](../development/database/batched_background_migrations.md#isolation), so an instance can remain operational while the migrations are processed. However, there may be performance degradation on larger instances that are heavily used while batched background migrations are run, so it's a good idea to [actively monitor the Sidekiq status](../user/admin_area/index.md#background-jobs) until all migrations are completed. +Prerequisites: -### Check the status of batched background migrations +- You must have administrator access to the instance. To check the status of batched background migrations: 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). 1. Select **Admin Area**. 1. Select **Monitoring > Background Migrations**. +1. Select **Queued** or **Finalizing** to see incomplete migrations, + and **Failed** for failed migrations. + +#### From the database -  +Prerequisites: -All migrations must have a `Finished` status before you upgrade GitLab. +- You must have administrator access to the instance. -The status of batched background migrations can also be queried directly in the database. +To query the database directly for the status of batched background migrations: -1. Log into a `psql` prompt according to the directions for your instance's installation method -(for example, `sudo gitlab-psql` for Omnibus installations). -1. Run the following query in the `psql` session to see details on incomplete batched background migrations: +1. Log into a `psql` prompt, according to the directions for your instance's + installation method. For example, `sudo gitlab-psql` for Linux package installations. +1. To see details on incomplete batched background migrations, run this query in + the `psql` session: ```sql - select job_class_name, table_name, column_name, job_arguments from batched_background_migrations where status <> 3; + SELECT + job_class_name, + table_name, + column_name, + job_arguments + FROM batched_background_migrations + WHERE status <> 3; ``` -If the migrations are not finished and you try to update to a later version, -GitLab prompts you with an error: +### Enable or disable advanced features -```plaintext -Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': -``` +Batched background migrations provide feature flags that enable you to customize +migrations or pause them entirely. These feature flags should only be disabled by +advanced users who understand the risks of doing so. -If you get this error, [check the batched background migration options](#database-migrations-failing-because-of-batched-background-migration-not-finished) to complete the upgrade. +#### Pause batched background migrations in GitLab 14.x -### Pause batched background migrations in GitLab 14.x +WARNING: +There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). +Refer to each feature's version history for more details. To pause an ongoing batched background migration, -[disable the batched background migrations feature](#enable-or-disable-background-migrations). +[disable the batched background migrations feature](../development/database/batched_background_migrations.md#enable-or-disable-background-migrations). Disabling the feature completes the current batch of migrations, then waits to start the next batch until after the feature is enabled again. +Prerequisites: + +- You must have administrator access to the instance. + Use the following database queries to see the state of the current batched background migration: 1. Obtain the ID of the running migration: @@ -170,86 +157,166 @@ Use the following database queries to see the state of the current batched backg command above) to proceed with the batch when ready. On larger instances, background migrations can take as long as 48 hours to complete each batch. -### Automatic batch size optimization +#### Automatic batch size optimization -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60133) -> in GitLab 13.12, [behind a feature flag](../user/feature_flags.md), -> [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/329511). -> - Enabled on GitLab.com. -> - Recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to -> [disable it](#enable-or-disable-automatic-batch-size-optimization). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60133) in GitLab 13.2 [with a flag](../administration/feature_flags.md) named `optimize_batched_migrations`. Enabled by default. +WARNING: There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). Refer to this feature's version history for more details. +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `optimize_batched_migrations`. +On GitLab.com, this feature is available. + To maximize throughput of batched background migrations (in terms of the number of tuples updated per time unit), batch sizes are automatically adjusted based on how long the previous batches took to complete. -### Enable or disable automatic batch size optimization +#### Parallel execution -Automatic batch size optimization for batched background migrations is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) -can opt to disable it. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104027) in GitLab 15.7 [with a flag](../administration/feature_flags.md) named `batched_migrations_parallel_execution`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/372316) in GitLab 15.11. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120808) in GitLab 16.1. Feature flag `batched_migrations_parallel_execution` removed. -To enable it: +WARNING: +There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). +Refer to this feature's version history for more details. + +To speed up the execution of batched background migrations, two migrations are executed at the same time. + +[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) can change +the number of batched background migrations executed in parallel: ```ruby -Feature.enable(:optimize_batched_migrations) +ApplicationSetting.update_all(database_max_running_batched_background_migrations: 4) ``` -To disable it: +### Fix and retry failed batched background migrations -```ruby -Feature.disable(:optimize_batched_migrations) +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67504) in GitLab 14.3. + +If you [check the status](#check-the-status-of-batched-background-migrations) of batched background migrations, +some migrations might display in the **Failed** tab with a **failed** status: + + + +You must resolve all failed batched background migrations to upgrade to a newer +version of GitLab. + +To determine why the batched background migration failed, +[view the failure error logs](../development/database/batched_background_migrations.md#viewing-failure-error-logs) or: + +Prerequisites: + +- You must have administrator access to the instance. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Monitoring > Background Migrations**. +1. Select the **Failed** tab. This displays a list of failed batched background migrations. +1. Select the failed **Migration** to see the migration parameters and the jobs that failed. +1. Under **Failed jobs**, select each **ID** to see why the job failed. + +If you are a GitLab customer, consider opening a [Support Request](https://support.gitlab.com/hc/en-us/requests/new) +to debug why the batched background migrations failed. + +To correct the problem, you can retry the failed batched background migrations: + +Prerequisites: + +- You must have administrator access to the instance. + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. Select **Monitoring > Background Migrations**. +1. Select the **Failed** tab. This displays a list of failed batched background migrations. +1. Select a failed batched background migration to retry by clicking on the retry button (**{retry}**). + +To monitor the retried batched background migrations, you can +[check the status of batched background migrations](#check-the-status-of-batched-background-migrations) +on a regular interval. + +## Background migrations + +In GitLab 13, background migrations were not batched. In GitLab 14 and later, this +type of migration was superseded by batched background migrations. + +### Check for pending background migrations + +To check for pending non-batched background migrations: + +::Tabs + +:::TabTitle Linux package (Omnibus) + +```shell +sudo gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' +sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.queued.count' ``` -### Parallel execution +:::TabTitle Self-compiled (source) -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104027) -> in GitLab 15.7, [behind a feature flag](../user/feature_flags.md), -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/372316) in GitLab 15.11. -> - Feature flag `batched_migrations_parallel_execution` removed in GitLab 16.1. +```shell +cd /home/git/gitlab +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.queued.count' +``` -There can be [risks when disabling released features](../administration/feature_flags.md#risks-when-disabling-released-features). -Refer to this feature's version history for more details. +::EndTabs -To speed up the execution of batched background migrations, two migrations are executed at the same time. +### Check for failed background migrations -[GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) can change -the number of batched background migrations executed in parallel: +To check for non-batched background migrations that have failed: -```ruby -ApplicationSetting.update_all(database_max_running_batched_background_migrations: 4) +::Tabs + +:::TabTitle Linux package (Omnibus) + +For GitLab versions 14.10 and later: + +```shell +sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.with_status(:failed).count' ``` -## Troubleshooting +For GitLab versions 14.0-14.9: -### Enable or disable background migrations +```shell +sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.failed.count' +``` -In extremely limited circumstances, a GitLab administrator can disable either or -both of these [feature flags](../administration/feature_flags.md): +:::TabTitle Self-compiled (source) -- `execute_background_migrations` -- `execute_batched_migrations_on_schedule` +For GitLab versions 14.10 and later: -These flags are enabled by default. Disable them only as a last resort -to limit database operations in special circumstances, like database host maintenance. +```shell +cd /home/git/gitlab +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.with_status(:failed).count' +``` -WARNING: -Do not disable either of these flags unless you fully understand the ramifications. If you disable -the `execute_background_migrations` or `execute_batched_migrations_on_schedule` feature flag, -GitLab upgrades might fail and data loss might occur. +For GitLab versions 14.0-14.9: + +```shell +cd /home/git/gitlab +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigration::BatchedMigration.failed.count' +``` + +::EndTabs + +## Troubleshooting ### Database migrations failing because of batched background migration not finished -When updating to GitLab 14.2 or later there might be a database migration failing with a message like: +When updating to GitLab version 14.2 or later, database migrations might fail with a message like: ```plaintext StandardError: An error has occurred, all later migrations canceled: Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': - {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"push_event_payloads", :column_name=>"event_id", :job_arguments=>[["event_id"], ["event_id_convert_to_bigint"]]} + {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", + :table_name=>"push_event_payloads", + :column_name=>"event_id", + :job_arguments=>[["event_id"], + ["event_id_convert_to_bigint"]] + } ``` First, check if you have followed the [version-specific upgrade instructions for 14.2](../update/index.md#1420). @@ -263,7 +330,7 @@ version and manually ensuring that the batched migrations complete successfully. #### Roll back and follow the required upgrade path -1. [Rollback and restore the previously installed version](../raketasks/backup_restore.md) +1. [Rollback and restore the previously installed version](../administration/backup_restore/index.md) 1. Update to either 14.0.5 or 14.1 **before** updating to 14.2+ 1. [Check the status](#check-the-status-of-batched-background-migrations) of the batched background migrations and make sure they are all marked as finished before attempting to upgrade again. If any remain marked as active, @@ -321,8 +388,14 @@ arguments. For example, if you receive an error similar to this: ```plaintext StandardError: An error has occurred, all later migrations canceled: -Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': - {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"push_event_payloads", :column_name=>"event_id", :job_arguments=>[["event_id"], ["event_id_convert_to_bigint"]]} +Expected batched background migration for the given configuration to be marked as +'finished', but it is 'active': + {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", + :table_name=>"push_event_payloads", + :column_name=>"event_id", + :job_arguments=>[["event_id"], + ["event_id_convert_to_bigint"]] + } ``` Plug the arguments from the error message into the command: @@ -395,17 +468,19 @@ The following operations can disrupt your GitLab performance. They run a number Run the following check. If it returns non-zero and the count does not decrease over time, follow the rest of the steps in this section. ```shell -# For Omnibus installations: +# For Linux package installations: sudo gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' -# For installations from source: +# For self-compiled installations: cd /home/git/gitlab sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' ``` It is safe to re-execute the following commands, especially if you have 1000+ pending jobs which would likely overflow your runtime memory. -**For Omnibus installations** +::Tabs + +:::TabTitle Linux package (Omnibus) ```shell # Start the rails console @@ -417,7 +492,7 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) } ``` -**For installations from source** +:::TabTitle Self-compiled (source) ```shell # Start the rails console @@ -429,16 +504,14 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) } ``` +::EndTabs + ### Background migrations stuck in 'pending' state WARNING: The following operations can disrupt your GitLab performance. They run a number of Sidekiq jobs that perform various database or file updates. -- GitLab 13.6 introduced an issue where a background migration named - `BackfillJiraTrackerDeploymentType2` can be permanently stuck in a - **pending** state across upgrades. To clean up this stuck migration, see the - [13.6.0 version-specific instructions](index.md#1360). - GitLab 14.2 introduced an issue where a background migration named `BackfillDraftStatusOnMergeRequests` can be permanently stuck in a **pending** state across upgrades when the instance lacks records that match @@ -469,17 +542,19 @@ it returns non-zero and the count does not decrease over time, follow the rest of the steps in this section. ```shell -# For Omnibus installations: +# For Linux package installations: sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count' -# For installations from source: +# For self-compiled installations: cd /home/git/gitlab sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count' ``` It is safe to re-attempt these migrations to clear them out from a pending status: -**For Omnibus installations** +::Tabs + +:::TabTitle Linux package (Omnibus) ```shell # Start the rails console @@ -493,7 +568,7 @@ Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job| end ``` -**For installations from source** +:::TabTitle Self-compiled (source) ```shell # Start the rails console @@ -506,3 +581,5 @@ Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job| puts "Result: #{result}" end ``` + +::EndTabs diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md index ecf0f68de0e..804fcc77ea1 100644 --- a/doc/update/deprecations.md +++ b/doc/update/deprecations.md @@ -5,9 +5,9 @@ info: "See the Technical Writers assigned to Development Guidelines: https://abo toc: false --- -# Deprecations by version +# Deprecations and removals by version -These GitLab features are deprecated and no longer recommended for use. +The following GitLab features are deprecated and no longer recommended for use. Each deprecated feature will be removed in a future release. Some features cause breaking changes when they are removed. @@ -24,8 +24,10 @@ and [GraphQL](https://docs.gitlab.com/ee/api/graphql/removed_items.html) depreca <!-- DO NOT EDIT THIS PAGE DIRECTLY -This page is automatically generated from the YAML files in `/data/deprecations` by the rake task -located at `lib/tasks/gitlab/docs/compile_deprecations.rake`. +This page is automatically generated from the template located at +`data/deprecations/templates/_deprecation_template.md.erb`, using +the YAML files in `/data/deprecations` by the rake task +located at `lib/tasks/gitlab/docs/compile_deprecations.rake`, For deprecation authors (usually Product Managers and Engineering Managers): @@ -50,25 +52,11 @@ For deprecation reviewers (Technical Writers only): <div class="deprecation breaking-change" data-milestone="17.0"> -### Accessibility Testing is deprecated - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390424). -</div> - -Due to low customer usage, Accessibility Testing is deprecated and will be removed. There is no planned replacement and users should stop using Accessibility Testing before GitLab 17.0. - -</div> - -<div class="deprecation breaking-change" data-milestone="17.0"> - ### Atlassian Crowd OmniAuth provider <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369117). </div> @@ -85,7 +73,7 @@ next major release, GitLab 16.0. This gem sees very little use and its <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/211643). </div> @@ -97,25 +85,11 @@ Because Cloud Native Buildpacks do not support automatic testing, the Auto Test <div class="deprecation breaking-change" data-milestone="17.0"> -### Browser Performance Testing is deprecated - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388719). -</div> - -Due to limited customer usage, Browser Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Browser Performance Testing before GitLab 17.0. - -</div> - -<div class="deprecation breaking-change" data-milestone="17.0"> - ### CiRunner.projects default sort is changing to `id_desc` <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372117). </div> @@ -130,7 +104,7 @@ If you rely on the order of the returned projects to be `id_asc`, change your sc <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/409332). </div> @@ -145,7 +119,7 @@ the aliasing for the `CiRunnerUpgradeStatusType` type will be removed. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383467). </div> @@ -162,7 +136,7 @@ These three variables will be removed in GitLab 17.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.1</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/414864). </div> @@ -170,6 +144,36 @@ In GitLab 11.11 the Windows Batch executor, the CMD shell was deprecated in GitL </div> +<div class="deprecation breaking-change" data-milestone="17.0"> + +### Deprecate `CiRunner` GraphQL fields duplicated in `CiRunnerManager` + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">16.2</span> +- End of Support: GitLab <span class="milestone">17.0</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/41518). +</div> + +These fields (`architectureName`, `ipAddress`, `platformName`, `revision`, `version`) are now deprecated from the [GraphQL `CiRunner`](https://docs.gitlab.com/ee/api/graphql/reference/#cirunner) type as they are duplicated with the introduction of runner managers grouped within a runner configuration. + +</div> + +<div class="deprecation breaking-change" data-milestone="17.0"> + +### Deprecate `message` field from Vulnerability Management features + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">16.1</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411573). +</div> + +This MR deprecates the `message` field on the `VulnerabilityCreate` GraphQL mutation, and in the `AdditionalInfo` column of the vulnerability export. +The message field was removed from security reports schema in GitLab 16.0 and is no longer being used elsewhere. + +</div> + <div class="deprecation " data-milestone="17.0"> ### Deprecate legacy shell escaping and quoting runner shell executor @@ -186,11 +190,25 @@ The runner's legacy escape sequence mechanism to handle variable expansion imple <div class="deprecation breaking-change" data-milestone="17.0"> +### Deprecated parameters related to custom text in the sign-in page + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">16.2</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124461). +</div> + +The parameters, `sign_in_text` and `help_text`, are deprecated in the [Settings API](https://docs.gitlab.com/ee/api/settings.html). To add a custom text to the sign-in and sign-up pages, use the `description` field in the [Appearance API](https://docs.gitlab.com/ee/api/appearance.html). + +</div> + +<div class="deprecation breaking-change" data-milestone="17.0"> + ### DingTalk OmniAuth provider <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390855). </div> @@ -205,7 +223,7 @@ major release, GitLab 17.0. This gem sees very little use and is better suited f <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/9661). </div> @@ -231,7 +249,7 @@ To avoid any disruptions, you should replace `filepath` with `direct_asset_path` <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4097). </div> @@ -252,7 +270,7 @@ Because the new values provide a streamlined, comprehensive method to enable TLS <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387937). </div> @@ -269,7 +287,7 @@ are deprecated and will be removed from the GraphQL API. For installation instru <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.6</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/382077). </div> @@ -292,7 +310,7 @@ This change is a breaking change. You should use an [authentication token](../ci <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/414236). </div> @@ -304,44 +322,46 @@ Use `dependencyProxyTotalSizeBytes` instead, introduced in GitLab 16.1. <div class="deprecation breaking-change" data-milestone="17.0"> -### GraphQL type, `RunnerMembershipFilter` renamed to `CiRunnerMembershipFilter` +### GraphQL field `registrySizeEstimated` has been deprecated <div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/409333). +- Announced in: GitLab <span class="milestone">16.2</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/416509). </div> -The GraphQL type, `RunnerMembershipFilter`, has been renamed to `CiRunnerMembershipFilter`. In GitLab 17.0, -the aliasing for the `RunnerMembershipFilter` type will be removed. +For clarity, the GraphQL field `registrySizeEstimated` was renamed to `containerRegistrySizeIsEstimated`, to match its counterpart. +`registrySizeEstimated` was deprecated in GitLab 16.2 and will be removed in GitLab 17.0. +Use `containerRegistrySizeIsEstimated` introduced in GitLab 16.2 instead. </div> <div class="deprecation breaking-change" data-milestone="17.0"> -### GraphQL: The `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` enum is deprecated. Use `DISABLED_AND_OVERRIDABLE` instead +### GraphQL type, `RunnerMembershipFilter` renamed to `CiRunnerMembershipFilter` <div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385636). +- Announced in: GitLab <span class="milestone">16.0</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/409333). </div> -In GitLab 17.0, the `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` GraphQL enum type will be replaced with the value, `DISABLED_AND_OVERRIDABLE`. +The GraphQL type, `RunnerMembershipFilter`, has been renamed to `CiRunnerMembershipFilter`. In GitLab 17.0, +the aliasing for the `RunnerMembershipFilter` type will be removed. </div> <div class="deprecation breaking-change" data-milestone="17.0"> -### Load Performance Testing is deprecated +### GraphQL: The `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` enum is deprecated. Use `DISABLED_AND_OVERRIDABLE` instead <div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388723). +- Announced in: GitLab <span class="milestone">15.8</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385636). </div> -Due to low customer usage, Load Performance Testing is deprecated and will be removed. There is no planned replacement and users should stop using Load Performance Testing before GitLab 17.0. +In GitLab 17.0, the `DISABLED_WITH_OVERRIDE` value of the `SharedRunnersSetting` GraphQL enum type will be replaced with the value, `DISABLED_AND_OVERRIDABLE`. </div> @@ -351,7 +371,7 @@ Due to low customer usage, Load Performance Testing is deprecated and will be re <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/370471). </div> @@ -369,11 +389,25 @@ settings for the group using either the GitLab UI or GraphQL API. <div class="deprecation breaking-change" data-milestone="17.0"> +### OmniAuth Facebook is deprecated + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">16.2</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/416000). +</div> + +OmniAuth Facebook support will be removed in GitLab 17.0. The last gem release was in 2021 and it is currently unmaintained. The current usage is less than 0.1%. If you use OmniAuth Facebook, switch to a [supported provider](https://docs.gitlab.com/ee/integration/omniauth.html#supported-providers) in advance of support removal. + +</div> + +<div class="deprecation breaking-change" data-milestone="17.0"> + ### Package pipelines in API payload is paginated <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/289956). </div> @@ -389,7 +423,7 @@ In milestone 17.0, we will remove the `pipelines` attribute from the API respons <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343475). </div> @@ -404,7 +438,7 @@ The [`project_fingerprint`](https://gitlab.com/groups/gitlab-org/-/epics/2791) a <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/9065). </div> @@ -425,7 +459,7 @@ PostgreSQL 14 will also be supported for instances that want to upgrade prior to <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390787). </div> @@ -444,7 +478,7 @@ While the above approach is recommended for most instances, Sidekiq can also be <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.6</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/379743). </div> @@ -474,7 +508,7 @@ This change is a breaking change. You should [create a runner in the UI](../ci/r <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.6</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/380872). </div> @@ -501,7 +535,7 @@ This change is a breaking change. You should [create a runner in the UI](../ci/r <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389467). </div> @@ -518,11 +552,19 @@ that is available now. We recommend this alternative solution because it provide <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/411239). </div> -The option to run self-managed installations of GitLab on a single database is now deprecated. From GitLab 17.0, we will require a [separate database for CI features](https://gitlab.com/groups/gitlab-org/-/epics/7509). With this change, self-managed versions of GitLab will behave similarly to GitLab.com. This change applies to installation methods with Omnibus GitLab, GitLab Helm chart, GitLab Operator, GitLab Docker images, and installation from source. Before upgrading to GitLab 17.0, please ensure [migration](https://docs.gitlab.com/ee/administration/postgresql/multiple_databases.html) to two databases. +From GitLab 17.0, we will require a [separate database for CI features](https://gitlab.com/groups/gitlab-org/-/epics/7509). +We recommend running both databases on the same Postgres instance(s) due to ease of management for most deployments. + +We are providing this as an informational advance notice but we do not recommend taking action yet. +We will have another update communicated (as well as the deprecation note) when we recommend admins to start the migration process. + +This change provides additional scalability for the largest of GitLab instances, like GitLab.com. +This change applies to all installation methods: Omnibus GitLab, GitLab Helm chart, GitLab Operator, GitLab Docker images, and installation from source. +Before upgrading to GitLab 17.0, please ensure you have [migrated](https://docs.gitlab.com/ee/administration/postgresql/multiple_databases.html) to two databases. </div> @@ -532,7 +574,7 @@ The option to run self-managed installations of GitLab on a single database is n <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). </div> @@ -558,7 +600,7 @@ For updates and details about this deprecation, follow [this epic](https://gitla <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387898). </div> @@ -582,7 +624,7 @@ automatically from GitLab 16.0 onwards. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372411). </div> @@ -602,7 +644,7 @@ we'll be introducing support in [this epic](https://gitlab.com/groups/gitlab-org <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383341). </div> @@ -629,7 +671,7 @@ From GitLab 17.0 and later, the runner registration methods implemented by the n <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390263). </div> @@ -643,7 +685,7 @@ We will be transitioning to a new IID as a result of moving requirements to a [w <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387751). </div> @@ -658,7 +700,7 @@ Due to limited customer usage and capabilities, the Visual Reviews feature for R <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385235). </div> @@ -668,11 +710,29 @@ The [`gitlab-runner exec`](https://docs.gitlab.com/runner/commands/#gitlab-runne <div class="deprecation breaking-change" data-milestone="17.0"> +### The pull-based deployment features of the GitLab agent for Kubernetes is deprecated + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">16.2</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/406545). +</div> + +We are deprecating the built-in pull-based deployment features of the GitLab agent for Kubernetes in favor of Flux and related integrations. + +The GitLab agent for Kubernetes **is not deprecated**. This change affects only the pull-based functionality of the agent. All other functionality will remain intact, and GitLab will continue to support the agent for Kubernetes. + +If you use the agent for pull-based deployments, you should [migrate to Flux](https://docs.gitlab.com/ee/user/clusters/agent/gitops/agent.html#migrate-to-flux). Because Flux is a mature CNCF project for GitOps, we decided to [integrate Flux with GitLab in February 2023](https://about.gitlab.com/blog/2023/02/08/why-did-we-choose-to-integrate-fluxcd-with-gitlab/). + +</div> + +<div class="deprecation breaking-change" data-milestone="17.0"> + ### Trigger jobs can mirror downstream pipeline status exactly <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/285493). </div> @@ -686,7 +746,7 @@ In some cases, like when a downstream pipeline had the `passed with warnings` st <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/9662). </div> @@ -706,7 +766,7 @@ You can still access unified approval rules with the API. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372332). </div> @@ -724,7 +784,7 @@ removed in 17.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.6</span> - End of Support: GitLab <span class="milestone">17.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/381111). </div> @@ -744,7 +804,7 @@ From GitLab 17.0 and later, the methods to register runners introduced by the ne <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/398132). </div> @@ -776,9 +836,12 @@ echo $( ruby -rsecurerandom -e "puts SecureRandom.base64(32)" ) > ~/.gitlab-mail If you run GitLab on more than one machine, you need to provide the secret key file for each machine. -We highly encourage GitLab administrators to start using the `webhook` delivery method for +We encourage GitLab administrators to switch to the webhook delivery method for `incoming_email_delivery_method` and `service_desk_email_delivery_method` instead of `sidekiq`. +[Issue 393157](https://gitlab.com/gitlab-org/gitlab/-/issues/393157) tracks improving email ingestion in general. +We hope this will simplify infrastructure setup and add several improvements to how you manage GitLab in the near future. + </div> <div class="deprecation breaking-change" data-milestone="17.0"> @@ -787,7 +850,7 @@ We highly encourage GitLab administrators to start using the `webhook` delivery <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343475). </div> @@ -819,7 +882,7 @@ Enabling the `ldap_settings_unlock_groups_by_owners` feature flag allowed non-LD <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366798). </div> @@ -848,7 +911,7 @@ In GitLab 16.0 and later: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366798). </div> @@ -896,7 +959,7 @@ be available in CI/CD jobs. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> - End of Support: GitLab <span class="milestone">16.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7772). </div> @@ -914,6 +977,28 @@ In GitLab versions 16.0 to 16.2, you can still [re-enable the bundled Grafana](h However, enabling the bundled Grafana will no longer work from GitLab 16.3. </div> + +<div class="deprecation breaking-change" data-milestone="16.3"> + +### License Compliance CI Template + +<div class="deprecation-notes"> +- Announced in: GitLab <span class="milestone">15.9</span> +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). +- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387561). +</div> + +**Update:** We previously announced we would remove the existing License Compliance CI template in GitLab 16.0. However, due to performance issues with the [license scanning of CycloneDX files](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/) we will do this change in 16.3 instead. + +The GitLab [License Compliance](https://docs.gitlab.com/ee/user/compliance/license_compliance/) CI template is now deprecated and is scheduled for removal in the GitLab 16.1 release. Users who wish to continue using GitLab for License Compliance should remove the License Compliance template from their CI pipeline and add the [Dependency Scanning template](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuration). The Dependency Scanning template is now capable of gathering the required license information so it is no longer necessary to run a separate License Compliance job. The License Compliance CI template should not be removed prior to verifying that the `license_scanning_sbom_scanner` and `package_metadata_synchronization` flags are enabled for the instance and that the instance has been upgraded to a version that supports [the new method of license scanning](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/). + +| CI Pipeline Includes | GitLab <= 15.8 | 15.9 <= GitLab < 16.3 | GitLab >= 16.3 | +| ------------- | ------------- | ------------- | ------------- | +| Both DS and LS templates | License data from LS job is used | License data from LS job is used | License data from DS job is used | +| DS template is included but LS template is not | No license data | License data from DS job is used | License data from DS job is used | +| LS template is included but DS template is not | License data from LS job is used | License data from LS job is used | No license data | + +</div> </div> <div class="milestone-wrapper" data-milestone="16.1"> @@ -937,28 +1022,6 @@ We will stop publishing runner images based on the following, end-of-life Alpine - Alpine 3.14 (end-of-life on 2023-05-23) </div> - -<div class="deprecation breaking-change" data-milestone="16.1"> - -### License Compliance CI Template - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387561). -</div> - -**Update:** We previously announced we would remove the existing License Compliance CI template in GitLab 16.0. However, due to performance issues with the [license scanning of CycloneDX files](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/) we will do this change in 16.1 instead. - -The GitLab [License Compliance](https://docs.gitlab.com/ee/user/compliance/license_compliance/) CI template is now deprecated and is scheduled for removal in the GitLab 16.1 release. Users who wish to continue using GitLab for License Compliance should remove the License Compliance template from their CI pipeline and add the [Dependency Scanning template](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuration). The Dependency Scanning template is now capable of gathering the required license information so it is no longer necessary to run a separate License Compliance job. The License Compliance CI template should not be removed prior to verifying that the `license_scanning_sbom_scanner` and `package_metadata_synchronization` flags are enabled for the instance and that the instance has been upgraded to a version that supports [the new method of license scanning](https://docs.gitlab.com/ee/user/compliance/license_scanning_of_cyclonedx_files/). - -| CI Pipeline Includes | GitLab <= 15.8 | 15.9 <= GitLab < 16.1 | GitLab >= 16.1 | -| ------------- | ------------- | ------------- | ------------- | -| Both DS and LS templates | License data from LS job is used | License data from LS job is used | License data from DS job is used | -| DS template is included but LS template is not | No license data | License data from DS job is used | License data from DS job is used | -| LS template is included but DS template is not | License data from LS job is used | License data from LS job is used | No license data | - -</div> </div> <div class="milestone-wrapper" data-milestone="16.0"> @@ -971,7 +1034,7 @@ The GitLab [License Compliance](https://docs.gitlab.com/ee/user/compliance/licen <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343988). </div> @@ -990,7 +1053,7 @@ set the `POSTGRES_ENABLED` CI/CD variable to `true`. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/container-registry/-/issues/854). </div> @@ -1008,7 +1071,7 @@ This breaking change will happen in GitLab 16.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4353). </div> @@ -1031,7 +1094,7 @@ and [connect Grafana to the GitLab UI](https://docs.gitlab.com/ee/administration <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369127). </div> @@ -1047,7 +1110,7 @@ release, GitLab 16.0. This gem sees very little use and its lack of upstream mai <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/353080). </div> @@ -1061,7 +1124,7 @@ When using the native HashiCorp Vault integration, CI/CD jobs will fail when no <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/408396). </div> @@ -1078,7 +1141,7 @@ Instead of changing which single module would be scanned, we [improved multi-mod <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/353097). </div> @@ -1094,7 +1157,7 @@ Instead, use the [`/approval_rules` endpoint](https://docs.gitlab.com/ee/api/mer <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/384455). </div> @@ -1111,7 +1174,7 @@ This unintended functionality is deprecated in GitLab 15.8 and will be removed i <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.6</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/379064). </div> @@ -1125,7 +1188,7 @@ From GitLab 13.6, users can [specify any runner configuration in the GitLab Runn <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388255). </div> @@ -1143,7 +1206,7 @@ config file locations instead, for example `config/redis.cache.yml` or <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/container-registry/-/issues/842). </div> @@ -1157,7 +1220,7 @@ The Container Registry [pull-through cache](https://docs.docker.com/registry/rec <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/371840). </div> @@ -1171,7 +1234,7 @@ All Container Scanning variables that are prefixed by `DOCKER_` in variable name <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387299). </div> @@ -1187,7 +1250,7 @@ to continue to use the GitLab for Jira Cloud app. Without OAuth, you can't manag <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/384198). </div> @@ -1201,7 +1264,7 @@ With the move to the new DAST API analyzer and the `DAST-API.gitlab-ci.yml` temp <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383467). </div> @@ -1221,7 +1284,7 @@ These two variables will be removed in GitLab 16.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/384340). </div> @@ -1237,7 +1300,7 @@ These three variables will be removed in GitLab 16.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/395708). </div> @@ -1259,7 +1322,7 @@ To prepare for this change, users on GitLab.com or self-managed GitLab 15.9 or l <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387560). </div> @@ -1273,7 +1336,7 @@ GitLab has deprecated Dependency Scanning support for Java versions 13, 14, 15, <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/328500). </div> @@ -1287,7 +1350,7 @@ The Deployment API will now return an error when `updated_at` filtering and `upd <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). </div> @@ -1305,7 +1368,7 @@ GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configu <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7278). </div> @@ -1326,7 +1389,7 @@ For more information, see [the deprecation notes for Consul 1.9.0](https://githu <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/391896). </div> @@ -1340,7 +1403,7 @@ The [`CI_PRE_CLONE_SCRIPT` variable](https://docs.gitlab.com/ee/ci/runners/saas/ <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387891). </div> @@ -1356,7 +1419,7 @@ will be able to import projects to that group. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/375505). </div> @@ -1370,7 +1433,7 @@ In GitLab 16.0 the GitLab Dependency Scanning analyzer will begin reporting deve <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389477). </div> @@ -1385,7 +1448,7 @@ We intend to replace this feature with the ability to [embed charts](https://git <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372770). </div> @@ -1407,7 +1470,7 @@ Users on self-managed instances should update their pipelines to ensure they do <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/382532). </div> @@ -1421,7 +1484,7 @@ From GitLab 16.0, when you search for environments with the API, you must use at <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> In the [GraphQL API](https://docs.gitlab.com/ee/api/graphql/), the `external` field of [`ReleaseAssetLink` type](https://docs.gitlab.com/ee/api/graphql/reference/index.html#releaseassetlink) was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. @@ -1436,7 +1499,7 @@ To avoid any disruptions to your workflow, please stop using the `external` fiel <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> In [Releases API](https://docs.gitlab.com/ee/api/releases/) and [Release Links API](https://docs.gitlab.com/ee/api/releases/links.html), the `external` field was used to indicate whether a [release link](https://docs.gitlab.com/ee/user/project/releases/release_fields.html#links) is internal or external to your GitLab instance. @@ -1467,7 +1530,7 @@ GitLab 16.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348909). </div> @@ -1500,7 +1563,7 @@ See [migrated group items](https://docs.gitlab.com/ee/user/group/import/#migrate <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/344648). </div> @@ -1521,7 +1584,7 @@ When checking if a runner is `paused`, API users are advised to check the boolea <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/360545). </div> @@ -1540,7 +1603,7 @@ be present during the 16.x cycle to avoid breaking the API signature, and will b <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/371485). </div> @@ -1554,7 +1617,7 @@ The `confidential` field for a `Note` will be deprecated and renamed to `interna <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7508). </div> @@ -1571,7 +1634,7 @@ The Jira DVCS connector is also deprecated for Jira 8.13 and earlier. You can on <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383039). </div> @@ -1586,7 +1649,7 @@ This port is used for much more than just metrics, which warranted this change t <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/393574). </div> @@ -1609,7 +1672,7 @@ You should update to the new configuration structure as soon as possible using <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390291). </div> @@ -1631,7 +1694,7 @@ didn't match. The change improves consistency between Omnibus GitLab and source <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214217). </div> @@ -1651,7 +1714,7 @@ Update any scripts or bookmarks that reference the legacy URLs. GitLab APIs are <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390417). </div> @@ -1665,7 +1728,7 @@ The [License-Check feature](https://docs.gitlab.com/ee/user/compliance/license_c <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387721). </div> @@ -1679,7 +1742,7 @@ With external authorization enabled, personal access tokens (PATs) and deploy to <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/3442). </div> @@ -1701,7 +1764,7 @@ The full GitLab Helm Chart 7.0 upgrade steps will be available in the [upgrade d <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390417). </div> @@ -1718,10 +1781,10 @@ The [Managed Licenses API](https://docs.gitlab.com/ee/api/managed_licenses.html) - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/368195). </div> -The [**Maximum number of active pipelines per project** limit](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits) was never enabled by default and will be removed in GitLab 16.0. This limit can also be configured in the Rails console under [`ci_active_pipelines`](https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-pipelines-running-concurrently). Instead, use the other recommended rate limits that offer similar protection: +The [**Maximum number of active pipelines per project** limit](https://docs.gitlab.com/ee/administration/settings/continuous_integration.html#set-cicd-limits) was never enabled by default and will be removed in GitLab 16.0. This limit can also be configured in the Rails console under [`ci_active_pipelines`](https://docs.gitlab.com/ee/administration/instance_limits.html#number-of-pipelines-running-concurrently). Instead, use the other recommended rate limits that offer similar protection: -- [**Pipelines rate limits**](https://docs.gitlab.com/ee/user/admin_area/settings/rate_limit_on_pipelines_creation.html). -- [**Total number of jobs in currently active pipelines**](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits). +- [**Pipelines rate limits**](https://docs.gitlab.com/ee/administration/settings/rate_limit_on_pipelines_creation.html). +- [**Total number of jobs in currently active pipelines**](https://docs.gitlab.com/ee/administration/settings/continuous_integration.html#set-cicd-limits). </div> @@ -1731,7 +1794,7 @@ The [**Maximum number of active pipelines per project** limit](https://docs.gitl <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346541). </div> @@ -1746,7 +1809,7 @@ However, since certificate-based integration with Kubernetes clusters is depreca <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369122). </div> @@ -1773,7 +1836,7 @@ default is applied: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388269). </div> @@ -1794,7 +1857,7 @@ and `config/redis.shared_state.yml` files. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389557). </div> @@ -1812,7 +1875,7 @@ The option to delete groups and projects immediately by default was deprecated t <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346335). </div> @@ -1826,7 +1889,7 @@ Previously, the [PipelineSecurityReportFinding GraphQL type was updated](https:/ <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/349185). </div> @@ -1846,7 +1909,7 @@ Support for PostgreSQL 13 was added to Geo in GitLab 15.2. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385798). </div> @@ -1890,7 +1953,7 @@ Alternatives to using the `gitlab:import:repos` Rake task include: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.3</span> - End of Support: GitLab <span class="milestone">15.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331468). </div> @@ -1906,7 +1969,7 @@ If you are using your own Redis 5.0 instance, you should upgrade it to Redis 6.0 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.2</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/334253). </div> @@ -1922,7 +1985,7 @@ This could be a breaking change for anyone that developed their own runner that <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390416). </div> @@ -1962,7 +2025,7 @@ Work to replace the PHPCS Security Audit-based analyzer is tracked in [issue 364 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390912). </div> @@ -2005,7 +2068,7 @@ Specifically, the following are being deprecated and will no longer be updated a <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/391822). </div> @@ -2040,7 +2103,7 @@ However, due to compatibility issues [discussed in the deprecation issue](https: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366477). </div> @@ -2061,7 +2124,7 @@ For more information, refer to [security report validation](https://docs.gitlab. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/377824). </div> @@ -2076,7 +2139,7 @@ and will be moved to the JiHu GitLab codebase. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/368828). </div> @@ -2090,7 +2153,7 @@ GitLab's operational container scanning capabilities no longer require starboard <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390266). </div> @@ -2113,7 +2176,7 @@ This may require updating your metrics collection targets to also scrape `/db_me <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385564). </div> @@ -2137,7 +2200,7 @@ To use the full state name, including the period, [migrate to the full state fil <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/382129). </div> @@ -2162,7 +2225,7 @@ This change affects the following REST and GraphQL API endpoints: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-com/Product/-/issues/4894). </div> @@ -2176,7 +2239,7 @@ The [Phabricator task importer](https://docs.gitlab.com/ee/user/project/import/p <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/386001). </div> @@ -2199,7 +2262,7 @@ To accommodate the changes, you might need to adjust the [`rules`](https://docs. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/365365). </div> @@ -2213,7 +2276,7 @@ In order to make the behavior of toggling the draft status of a merge request mo <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/350670). </div> @@ -2227,7 +2290,7 @@ Toggling notes confidentiality with REST and GraphQL APIs is being deprecated. U <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/367166). </div> @@ -2242,7 +2305,7 @@ You can use the vulnerabilityFindingDismiss GraphQL mutation to set the status o <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376216). </div> @@ -2264,7 +2327,7 @@ Moving forward, we'll continue to invest in developing and releasing new feature <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/393836). </div> @@ -2286,7 +2349,7 @@ In GitLab 16.0 we will remove the ability to use a global ID in the work items p <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> - End of Support: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/377825). </div> @@ -2301,7 +2364,7 @@ and will be moved to the JiHu GitLab codebase. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352957). </div> @@ -2330,7 +2393,7 @@ The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in G <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/381669). </div> @@ -2344,7 +2407,7 @@ The `POST ci/lint` API endpoint is deprecated in 15.7, and will be removed in 16 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/365939). </div> @@ -2358,7 +2421,7 @@ To avoid confusion and duplication, the `environment_tier` parameter is deprecat <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/334018). </div> @@ -2378,7 +2441,7 @@ We plan to continue to support the `started` state in REST API version until the <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/375645). </div> @@ -2420,7 +2483,7 @@ Starting in GitLab 15.7 we started providing packages for openSUSE Leap 15.4, an <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> - End of Support: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387976). </div> @@ -2442,7 +2505,7 @@ We are deprecating support for [uploading backups to remote storage](https://doc <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383889). </div> @@ -2456,7 +2519,7 @@ The Live Preview feature of the Web IDE was intended to provide a client-side pr <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). </div> @@ -2485,7 +2548,7 @@ GitLab self-managed customers can still use the feature [with a feature flag](ht <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">15.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/29407). </div> @@ -2548,7 +2611,7 @@ This is not expected to cause any incompatibilities with the previous version of <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352554). </div> @@ -2565,7 +2628,6 @@ In GitLab 15.4, GitLab SAST will no longer use the following analyzers: NOTE: This change was originally planned for GitLab 15.0 and was postponed to GitLab 15.4. -See [the removal notice](./removals.md#sast-analyzer-consolidation-and-cicd-template-changes) for further details. These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). Effective immediately, they will receive only security updates; other routine improvements or updates are not guaranteed. @@ -2578,8 +2640,8 @@ This change will be reflected in the automatic language detection portion of the If you've already dismissed a vulnerability finding from one of the deprecated analyzers, the replacement attempts to respect your previous dismissal. The system behavior depends on: -- whether you’ve excluded the Semgrep-based analyzer from running in the past. -- which analyzer first discovered the vulnerabilities shown in the project’s Vulnerability Report. +- whether you've excluded the Semgrep-based analyzer from running in the past. +- which analyzer first discovered the vulnerabilities shown in the project's Vulnerability Report. See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. @@ -2651,7 +2713,7 @@ Long term service and support (LTSS) for [Debian 9 Stretch ends in July 2022](ht <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337993). </div> @@ -2669,7 +2731,7 @@ dramatically slow down GitLab instances. For this reason, they are being removed <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/26600). </div> @@ -2690,7 +2752,7 @@ GitLab will publish additional guidance to assist affected customers in migratin <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342800). </div> @@ -2704,7 +2766,7 @@ In GitLab 15.0 we are going to limit the number of characters in CI/CD job names <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345347). </div> @@ -2722,7 +2784,7 @@ Administrators who need to add runners for multiple projects can register a runn <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> All functionality related to GitLab's Container Network Security and Container Host Security categories is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies into GitLab, add the desired Helm charts into your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through GitLab [CI/CD](https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html). @@ -2813,7 +2875,7 @@ in the Vulnerability Report. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/334060). </div> @@ -2845,7 +2907,7 @@ gemnasium-python-dependency_scanning: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> In GitLab 15.0, for Dependency Scanning, the default version of Java that the scanner expects will be updated from 11 to 17. Java 17 is [the most up-to-date Long Term Support (LTS) version](https://en.wikipedia.org/wiki/Java_version_history). Dependency scanning continues to support the same [range of versions (8, 11, 13, 14, 15, 16, 17)](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#supported-languages-and-package-managers), only the default version is changing. If your project uses the previous default of Java 11, be sure to [set the `DS_Java_Version` variable to match](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning). @@ -2927,7 +2989,7 @@ The following `geo:db:*` tasks will be replaced with their corresponding `db:*:g <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/262019). </div> @@ -2941,7 +3003,7 @@ The feature flag `PUSH_RULES_SUPERSEDE_CODE_OWNERS` is being removed in GitLab 1 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/350275). </div> @@ -2984,7 +3046,7 @@ in the Vulnerability Report. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The [external status check API](https://docs.gitlab.com/ee/api/status_checks.html) was originally implemented to @@ -3023,7 +3085,7 @@ In 15.0, support for daemon mode for GitLab Pages will be removed. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/6). </div> @@ -3055,7 +3117,7 @@ and will remove it in GitLab 15.0 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/257883). </div> @@ -3119,7 +3181,7 @@ an inline argument expression). <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The GitLab Package stage offers a Package Registry, Container Registry, and Dependency Proxy to help you manage all of your dependencies using GitLab. Each of these product categories has a variety of settings that can be adjusted using the API. @@ -3139,7 +3201,7 @@ The permissions model for GraphQL is being updated. After 15.0, users with the G <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28192). </div> @@ -3155,7 +3217,7 @@ In GitLab 15.0 and later, the default value for this configuration option will c <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/335707). </div> @@ -3171,7 +3233,7 @@ If you are using our License Compliance API you should stop using the `approved` <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338182). </div> @@ -3189,7 +3251,7 @@ This deprecation mainly impacts users compiling GitLab from source because Omnib <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346485). </div> @@ -3215,7 +3277,7 @@ The [`custom_hooks_dir`](https://docs.gitlab.com/ee/administration/server_hooks. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The OAuth implicit grant authorization flow will be removed in our next major release, GitLab 15.0. Any applications that use OAuth implicit grant should switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). @@ -3228,7 +3290,7 @@ The OAuth implicit grant authorization flow will be removed in our next major re <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> By default, all new applications expire access tokens after 2 hours. In GitLab 14.2 and earlier, OAuth access tokens @@ -3239,7 +3301,7 @@ You should [opt in](https://docs.gitlab.com/ee/integration/oauth_provider.html#e tokens before GitLab 15.0 is released: 1. Edit the application. -1. Select **Expire access tokens** to enable them. Tokens must be revoked or they don’t expire. +1. Select **Expire access tokens** to enable them. Tokens must be revoked or they don't expire. </div> @@ -3249,7 +3311,7 @@ tokens before GitLab 15.0 is released: <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337384). </div> @@ -3267,7 +3329,7 @@ Note that we are not deprecating the Kerberos SPNEGO integration, only the old p <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351962). </div> @@ -3283,7 +3345,7 @@ Unexpected behavior in a security feature is inherently dangerous, so we have de <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351963). </div> @@ -3299,7 +3361,7 @@ Unexpected behavior in a security feature is inherently dangerous, so we have de <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352549). </div> @@ -3324,11 +3386,14 @@ If you rely on Java 8 being present in the analyzer environment, you must take a <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/359133). </div> -As Advanced Search migrations usually require support multiple code paths for a long period of time, it’s important to clean those up when we safely can. We use GitLab major version upgrades as a safe time to remove backward compatibility for indices that have not been fully migrated. See the [upgrade documentation](https://docs.gitlab.com/ee/update/index.html#upgrading-to-a-new-major-version) for details. +As Advanced Search migrations usually require support multiple code paths for a long period of time, +it's important to clean those up when we safely can. We use GitLab major version upgrades as a safe +time to remove backward compatibility for indices that have not been fully migrated. See the +[upgrade documentation](https://docs.gitlab.com/ee/update/index.html#upgrading-to-a-new-major-version) for details. </div> @@ -3354,7 +3419,7 @@ It is now considered deprecated, and will be removed in GitLab 15.0. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/332323). </div> @@ -3368,7 +3433,7 @@ The `instanceStatisticsMeasurements` GraphQL node has been renamed to `usageTren <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352488). </div> @@ -3388,10 +3453,10 @@ For more information, check the [summary section of the deprecation issue](https <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> -The [required pipeline configuration](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#required-pipeline-configuration) feature is deprecated in GitLab 14.8 for Premium customers and is scheduled for removal in GitLab 15.0. This feature is not deprecated for GitLab Ultimate customers. +The [required pipeline configuration](https://docs.gitlab.com/ee/administration/settings/continuous_integration.html#required-pipeline-configuration) feature is deprecated in GitLab 14.8 for Premium customers and is scheduled for removal in GitLab 15.0. This feature is not deprecated for GitLab Ultimate customers. This change to move the feature to GitLab's Ultimate tier is intended to help our features better align with our [pricing philosophy](https://about.gitlab.com/company/pricing/#three-tiers) as we see demand for this feature originating primarily from executives. @@ -3406,7 +3471,7 @@ This change will also help GitLab remain consistent in its tiering strategy with <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/350510). </div> @@ -3446,7 +3511,7 @@ in the Vulnerability Report. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352553). </div> @@ -3530,7 +3595,7 @@ in the Vulnerability Report. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352564). </div> @@ -3558,7 +3623,7 @@ See the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352564 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/350936). </div> @@ -3603,7 +3668,7 @@ Specifically, the following are being deprecated and will no longer be updated a <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347509). </div> @@ -3650,7 +3715,7 @@ Current users of the Static Site Editor can view the [documentation](https://doc <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. @@ -3663,7 +3728,7 @@ Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> Although not recommended or documented, it was possible to deploy a gRPC-aware proxy between Gitaly and @@ -3686,14 +3751,14 @@ the [relevant epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/463). <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> To simplify setting a test coverage pattern, in GitLab 15.0 the [project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-using-project-settings-removed) is being removed. -Instead, using the project’s `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set +Instead, using the project's `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set testing coverage results in merge requests. </div> @@ -3704,7 +3769,7 @@ testing coverage results in merge requests. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346540). </div> @@ -3718,7 +3783,7 @@ Tracing in GitLab is an integration with Jaeger, an open-source end-to-end distr <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336912). </div> @@ -3734,7 +3799,7 @@ The `GET /groups/:id/registry/repositories` endpoint will remain, but won't retu <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343210). </div> @@ -3750,7 +3815,7 @@ If you monitor Value Stream Analytics metrics and rely on the date filter, to av <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The vulnerability check feature is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. @@ -3770,7 +3835,7 @@ The new security approvals feature is similar to vulnerability check. For exampl <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/327453). </div> @@ -3786,7 +3851,7 @@ In milestone 15.0, we will completely remove `Version` from `PackageType`. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348980). </div> @@ -3803,7 +3868,7 @@ only supported report file in 15.0, but this is the first step towards GitLab su <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345451). </div> @@ -3817,7 +3882,7 @@ The GraphQL API field `defaultMergeCommitMessageWithDescription` has been deprec <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/276777). </div> @@ -3833,7 +3898,7 @@ In milestone 15.0, we will remove the feature flag entirely. Moving forward, you <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342882). </div> @@ -3852,7 +3917,7 @@ To mitigate possible performance problems, we will remove the `versions` field's <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The `projectFingerprint` field in the [PipelineSecurityReportFinding](https://docs.gitlab.com/ee/api/graphql/reference/index.html#pipelinesecurityreportfinding) @@ -3868,7 +3933,7 @@ exposed in the UUID field. Data previously available in the projectFingerprint f <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). </div> @@ -3882,7 +3947,7 @@ In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Ge <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). </div> @@ -3896,7 +3961,7 @@ In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Ge <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines that use these keywords will stop working, so you must switch to `stage` and `stages`, which have the same behavior. @@ -3909,7 +3974,7 @@ The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333233). </div> @@ -3925,7 +3990,7 @@ which isn't being used in GitLab anymore. <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). - To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/289832). </div> @@ -3941,7 +4006,7 @@ If you have explicitly excluded bundler-audit using DS_EXCLUDED_ANALYZERS you wi <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The Container Registry supports [authentication](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#auth) with `htpasswd`. It relies on an [Apache `htpasswd` file](https://httpd.apache.org/docs/2.4/programs/htpasswd.html), with passwords hashed using `bcrypt`. @@ -3956,7 +4021,7 @@ Since it isn't used in the context of GitLab (the product), `htpasswd` authentic <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The `user_email_lookup_limit` [API field](https://docs.gitlab.com/ee/api/settings.html) is deprecated and will be removed in GitLab 15.0. Until GitLab 15.0, `user_email_lookup_limit` is aliased to `search_rate_limit` and existing workflows will continue to work. @@ -3976,7 +4041,7 @@ Any API calls attempting to change the rate limits for `user_email_lookup_limit` <div class="deprecation-notes"> - Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). +- This is a [breaking change](https://docs.gitlab.com/ee/update/terminology.html#breaking-change). </div> The GitLab Composer repository can be used to push, search, fetch metadata about, and download PHP dependencies. All these actions require authentication, except for downloading dependencies. diff --git a/doc/update/img/batched_background_migrations_failed_v14_3.png b/doc/update/img/batched_background_migrations_failed_v14_3.png Binary files differnew file mode 100644 index 00000000000..dc8e3ff326b --- /dev/null +++ b/doc/update/img/batched_background_migrations_failed_v14_3.png diff --git a/doc/update/img/batched_background_migrations_queued_v14_0.png b/doc/update/img/batched_background_migrations_queued_v14_0.png Binary files differdeleted file mode 100644 index 0b0792b5e7a..00000000000 --- a/doc/update/img/batched_background_migrations_queued_v14_0.png +++ /dev/null diff --git a/doc/update/index.md b/doc/update/index.md index 0380f1a69ef..ced7ed86691 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -162,22 +162,14 @@ cannot guarantee that upgrading between major versions is seamless. A *major* upgrade requires the following steps: -1. Start by identifying a [supported upgrade path](#upgrade-paths). This is essential for a successful *major* version upgrade. -1. Upgrade to the latest minor version of the preceding major version. -1. Upgrade to the "dot zero" release of the next major version (`X.0.Z`). -1. Optional. Follow the [upgrade path](#upgrade-paths), and proceed with upgrading to newer releases of that major version. - -It's also important to ensure that any [background migrations have been fully completed](background_migrations.md) -before upgrading to a new major version. - -If you have enabled the [Elasticsearch integration](../integration/advanced_search/elasticsearch.md) **(PREMIUM SELF)**, then -[ensure all advanced search migrations are completed](#checking-for-pending-advanced-search-migrations) in the last minor version in -your current version -before proceeding with the major version upgrade. - -If your GitLab instance has any runners associated with it, it is very -important to upgrade GitLab Runner to match the GitLab minor version that was -upgraded to. This is to ensure [compatibility with GitLab versions](https://docs.gitlab.com/runner/#gitlab-runner-versions). +1. Identify a [supported upgrade path](#upgrade-paths). +1. Ensure that any [background migrations have been fully completed](background_migrations.md) + before upgrading to a new major version. +1. If you have enabled the [Elasticsearch integration](../integration/advanced_search/elasticsearch.md), then + before proceeding with the major version upgrade, [ensure that all advanced search migrations are completed](#checking-for-pending-advanced-search-migrations). +1. If your GitLab instance has any runners associated with it, it is very + important to upgrade them to match the current GitLab version. This ensures + [compatibility with GitLab versions](https://docs.gitlab.com/runner/#gitlab-runner-versions). ## Upgrade paths @@ -186,6 +178,15 @@ If you don't want any downtime, read how to [upgrade with zero downtime](zero_do For a dynamic view of examples of supported upgrade paths, try the [Upgrade Path tool](https://gitlab-com.gitlab.io/support/toolbox/upgrade-path/) maintained by the [GitLab Support team](https://about.gitlab.com/handbook/support/#about-the-support-team). To share feedback and help improve the tool, create an issue or MR in the [upgrade-path project](https://gitlab.com/gitlab-com/support/toolbox/upgrade-path). +Required upgrade stops are versions of GitLab that you must upgrade to before upgrading to later versions. Required upgrade stops allow required background +migrations to finish. + +During GitLab 16.x, we are scheduling two or three required upgrade stops. We will give at least two milestones of notice when we +schedule a required upgrade stop. + +The first planned required upgrade stop is scheduled for GitLab 16.3. If nothing is introduced requiring an upgrade stop, GitLab 16.3 will be treated as a +regular upgrade. + Find where your version sits in the upgrade path below, and upgrade GitLab accordingly, while also consulting the [version-specific upgrade instructions](#version-specific-upgrading-instructions): @@ -284,6 +285,13 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap [backfill `prepared_at` values on the `merge_requests` table](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111865). This migration may take multiple days to complete on larger GitLab instances. Make sure the migration has completed successfully before upgrading to 16.1.0. +- Geo: Some project imports do not initialize wiki repositories on project creation. Since the migration of project wikis to SSF, [missing wiki repositories are being incorrectly flagged as failing verification](https://gitlab.com/gitlab-org/gitlab/-/issues/409704). This is not a result of an actual replication/verification failure but an invalid internal state for these missing repositories inside Geo and results in errors in the logs and the verification progress reporting a failed state for these wiki repositories. If you have not imported projects you are not impacted by this issue. + - Impacted versions: GitLab versions 15.11.x, 16.0.x, and 16.1.0 - 16.1.2. + - Versions containing fix: GitLab 16.1.3 and later. +- Geo: Since the migration of project designs to SSF, [missing design repositories are being incorrectly flagged as failing verification](https://gitlab.com/gitlab-org/gitlab/-/issues/414279). This is not a result of an actual replication/verification failure but an invalid internal state for these missing repositories inside Geo and results in errors in the logs and the verification progress reporting a failed state for these design repositories. You could be impacted by this issue even if you have not imported projects. + - Impacted versions: GitLab versions 16.1.x. + - Versions containing fix: GitLab 16.2.0 and later. +- For self-compiled installations: You must remove any settings related to Puma worker killer from the `puma.rb` configuration file, since those have been [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118645). For more information, see the [`puma.rb.example`](https://gitlab.com/gitlab-org/gitlab/-/blob/16-0-stable-ee/config/puma.rb.example) file. ### 16.0.0 @@ -294,18 +302,35 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap all queues. This behavior does not apply if you have configured the [routing rules](../administration/sidekiq/processing_specific_job_classes.md#routing-rules). - Docker 20.10.10 or later is required to run the GitLab Docker image. Older versions [throw errors on startup](../install/docker.md#threaderror-cant-create-thread-operation-not-permitted). +- Geo: Some project imports do not initialize wiki repositories on project creation. Since the migration of project wikis to SSF, [missing wiki repositories are being incorrectly flagged as failing verification](https://gitlab.com/gitlab-org/gitlab/-/issues/409704). This is not a result of an actual replication/verification failure but an invalid internal state for these missing repositories inside Geo and results in errors in the logs and the verification progress reporting a failed state for these wiki repositories. If you have not imported projects you are not impacted by this issue. + - Impacted versions: GitLab versions 15.11.x, 16.0.x, and 16.1.0 - 16.1.2. + - Versions containing fix: GitLab 16.1.3 and later. +- Starting with 16.0, GitLab self-managed installations now have two database connections by default, instead of one. This change doubles the number of PostgreSQL connections. It makes self-managed versions of GitLab behave similarly to GitLab.com, and is a step toward enabling a separate database for CI features for self-managed versions of GitLab. Before upgrading to 16.0, determine if you need to [increase max connections for PostgreSQL](https://docs.gitlab.com/omnibus/settings/database.html#configuring-multiple-database-connections). + - This change applies to installation methods with Linux packages (Omnibus GitLab), GitLab Helm chart, GitLab Operator, GitLab Docker images, and installation from source. ### 15.11.1 - Many [project importers](../user/project/import/index.md) and [group importers](../user/group/import/index.md) now require the Maintainer role instead of only requiring the Developer role. For more information, see the documentation for any importers you use. +- Geo: Some project imports do not initialize wiki repositories on project creation. Since the migration of project wikis to SSF, [missing wiki repositories are being incorrectly flagged as failing verification](https://gitlab.com/gitlab-org/gitlab/-/issues/409704). This is not a result of an actual replication/verification failure but an invalid internal state for these missing repositories inside Geo and results in errors in the logs and the verification progress reporting a failed state for these wiki repositories. If you have not imported projects you are not impacted by this issue. + - Impacted versions: GitLab versions 15.11.x, 16.0.x, and 16.1.0 - 16.1.2. + - Versions containing fix: GitLab 16.1.3 and later. +- Geo: A [bug](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7841) in the built-in `pg-upgrade` tool prevents upgrading the bundled PostgreSQL database to version 13. This leaves the secondary site in a broken state, and prevents upgrading the Geo installation to GitLab 16.x ([PostgreSQL 12 support has removed in 16.0](deprecations.md#postgresql-12-deprecated) and later releases). This occurs on secondary sites using the bundled PostgreSQL software, running both the secondary main Rails database and tracking database on the same node. There is a manual [workaround](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7841#workaround) for those impacted until a fix is backported to 15.11. + - Impacted versions: GitLab versions 15.2 - 15.11 + - Version 16.0 and later are not impacted. Note, 15.11 is a mandatory upgrade stop on the way to 16.0. ### 15.11.0 - Upgrades to GitLab 15.11 directly from GitLab versions 15.5.0 and earlier on self-managed installs will fail due to a missing migration until the fix for [issue 408304](https://gitlab.com/gitlab-org/gitlab/-/issues/408304) is released in version 15.11.3. Affected users wanting to upgrade to 15.11 can either: - Perform an intermediate upgrade to any version between 15.5 and 15.10 before upgrading to 15.11, or - Target version 15.11.3 or later. +- Geo: Some project imports do not initialize wiki repositories on project creation. Since the migration of project wikis to SSF, [missing wiki repositories are being incorrectly flagged as failing verification](https://gitlab.com/gitlab-org/gitlab/-/issues/409704). This is not a result of an actual replication/verification failure but an invalid internal state for these missing repositories inside Geo and results in errors in the logs and the verification progress reporting a failed state for these wiki repositories. If you have not imported projects you are not impacted by this issue. + - Impacted versions: GitLab versions 15.11.x, 16.0.x, and 16.1.0 - 16.1.2. + - Versions containing fix: GitLab 16.1.3 and later. +- Geo: A [bug](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7841) in the built-in `pg-upgrade` tool prevents upgrading the bundled PostgreSQL database to version 13. This leaves the secondary site in a broken state, and prevents upgrading the Geo installation to GitLab 16.x ([PostgreSQL 12 support has removed in 16.0](deprecations.md#postgresql-12-deprecated) and later releases). This occurs on secondary sites using the bundled PostgreSQL software, running both the secondary main Rails database and tracking database on the same node. There is a manual [workaround](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7841#workaround) for those impacted until a fix is backported to 15.11. + - Impacted versions: GitLab versions 15.2 - 15.11 + - Version 16.0 and later are not impacted. Note, 15.11 is a mandatory upgrade stop on the way to 16.0. ### 15.10.5 @@ -317,6 +342,51 @@ and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with ap - Gitaly configuration changes significantly in Omnibus GitLab 16.0. You can begin migrating to the new structure in Omnibus GitLab 15.10 while backwards compatibility is maintained in the lead up to Omnibus GitLab 16.0. [Read more about this change](#gitaly-omnibus-gitlab-configuration-structure-change). +- You might encounter the following error while upgrading to GitLab 15.10 or later: + + ```shell + STDOUT: rake aborted! + StandardError: An error has occurred, all later migrations canceled: + PG::CheckViolation: ERROR: check constraint "check_70f294ef54" is violated by some row + ``` + + This error is caused by a [batched background migration introduced in GitLab 15.8](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107701) + not being finalized before GitLab 15.10. To resolve this error: + + 1. Execute the following SQL statement using the database console (`sudo gitlab-psql` for Linux package installs): + + ```sql + UPDATE oauth_access_tokens SET expires_in = '7200' WHERE expires_in IS NULL; + ``` + + 1. [Re-run database migrations](../administration/raketasks/maintenance.md#run-incomplete-database-migrations). + +- You might also encounter the following error while upgrading to GitLab 15.10 or later: + + ```shell + "exception.class": "ActiveRecord::StatementInvalid", + "exception.message": "PG::SyntaxError: ERROR: zero-length delimited identifier at or near \"\"\"\"\nLINE 1: ...COALESCE(\"lock_version\", 0) + 1 WHERE \"ci_builds\".\"\" IN (SEL...\n + ``` + + This error is caused by a [batched background migration introduced in GitLab 14.9](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81410) + not being finalized before upgrading to GitLab 15.10 or later. To resolve this error, it is safe to [mark the migration as complete](background_migrations.md#mark-a-batched-migration-finished): + + ```ruby + # Start the rails console + + connection = Ci::ApplicationRecord.connection + + Gitlab::Database::SharedModel.using_connection(connection) do + migration = Gitlab::Database::BackgroundMigration::BatchedMigration.find_for_configuration( + Gitlab::Database.gitlab_schemas_for_connection(connection), 'NullifyOrphanRunnerIdOnCiBuilds', :ci_builds, :id, []) + + # mark all jobs completed + migration.batched_jobs.update_all(status: Gitlab::Database::BackgroundMigration::BatchedJob.state_machine.states[:succeeded].value) + migration.update_attribute(:status, Gitlab::Database::BackgroundMigration::BatchedMigration.state_machine.states[:finished].value) + end + ``` + +For more information, see [issue 415724](https://gitlab.com/gitlab-org/gitlab/-/issues/415724). ### 15.9.0 @@ -780,7 +850,7 @@ A [license caching issue](https://gitlab.com/gitlab-org/gitlab/-/issues/376706) - If you run external PostgreSQL, particularly AWS RDS, [check you have a PostgreSQL bug fix](#postgresql-segmentation-fault-issue) to avoid the database crashing. -- The use of encrypted S3 buckets with storage-specific configuration is no longer supported after [removing support for using `background_upload`](removals.md#background-upload-for-object-storage). +- The use of encrypted S3 buckets with storage-specific configuration is no longer supported after [removing support for using `background_upload`](deprecations.md#background-upload-for-object-storage). - The [certificate-based Kubernetes integration (DEPRECATED)](../user/infrastructure/clusters/index.md#certificate-based-kubernetes-integration-deprecated) is disabled by default, but you can be re-enable it through the [`certificate_based_clusters` feature flag](../administration/feature_flags.md#how-to-enable-and-disable-features-behind-flags) until GitLab 16.0. - When you use the GitLab Helm Chart project with a custom `serviceAccount`, ensure it has `get` and `list` permissions for the `serviceAccount` and `secret` resources. - The [`custom_hooks_dir`](../administration/server_hooks.md#create-global-server-hooks-for-all-repositories) setting for configuring global server hooks is now configured in diff --git a/doc/update/package/convert_to_ee.md b/doc/update/package/convert_to_ee.md index 0e8e0b1e569..31de896349c 100644 --- a/doc/update/package/convert_to_ee.md +++ b/doc/update/package/convert_to_ee.md @@ -22,7 +22,7 @@ that may require Support intervention. The steps can be summed up to: -1. Make a [GitLab backup](../../raketasks/backup_gitlab.md). +1. Make a [GitLab backup](../../administration/backup_restore/backup_gitlab.md). 1. Find the currently installed GitLab version: @@ -69,7 +69,7 @@ The steps can be summed up to: If you want to use `dpkg`/`rpm` instead of `apt-get`/`yum`, go through the first step to find the current GitLab version, then follow [Upgrade using a manually-downloaded package](index.md#upgrade-using-a-manually-downloaded-package), - and then [add your license](../../user/admin_area/license.md). + and then [add your license](../../administration/license.md). 1. Install the `gitlab-ee` package. The install automatically uninstalls the `gitlab-ce` package on your GitLab server. `reconfigure` @@ -99,7 +99,7 @@ The steps can be summed up to: sudo gitlab-ctl reconfigure ``` -1. Now activate GitLab Enterprise Edition by [adding your license](../../user/admin_area/license.md). +1. Now activate GitLab Enterprise Edition by [adding your license](../../administration/license.md). 1. After you confirm that GitLab is working as expected, you may remove the old Community Edition repository: diff --git a/doc/update/package/downgrade.md b/doc/update/package/downgrade.md index 7b48f1f4045..14b9bd981fd 100644 --- a/doc/update/package/downgrade.md +++ b/doc/update/package/downgrade.md @@ -12,7 +12,7 @@ of a package. WARNING: You must at least have a database backup created under the version you are downgrading to. Ideally, you should have a -[full backup archive](../../raketasks/backup_restore.md) +[full backup archive](../../administration/backup_restore/index.md) on hand. The example below demonstrates the downgrade procedure when downgrading between minor @@ -79,5 +79,5 @@ Steps: sudo gitlab-ctl reconfigure ``` -1. [Restore GitLab](../../raketasks/restore_gitlab.md#restore-for-omnibus-gitlab-installations) +1. [Restore GitLab](../../administration/backup_restore/restore_gitlab.md#restore-for-omnibus-gitlab-installations) to complete the downgrade. diff --git a/doc/update/package/index.md b/doc/update/package/index.md index 3e0d09eb36e..e61eaae5883 100644 --- a/doc/update/package/index.md +++ b/doc/update/package/index.md @@ -39,6 +39,7 @@ GitLab package. Upgrading versions might need some manual intervention. For more information, check the version your are upgrading to: +- [GitLab 16](https://docs.gitlab.com/omnibus/update/gitlab_16_changes.html) - [GitLab 15](https://docs.gitlab.com/omnibus/update/gitlab_15_changes.html) - [GitLab 14](https://docs.gitlab.com/omnibus/update/gitlab_14_changes.html) - [GitLab 13](https://docs.gitlab.com/omnibus/update/gitlab_13_changes.html) @@ -56,7 +57,7 @@ sudo touch /etc/gitlab/skip-auto-backup ``` Nevertheless, it is highly recommended to maintain a full up-to-date -[backup](../../raketasks/backup_restore.md) on your own. +[backup](../../administration/backup_restore/index.md) on your own. ## Upgrade using the official repositories @@ -178,7 +179,7 @@ To download and install GitLab: # Debian/Ubuntu dpkg -i <package_name> - # RHEL/CentOS 6 and 7 + # RHEL/CentOS 6 and 7 rpm -Uvh <package_name> # RHEL/CentOS 8 diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index 964c6430a16..3211b732f0a 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -14,7 +14,7 @@ You can select the tag in the version dropdown list in the upper-left corner of ### 0. Backup -Make a backup just in case things go south. Depending on the installation method, backup commands vary. See the [backing up and restoring GitLab](../raketasks/backup_restore.md) documentation. +Make a backup just in case things go south. Depending on the installation method, backup commands vary. See the [backing up and restoring GitLab](../administration/backup_restore/index.md) documentation. ### 1. Stop server diff --git a/doc/update/plan_your_upgrade.md b/doc/update/plan_your_upgrade.md index 9378b104c81..634c430e251 100644 --- a/doc/update/plan_your_upgrade.md +++ b/doc/update/plan_your_upgrade.md @@ -11,7 +11,7 @@ GitLab instance. General notes: -- If possible, we recommend you test out the upgrade in a test environment before +- If possible, you should test out the upgrade in a test environment before updating your production instance. Ideally, your test environment should mimic your production environment as closely as possible. - If [working with Support](https://about.gitlab.com/support/scheduling-upgrade-assistance/) @@ -75,25 +75,25 @@ Create a backup of GitLab and all its data (database, repositories, uploads, bui artifacts, LFS objects, registry, pages). This is vital for making it possible to roll back GitLab to a working state if there's a problem with the upgrade: -- Create a [GitLab backup](../raketasks/backup_restore.md). +- Create a [GitLab backup](../administration/backup_restore/index.md). Make sure to follow the instructions based on your installation method. - Don't forget to back up the [secrets and configuration files](../raketasks/backup_gitlab.md#storing-configuration-files). + Don't forget to back up the [secrets and configuration files](../administration/backup_restore/backup_gitlab.md#storing-configuration-files). - Alternatively, create a snapshot of your instance. If this is a multi-node installation, you must snapshot every node. **This process is out of scope for GitLab Support.** ### Restore GitLab -If you have a test environment that mimics your production one, we recommend testing the restoration to ensure that everything works as you expect. +If you have a test environment that mimics your production one, you should test the restoration to ensure that everything works as you expect. To restore your GitLab backup: - Before restoring, make sure to read about the - [prerequisites](../raketasks/backup_restore.md#restore-gitlab), most importantly, + [prerequisites](../administration/backup_restore/index.md#restore-gitlab), most importantly, the versions of the backed up and the new GitLab instance must be the same. -- [Restore GitLab](../raketasks/backup_restore.md#restore-gitlab). +- [Restore GitLab](../administration/backup_restore/index.md#restore-gitlab). Make sure to follow the instructions based on your installation method. - Confirm that the [secrets and configuration files](../raketasks/backup_gitlab.md#storing-configuration-files) are also restored. + Confirm that the [secrets and configuration files](../administration/backup_restore/backup_gitlab.md#storing-configuration-files) are also restored. - If restoring from a snapshot, know the steps to do this. **This process is out of scope for GitLab Support.** diff --git a/doc/update/removals.md b/doc/update/removals.md index a5cd33e50d9..56f8b34fd27 100644 --- a/doc/update/removals.md +++ b/doc/update/removals.md @@ -1,2593 +1,11 @@ --- -stage: none -group: none -info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments-to-development-guidelines" +redirect_to: 'deprecations.md' +remove_date: '2023-09-28' --- -# Removals by version +This document was moved to [another location](deprecations.md). -In each release, GitLab removes features that were [deprecated](deprecations.md) in an earlier release. -Some features cause breaking changes when they are removed. - -**{rss}** **To be notified of upcoming breaking changes**, -add this URL to your RSS feed reader: `https://about.gitlab.com/breaking-changes.xml` - -<!-- vale off --> - -<!-- -DO NOT EDIT THIS PAGE DIRECTLY - -This page is automatically generated from the YAML files in `/data/removals` by the rake task -located at `lib/tasks/gitlab/docs/compile_removals.rake`. - -For removal authors (usually Product Managers and Engineering Managers): - -- To add a removal, use the example.yml file in `/data/removals/templates` as a template. -- For more information about authoring removals, check the the removal item guidance: - https://about.gitlab.com/handbook/marketing/blog/release-posts/#creating-a-removal-entry - -For removal reviewers (Technical Writers only): - -- To update the removal doc, run: `bin/rake gitlab:docs:compile_removals` -- To verify the removals doc is up to date, run: `bin/rake gitlab:docs:check_removals` -- For more information about updating the removal doc, see the removal doc update guidance: - https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-removals-doc ---> - -{::options parse_block_html="true" /} - -## Removed in 16.0 - -### Auto DevOps no longer provisions a database by default - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/343988). -</div> - -Currently, Auto DevOps provisions an in-cluster PostgreSQL database by default. -In GitLab 16.0, databases will be provisioned only for users who opt in. This -change supports production deployments that require more robust database management. - -If you want Auto DevOps to provision an in-cluster database, -set the `POSTGRES_ENABLED` CI/CD variable to `true`. - -### Azure Storage Driver defaults to the correct root prefix - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/container-registry/-/issues/854). -</div> - -The Azure Storage Driver used to write to `//` as the default root directory. This default root directory appeared in some places in the Azure UI as `/<no-name>/`. We maintained this legacy behavior to support older deployments using this storage driver. However, when moving to Azure from another storage driver, this behavior hides all your data until you configure the storage driver with `trimlegacyrootprefix: true` to build root paths without an extra leading slash. - -In GitLab 16.0, the new default configuration for the storage driver uses `trimlegacyrootprefix: true`, and `/` is the default root directory. You can set your configuration to `trimlegacyrootprefix: false` if needed, to revert to the previous behavior. - -### Bundled Grafana Helm Chart - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/4353). -</div> - -The Grafana Helm chart that was bundled with the GitLab Helm Chart is removed in the GitLab Helm Chart 7.0 release (releasing along with GitLab 16.0). - -The `global.grafana.enabled` setting for the GitLab Helm Chart has also been removed alongside the Grafana Helm chart. - -If you're using the bundled Grafana, you should switch to the [newer chart version from Grafana Labs](https://artifacthub.io/packages/helm/grafana/grafana) -or a Grafana Operator from a trusted provider. - -In your new Grafana instance, [configure the GitLab provided Prometheus as a data source](https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html#configure-grafana) -and [connect Grafana to the GitLab UI](https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html#integrate-with-gitlab-ui). - -### CAS OmniAuth provider is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369127). -</div> - -The `omniauth-cas3` gem that provides GitLab with the CAS OmniAuth provider is being removed. You can no longer authenticate into a GitLab instance through CAS. This gem sees very little use. [The gem](https://rubygems.org/gems/omniauth-cas3/) has not had a new release in almost 5 years, which means that its dependencies are out of date and required manual patching during GitLab's [upgrade to OmniAuth 2.0](https://gitlab.com/gitlab-org/gitlab/-/issues/30073). - -### CiCdSettingsUpdate mutation renamed to ProjectCiCdSettingsUpdate - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/361801). -</div> - -The `CiCdSettingsUpdate` mutation was renamed to `ProjectCiCdSettingsUpdate` in GitLab 15.0. -The `CiCdSettingsUpdate` mutation will be removed in GitLab 16.0. -Any user scripts that use the `CiCdSettingsUpdate` mutation must be updated to use `ProjectCiCdSettingsUpdate` -instead. - -### Conan project-level search returns only project-specific results" - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/384455). -</div> - -The [GitLab Conan repository](https://docs.gitlab.com/ee/user/packages/conan_repository/) supports the `conan search` command, but when searching a project-level endpoint, instance-level Conan packages could have been returned. This unintended functionality is removed in GitLab 16.0. The search endpoint for the project level now only returns packages from the target project. - -### Configuring Redis config file paths using environment variables is no longer supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388255). -</div> - -You can no longer specify Redis configuration file locations -using the environment variables like `GITLAB_REDIS_CACHE_CONFIG_FILE` or -`GITLAB_REDIS_QUEUES_CONFIG_FILE`. Use the default -configuration file locations instead, for example `config/redis.cache.yml` or -`config/redis.queues.yml`. - -### Container Registry pull-through cache is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/container-registry/-/issues/937). -</div> - -The Container Registry [pull-through cache](https://docs.docker.com/registry/recipes/mirror/) was deprecated in GitLab 15.8 and removed in GitLab 16.0. This feature is part of the upstream [Docker Distribution project](https://github.com/distribution/distribution) but we are removing that code in favor of the GitLab Dependency Proxy. Use the GitLab Dependency Proxy to proxy and cache container images from Docker Hub. - -### Container Scanning variables that reference Docker removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/371840). -</div> - -All Container Scanning variables with a name prefixed by `DOCKER_` have been removed. This includes: - -- `DOCKER_IMAGE` -- `DOCKER_PASSWORD` -- `DOCKER_USER` -- `DOCKERFILE_PATH` - -Instead, use the [new variable names](https://docs.gitlab.com/ee/user/application_security/container_scanning/#available-cicd-variables): - -- `CS_IMAGE` -- `CS_REGISTRY_PASSWORD` -- `CS_REGISTRY_USER` -- `CS_DOCKERFILE_PATH` - -### Default value of `ttl_days` now 30 days - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369122). -</div> - -From GitLab 16.0, any personal, project, or group access token [must have an expiration date](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96594). If you create a personal access token with the GitLab Shell command `personal_access_token` without specifying `ttl_days`, a default value of 30 days is now applied. - -### Dependency Scanning ends support for Java 13, 14, 15, and 16 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387560). -</div> - -Dependency Scanning no longer supports projects that use Java versions 13, 14, 15, and 16. - -### Developer role providing the ability to import projects to a group - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/387891). -</div> - -The ability for users with the Developer role for a group to import projects to that group was deprecated in GitLab -15.8 and is removed in GitLab 16.0. - -From GitLab 16.0, only users with at least the Maintainer role for a group can import projects to that group. - -### Embedding Grafana panels in Markdown is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389477). -</div> - -The ability to add Grafana panels in GitLab Flavored Markdown is removed. -We intend to replace this feature with the ability to [embed charts](https://gitlab.com/groups/gitlab-org/opstrace/-/epics/33) -with the [GitLab Observability UI](https://gitlab.com/gitlab-org/opstrace/opstrace-ui). - -### Enforced validation of CI/CD parameter character lengths - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372770). -</div> - -Previously, only CI/CD [job names](https://docs.gitlab.com/ee/ci/jobs/index.html#job-name-limitations) had a strict 255-character limit. Now, more CI/CD keywords are validated to ensure they stay under the limit. - -The following to 255 characters are now strictly limited to 255 characters: - -- The `stage` keyword. -- The `ref` parameter, which is the Git branch or tag name for the pipeline. -- The `description` and `target_url` parameters, used by external CI/CD integrations. - -Users on self-managed instances should update their pipelines to ensure they do not use parameters that exceed 255 characters. Users on GitLab.com do not need to make any changes, as these parameters are already limited in that database. - -### GitLab administrators must have permission to modify protected branches or tags - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">16.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/12776). -</div> - -GitLab administrators can no longer perform actions on protected branches or tags unless they have been explicitly granted that permission. These actions include pushing and merging into a [protected branch](https://docs.gitlab.com/ee/user/project/protected_branches.html), unprotecting a branch, and creating [protected tags](https://docs.gitlab.com/ee/user/project/protected_tags.html). - -### GitLab.com importer - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-com/Product/-/issues/4895). -</div> - -The [GitLab.com importer](https://docs.gitlab.com/ee/user/project/import/gitlab_com.html) was deprecated in GitLab 15.8 and is removed in GitLab 16.0. - -The GitLab.com importer was introduced in 2015 for importing a project from GitLab.com to a self-managed GitLab instance through the UI. - -This feature was available on self-managed instances only. [Migrating GitLab groups and projects by direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) -supersedes the GitLab.com importer and provides a more cohesive importing functionality. - -See [migrated group items](https://docs.gitlab.com/ee/user/group/import/#migrated-group-items) and [migrated project items](https://docs.gitlab.com/ee/user/group/import/#migrated-project-items) for an overview. - -### GraphQL API: Runner status no longer returns `PAUSED` and `ACTIVE` values - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/344648). -</div> - -In GitLab 16.0 and later, the GraphQL query for runners will no longer return the statuses `PAUSED` and `ACTIVE`. - -- `PAUSED` has been replaced with the field, `paused: true`. -- `ACTIVE` has been replaced with the field, `paused: false`. - -### Jira DVCS connector for Jira Cloud and Jira 8.13 and earlier - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/362168). -</div> - -The [Jira DVCS connector](https://docs.gitlab.com/ee/integration/jira/dvcs/) for Jira Cloud was deprecated in GitLab 15.1 and has been removed in 16.0. Use the [GitLab for Jira Cloud app](https://docs.gitlab.com/ee/integration/jira/connect-app.html) instead. The Jira DVCS connector was also deprecated for Jira 8.13 and earlier. You can only use the Jira DVCS connector with Jira Data Center or Jira Server in Jira 8.14 and later. Upgrade your Jira instance to Jira 8.14 or later, and reconfigure the Jira integration in your GitLab instance. -If you cannot upgrade your Jira instance in time and are on GitLab self-managed version, we offer a workaround until GitLab 16.6. This breaking change is deployed in GitLab 16.0 behind a feature flag named `jira_dvcs_end_of_life_amnesty`. The flag is disabled by default, but you can ask an administrator to enable the flag at any time. For questions related to this announcement, see the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/408185). - -### Legacy Gitaly configuration method - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/393574). -</div> - -Previously, Gitaly configuration keys for Omnibus GitLab were scattered throughout the configuration file. In GitLab -15.10, we added support for a single configuration structure that matches Gitaly internal configuration. Both methods -of configuring Gitaly were supported in parallel. - -In GitLab 16.0, we removed support for the former configuration method and now only support the new configuration -method. - -Before upgrading to GitLab 16.0, administrators must migrate to the new single configuration structure. For -instructions, see [Gitaly - Omnibus GitLab configuration structure change](https://docs.gitlab.com/ee/update/#gitaly-omnibus-gitlab-configuration-structure-change). - -### Legacy Gitaly configuration methods with variables - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). -</div> - -The environment variables `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` were deprecated in GitLab 14.8 and are removed -in GitLab 16.0. These variables are replaced with standard -[`config.toml` Gitaly configuration](https://docs.gitlab.com/ee/administration/gitaly/reference.html). - -GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly must switch to configuring -using `config.toml`. - -### Legacy Praefect configuration method - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390291). -</div> - -Previously, Praefect configuration keys for Omnibus GitLab were scattered throughout the configuration file. In GitLab -15.9, we added support for a single configuration structure that matches Praefect internal configuration. Both methods -of configuring Praefect were supported in parallel. - -In GitLab 16.0, we removed support for the former configuration method and now only support the new configuration -method. - -Before upgrading to GitLab 16.0, administrators must migrate to the new single configuration structure. For -instructions, see [Praefect - Omnibus GitLab configuration structure change](https://docs.gitlab.com/ee/update/#praefect-omnibus-gitlab-configuration-structure-change). - -### Legacy routes removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214217). -</div> - -GitLab 16.0 removes legacy URLs from the GitLab application. - -When subgroups were introduced in GitLab 9.0, a `/-/` delimiter was added to URLs to signify the end of a group path. All GitLab URLs now use this delimiter for project, group, and instance level features. - -URLs that do not use the `/-/` delimiter are planned for removal in GitLab 16.0. For the full list of these URLs, along with their replacements, see [issue 28848](https://gitlab.com/gitlab-org/gitlab/-/issues/28848#release-notes). - -Update any scripts or bookmarks that reference the legacy URLs. GitLab APIs are not affected by this change. - -### License-Check and the Policies tab on the License Compliance page - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390417). -</div> - -The License Check Policies feature has been removed. Additionally, the Policies tab on the License Compliance page and all APIs related to the License Check feature have been removed. To enforce approvals based on detected licenses, use the [License Approval policy](https://docs.gitlab.com/ee/user/compliance/license_approval_policies.html) feature instead. - -### Limit CI_JOB_TOKEN scope is disabled - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/395708). -</div> - -In GitLab 14.4 we introduced the ability to [limit your project's CI/CD job token](https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html#limit-your-projects-job-token-access) (`CI_JOB_TOKEN`) access to make it more secure. You could use the **Limit CI_JOB_TOKEN access** setting to prevent job tokens from your project's pipelines from being used to **access other projects**. When enabled with no other configuration, your pipelines could not access any other projects. To use job tokens to access other projects from your project's pipelines, you needed to list those other projects explicitly in the setting's allowlist, and you needed to be a maintainer in _all_ the projects. You might have seen this mentioned as the "outbound scope" of the job token. - -The job token functionality was updated in 15.9 with a [better security setting](https://docs.gitlab.com/ee/ci/jobs/ci_job_token.html#allow-access-to-your-project-with-a-job-token). Instead of securing your own project's job tokens from accessing other projects, the new workflow is to secure your own project from being accessed by other projects' job tokens without authorization. You can see this as an "inbound scope" for job tokens. When this new **Allow access to this project with a CI_JOB_TOKEN** setting is enabled with no other configuration, job tokens from other projects cannot **access your project**. If you want a project to have access to your own project, you must list it in the new setting's allowlist. You must be a maintainer in your own project to control the new allowlist, but you only need to have the Guest role in the other projects. This new setting is enabled by default for all new projects. - -In GitLab 16.0, the old **Limit CI_JOB_TOKEN access** setting is disabled by default for all **new** projects. In existing projects with this setting currently enabled, it will continue to function as expected, but you are unable to add any more projects to the old allowlist. If the setting is disabled in any project, it is not possible to re-enable this setting in 16.0 or later. To control access between your projects, use the new **Allow access** setting instead. - -In 17.0, we plan to remove the **Limit** setting completely, and set the **Allow access** setting to enabled for all projects. This change ensures a higher level of security between projects. If you currently use the **Limit** setting, you should update your projects to use the **Allow access** setting instead. If other projects access your project with a job token, you must add them to the **Allow access** setting's allowlist. - -To prepare for this change, users on GitLab.com or self-managed GitLab 15.9 or later can enable the **Allow access** setting now and add the other projects. It will not be possible to disable the setting in 17.0 or later. - -### Managed Licenses API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390417). -</div> - -The [Managed Licenses API](https://archives.docs.gitlab.com/15.8/ee/api/managed_licenses.html) has been removed. To enforce approvals in merge requests when non-compliant licenses are detected, use the [License Approval policy](https://docs.gitlab.com/ee/user/compliance/license_approval_policies.html) feature instead. - -Our [GraphQL APIs](https://docs.gitlab.com/ee/api/graphql/reference/) can be used to create a Security Policy Project, [update the policy.yml](https://docs.gitlab.com/ee/api/graphql/reference/#mutationscanexecutionpolicycommit) in the Security Policy Project, and enforce those policies. - -To query a list of dependencies and components, use our [Dependencies REST API](https://docs.gitlab.com/ee/api/dependencies.html) or [export from the Dependency List](https://docs.gitlab.com/ee/user/application_security/dependency_list/). - -### Maximum number of active pipelines per project limit (`ci_active_pipelines`) - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.3</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/368195). -</div> - -The [**Maximum number of active pipelines per project** limit](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits) has been removed. Instead, use the other recommended rate limits that offer similar protection: - -- [**Pipelines rate limits**](https://docs.gitlab.com/ee/user/admin_area/settings/rate_limit_on_pipelines_creation.html). -- [**Total number of jobs in currently active pipelines**](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#set-cicd-limits). - -### Monitoring performance metrics through Prometheus is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346541). -</div> - -We previously launched a solution that allows you to view performance metrics by displaying data stored in a Prometheus instance. -The Prometheus instance can be set up as a GitLab-managed app or you can connect a previously configured Prometheus instance. -The latter is known as an "external Prometheus" in GitLab. The value we provided was to enable you to easily set up monitoring -(using GitLab Managed Apps) and have the visualization of the metrics all in the same tool you used to build the application. - -However, as we are removing certificate-based integrations, the full monitoring experience is also deprecated as you will not -have the option to easily set up Prometheus from GitLab. Furthermore, we plan to consolidate on -a focused observability dashboard experience instead of having multiple paths to view metrics. Because of this, we are also removing the external -Prometheus experience, together with the metrics visualization capability. - -This removal only refers to the GitLab Metrics capabilities, and **does not** include: - -- Deprecating [alerts for Prometheus](https://gitlab.com/gitlab-org/gitlab/-/issues/338834). -- [Capabilities that GitLab comes with that allow operators of GitLab to retrieve metrics from those instances](https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_metrics.html). - -### Non-expiring access tokens no longer supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/369123). -</div> - -Currently, you can create access tokens that have no expiration date. These access tokens are valid indefinitely, which presents a security risk if the access token is -divulged. Because expiring access tokens are better, from GitLab 15.4 we [populate a default expiration date](https://gitlab.com/gitlab-org/gitlab/-/issues/348660). - -In GitLab 16.0, any personal, project, or group access token that does not have an expiration date will automatically have an expiration date set at 365 days later than the current date. - -### Non-standard default Redis ports are no longer supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388269). -</div> - -If GitLab starts without any Redis configuration file present, -GitLab assumes it can connect to three Redis servers at `localhost:6380`, -`localhost:6381` and `localhost:6382`. We are changing this behavior -so GitLab assumes there is one Redis server at `localhost:6379`. - -If you want to keep using the three servers, you must configure -the Redis URLs by editing the `config/redis.cache.yml`,`config/redis.queues.yml`, -and `config/redis.shared_state.yml` files. - -### PipelineSecurityReportFinding name GraphQL field - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.1</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346335). -</div> - -Previously, the [PipelineSecurityReportFinding GraphQL type was updated](https://gitlab.com/gitlab-org/gitlab/-/issues/335372) to include a new `title` field. This field is an alias for the current `name` field, making the less specific `name` field redundant. The `name` field is removed from the PipelineSecurityReportFinding type in GitLab 16.0. - -### PostgreSQL 12 compatibility - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7395). -</div> - -In GitLab 16.0, PostgreSQL 13 is the minimum supported PostgreSQL version. PostgreSQL 12 is no longer shipped with the GitLab Omnibus package. -Before upgrading to GitLab 16.0, if you are: - -- Still using GitLab-packaged PostgreSQL 12, you must [perform a database upgrade](https://docs.gitlab.com/omnibus/settings/database.html#upgrade-packaged-postgresql-server) - to PostgreSQL 13. -- Using an externally-provided PostgreSQL 12, you must upgrade to PostgreSQL 13 or later to meet the - [minimum version requirements](https://docs.gitlab.com/ee/install/requirements.html#postgresql-requirements). - -### Praefect custom metrics endpoint configuration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390266). -</div> - -Support for using the `prometheus_exclude_database_from_default_metrics` configuration value was deprecated in -GitLab 15.9 and is removed in GitLab 16.0. We made this change to improve the performance of Praefect. -All metrics that scrape the Praefect database are now exported to the `/db_metrics` endpoint. - -You must update your metrics collection targets to use the `/db_metrics` endpoint. - -### Project REST API field `operations_access_level` removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/385798). -</div> - -In project REST API endpoints, the `operations_access_level` field -is removed in favor of more specialized fields like: - -- `releases_access_level` -- `environments_access_level` -- `feature_flags_access_level` -- `infrastructure_access_level` -- `monitor_access_level` - -### Rake task for importing bare repositories - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-com/Product/-/issues/5255). -</div> - -The [Rake task for importing bare repositories](https://docs.gitlab.com/ee/raketasks/import.html) `gitlab:import:repos` was deprecated in GitLab 15.8 and is removed in GitLab 16.0. - -This Rake task imported a directory tree of repositories into a GitLab instance. These repositories must have been -managed by GitLab previously, because the Rake task relied on the specific directory structure or a specific custom Git setting in order to work (`gitlab.fullpath`). - -Importing repositories using this Rake task had limitations. The Rake task: - -- Only knew about project and project wiki repositories and didn't support repositories for designs, group wikis, or snippets. -- Permitted you to import non-hashed storage projects even though these aren't supported. -- Relied on having Git config `gitlab.fullpath` set. [Epic 8953](https://gitlab.com/groups/gitlab-org/-/epics/8953) proposes removing support for this setting. - -Alternatives to using the `gitlab:import:repos` Rake task include: - -- Migrating projects using either [an export file](https://docs.gitlab.com/ee/user/project/settings/import_export.html) or - [direct transfer](https://docs.gitlab.com/ee/user/group/import/#migrate-groups-by-direct-transfer-recommended) migrate repositories as well. -- Importing a [repository by URL](https://docs.gitlab.com/ee/user/project/import/repo_by_url.html). -- Importing a [repositories from a non-GitLab source](https://docs.gitlab.com/ee/user/project/import/). - -### Redis 5 compatibility - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331468). -</div> - -In GitLab 13.9, we updated the Omnibus GitLab package and GitLab Helm chart 4.9 to Redis 6. Redis 5 reached end of life in April 2022 and is not supported. - -GitLab 16.0, we have removed support for Redis 5. If you are using your own Redis 5.0 instance, you must upgrade it to Redis 6.0 or later before upgrading to GitLab 16.0 -or later. - -### Removal of job_age parameter in `POST /jobs/request` Runner endpoint - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.2</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/334253). -</div> - -The `job_age` parameter, returned from the `POST /jobs/request` API endpoint used in communication with GitLab Runner, has been removed in GitLab 16.0. - -This could be a breaking change for anyone that developed their own runner that relies on this parameter being returned by the endpoint. This is not a breaking change for anyone using an officially released version of GitLab Runner, including public shared runners on GitLab.com. - -### Remove legacy configuration fields in GitLab Runner Helm Chart - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.6</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/379064). -</div> - -In GitLab 13.6 and later, users can [specify any runner configuration in the GitLab Runner Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html). When this features was released, we deprecated the fields in the GitLab Helm Chart configuration specific to the runner. As of v1.0 of the GitLab Runner Helm chart (GitLab 16.0), the following fields have been removed and are no longer supported: - -- `image` -- `rbac.resources` -- `rbac.verbs` -- `runners.image` -- `runners.imagePullSecrets` -- `runners.imagePullPolicy` -- `runners.requestConcurrency` -- `runners.privileged` -- `runners.namespace` -- `runners.pollTimeout` -- `runners.outputLimit` -- `runners.cache.cacheType` -- `runners.cache.cachePath` -- `runners.cache.cacheShared` -- `runners.cache.s3ServerAddress` -- `runners.cache.s3BucketLocation` -- `runners.cache.s3CacheInsecure` -- `runners.cache.gcsBucketName` -- `runners.builds` -- `runners.services` -- `runners.helpers` -- `runners.pod_security_context` -- `runners.serviceAccountName` -- `runners.cloneUrl` -- `runners.nodeSelector` -- `runners.nodeTolerations` -- `runners.podLabels` -- `runners.podAnnotations` -- `runners.env` - -### Remove the deprecated `environment_tier` parameter from the DORA API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.2</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/365939). -</div> - -The `environment_tier` parameter has been superseded by the `environment_tiers` parameter. - -If you use the `environment_tier` parameter in your integration (REST or GraphQL) then you need to replace it with the `environment_tiers` parameter which accepts an array of strings. - -### Removed `external` field from GraphQL `ReleaseAssetLink` type - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388983). -</div> - -From GitLab 15.9, all Release links are external. The `external` field of the `ReleaseAssetLink` type was deprecated in 15.9, and removed in GitLab 16.0. - -### Removed `external` field from Releases and Release link APIs - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388984). -</div> - -From GitLab 15.9, all Release links are external. The `external` field in the Releases and Release link APIs was deprecated in 15.9, and removed in GitLab 16.0. - -### Secure JWT token setting is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366798). -</div> - -As part of [the deprecation of old versions of JSON web tokens](https://docs.gitlab.com/ee/update/deprecations.html#old-versions-of-json-web-tokens-are-deprecated), the **Limit JSON Web Token (JWT)** project setting has been removed. This setting was a temporary solution to help users transition to [ID tokens](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html), as a way to switch between the old and new tokens, but the setting is no longer needed. In GitLab 16.0 and later, you can simply start using ID tokens in any job. When you use the `id_tokens` keyword in a job, that job uses only ID tokens and the old `CI_JOB_JWT*` tokens are not available. In jobs that do not use the `id_tokens` keyword, the old behavior remains unchanged. - -The old `CI_JOB_JWT*` tokens will be completely removed in GitLab 16.5, so you must switch to ID tokens before that release. - -### Secure scanning `_DISABLED` variables now require the value `"true"` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/391822). -</div> - -In GitLab 16.0, we've changed how values for CI/CD variables like `SAST_DISABLED` and `DEPENDENCY_SCANNING_DISABLED` are handled. - -Now, scanning is disabled only if the value is `"true"`, for example `SAST_DISABLED: "true"`. Previously, even if the value were `"false"`, like `SAST_DISABLED: "false"`, scanning would still be disabled. - -This change was previously released in the Latest versions of the CI/CD templates because of the potential to disrupt customized CI/CD pipeline configurations. - -The following templates have been updated: - -- API Fuzzing: [`API-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/API-Fuzzing.gitlab-ci.yml) -- Container Scanning: [`Container-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Container-Scanning.gitlab-ci.yml) -- Coverage-Guided Fuzzing: [`Coverage-Fuzzing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Coverage-Fuzzing.gitlab-ci.yml) -- DAST: [`DAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST.gitlab-ci.yml) -- DAST API: [`DAST-API.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/DAST-API.gitlab-ci.yml) -- Dependency Scanning: [`Dependency-Scanning.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Dependency-Scanning.gitlab-ci.yml) -- IaC Scanning: [`SAST-IaC.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml) -- SAST: [`SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) -- Secret Detection: [`Secret-Detection.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Secret-Detection.gitlab-ci.yml) - -If you currently use the `_DISABLED` variables but set a value other than `"true"` to disable scanning, change the value to `"true"`. - -### Security report schemas version 14.x.x - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/366477). -</div> - -Version 14.x.x [security report schemas](https://gitlab.com/gitlab-org/security-products/security-report-schemas) have been removed. -Security reports that use schema version 14.x.x will cause an error in the pipeline's **Security** tab. For more information, refer to [security report validation](https://docs.gitlab.com/ee/user/application_security/#security-report-validation). - -### Self-monitoring project is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/10030). -</div> - -GitLab self-monitoring project was meant to enable self-hosted GitLab administrators to visualize performance metrics of GitLab within GitLab itself. This feature relied on GitLab Metrics dashboards. With metrics dashboard being removed, self-monitoring project is also removed. We recommended that self-hosted users monitor their GitLab instance with alternative visualization tools, such as Grafana. - -### Starboard directive in the config for the GitLab agent for Kubernetes removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/368828). -</div> - -The GitLab operational container scanning feature no longer requires you to install Starboard. The `starboard:` directive in configuration files for the GitLab agent for Kubernetes has been removed. Use the `container_scanning:` directive instead. - -### Stop publishing GitLab Runner images based on Windows Server 2004 and 20H2 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">16.0</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/31001). -</div> - -As of GitLab 16.0, GitLab Runner images based on Windows Server 2004 and 20H2 will not be provided as these operating systems are end-of-life. - -### The Phabricator task importer - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-com/Product/-/issues/4894). -</div> - -The [Phabricator task importer](https://docs.gitlab.com/ee/user/project/import/phabricator.html) was deprecated in -GitLab 15.7 and is removed in 16.0. - -The Phabricator project hasn't been actively maintained since June 1, 2021. We haven't observed imports using this -tool. There has been no activity on the open related issues on GitLab. - -### The Security Code Scan-based GitLab SAST analyzer is now removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390416). -</div> - -GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. -We've reduced the number of supported analyzers used by default in GitLab SAST. -This is part of our long-term strategy to deliver a faster, more consistent user experience across different programming languages. - -As of GitLab 16.0, the [SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) no longer uses the [Security Code Scan](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan)-based analyzer for .NET. -We've removed this analyzer from the SAST CI/CD template and replaced it with GitLab-supported detection rules for C# in the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). - -Because this analyzer has reached End of Support in GitLab 16.0, we won't provide further updates to it. -However, we won't delete any container images we previously published for this analyzer or remove the ability to run it by using a [custom CI/CD pipeline job](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportssast). - -If you've already dismissed a vulnerability finding from the deprecated analyzer, the replacement attempts to respect your previous dismissal. See [Vulnerability translation documentation](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#vulnerability-translation) for further details. - -If you customize the behavior of GitLab SAST by disabling the Semgrep-based analyzer or depending on specific SAST jobs in your pipelines, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/390416#actions-required). - -### The stable Terraform CI/CD template has been replaced with the latest template - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/386001). -</div> - -With every major GitLab version, we update the stable Terraform templates with the current latest templates. -This change affects the [quickstart](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) -and the [base](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) templates. - -The new templates do not change the directory to `$TF_ROOT` explicitly: `gitlab-terraform` gracefully -handles directory changing. If you altered the job scripts to assume that the current working directory is `$TF_ROOT`, you must manually add `cd "$TF_ROOT"` now. - -Because the latest template introduces Merge Request Pipeline support which is not supported in Auto DevOps, -those rules are not yet integrated into the stable template. -However, we may introduce them later on, which may break your Terraform pipelines in regards to which jobs are executed. - -To accommodate the changes, you might need to adjust the [`rules`](https://docs.gitlab.com/ee/ci/yaml/#rules) in your -`.gitlab-ci.yml` file. - -### Two DAST API variables have been removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383467). -</div> - -The variables `DAST_API_HOST_OVERRIDE` and `DAST_API_SPECIFICATION` have been removed from use for DAST API scans. - -`DAST_API_HOST_OVERRIDE` has been removed in favor of using the `DAST_API_TARGET_URL` to automatically override the host in the OpenAPI specification. - -`DAST_API_SPECIFICATION` has been removed in favor of `DAST_API_OPENAPI`. To continue using an OpenAPI specification to guide the test, users must replace the `DAST_API_SPECIFICATION` variable with the `DAST_API_OPENAPI` variable. The value can remain the same, but the variable name must be replaced. - -### Use of `id` field in vulnerabilityFindingDismiss mutation - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/367166). -</div> - -You can use the vulnerabilityFindingDismiss GraphQL mutation to set the status of a vulnerability finding to `Dismissed`. Previously, this mutation used the `id` field to identify findings uniquely. However, this did not work for dismissing findings from the pipeline security tab. Therefore, using the `id` field as an identifier has been dropped in favor of the `uuid` field. Using the 'uuid' field as an identifier allows you to dismiss the finding from the pipeline security tab. - -### Vulnerability confidence field - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.4</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372332). -</div> - -In GitLab 15.3, [security report schemas below version 15 were deprecated](https://docs.gitlab.com/ee/update/deprecations.html#security-report-schemas-version-14xx). -The `confidence` attribute on vulnerability findings exists only in schema versions before `15-0-0` and in GitLab prior to 15.4. To maintain consistency -between the reports and our public APIs, the `confidence` attribute on any vulnerability-related components of our GraphQL API is now removed. - -### `CI_BUILD_*` predefined variables removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352957). -</div> - -The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in GitLab 9.0, and removed in GitLab 16.0. If you still use these variables, you must change to the replacement [predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) which are functionally identical: - -| Removed variable | Replacement variable | -| --------------------- |------------------------ | -| `CI_BUILD_BEFORE_SHA` | `CI_COMMIT_BEFORE_SHA` | -| `CI_BUILD_ID` | `CI_JOB_ID` | -| `CI_BUILD_MANUAL` | `CI_JOB_MANUAL` | -| `CI_BUILD_NAME` | `CI_JOB_NAME` | -| `CI_BUILD_REF` | `CI_COMMIT_SHA` | -| `CI_BUILD_REF_NAME` | `CI_COMMIT_REF_NAME` | -| `CI_BUILD_REF_SLUG` | `CI_COMMIT_REF_SLUG` | -| `CI_BUILD_REPO` | `CI_REPOSITORY_URL` | -| `CI_BUILD_STAGE` | `CI_JOB_STAGE` | -| `CI_BUILD_TAG` | `CI_COMMIT_TAG` | -| `CI_BUILD_TOKEN` | `CI_JOB_TOKEN` | -| `CI_BUILD_TRIGGERED` | `CI_PIPELINE_TRIGGERED` | - -### `CI_PRE_CLONE_SCRIPT` variable on GitLab SaaS Runners has been removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29405). -</div> - -In GitLab 16.0 and later, the `CI_PRE_CLONE_SCRIPT` variable option on GitLab SaaS Runners has been removed. The `CI_PRE_CLONE_SCRIPT` variable enabled you to run commands in your CI/CD job before the runner executed `git-init` and `git-fetch`. You should use the `pre_get_sources_script` hook instead. For more information, see the blog post, [Guide to pre_clone_script changes on GitLab SaaS Linux Runners](https://about.gitlab.com/blog/2023/03/27/changes-to-the-preclonescript/). - -### `POST /projects/:id/merge_requests/:merge_request_iid/approvals` removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">12.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/353097). -</div> - -The `/approvals` endpoint was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/11132) in GitLab 12.3. To change the approvals required for a merge request via the API, use the `/approval_rules` endpoint described in [Create merge request level rule](https://docs.gitlab.com/ee/api/merge_request_approvals.html#create-merge-request-level-rule). - -### `POST ci/lint` API endpoint removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/381669). -</div> - -The `POST ci/lint` API endpoint was deprecated in 15.7, and removed in 16.0. This endpoint did not validate the full range of CI/CD configuration options. Instead, use [`POST /projects/:id/ci/lint`](https://docs.gitlab.com/ee/api/lint.html#validate-a-ci-yaml-configuration-with-a-namespace), which properly validates CI/CD configuration. - -### `docker-ssh` and `docker-ssh+machine` executors are removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">10.0</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29406). -</div> - -In GitLab 16.0 and later, the `docker-ssh` and `docker+machine-ssh` executors for GitLab Runner have been removed from the GitLab Runner [code base](https://gitlab.com/gitlab-org/gitlab-runner). - -### vulnerabilityFindingDismiss GraphQL mutation - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/375645). -</div> - -The `VulnerabilityFindingDismiss` GraphQL mutation has been removed. This mutation was not used often as the Vulnerability Finding ID was not available to users (this field was [deprecated in 15.3](https://docs.gitlab.com/ee/update/deprecations.html#use-of-id-field-in-vulnerabilityfindingdismiss-mutation)). Instead of `VulnerabilityFindingDismiss`, you should use `VulnerabilityDismiss` to dismiss vulnerabilities in the Vulnerability Report or `SecurityFindingDismiss` for security findings in the CI Pipeline Security tab. - -## Removed in 15.11 - -### Exporting and importing projects in JSON format not supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.10</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389888). -</div> - -GitLab previously created project file exports in JSON format. In GitLab 12.10, NDJSON was added as the preferred format. - -To support transitions, importing JSON-formatted project file exports was still possible if you configured the -relevant feature flags. - -From GitLab 15.11, importing a JSON-formatted project file exports is not supported. - -### openSUSE Leap 15.3 packages - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/7371). -</div> - -Distribution support and security updates for openSUSE Leap 15.3 [ended December 2022](https://en.opensuse.org/Lifetime#Discontinued_distributions). - -Starting in GitLab 15.7 we started providing packages for openSUSE Leap 15.4, and in GitLab 15.11 we stopped providing packages for openSUSE Leap 15.3. - -To continue using GitLab, [upgrade](https://en.opensuse.org/SDB:System_upgrade) to openSUSE Leap 15.4. - -## Removed in 15.9 - -### Live Preview no longer available in the Web IDE - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/383889). -</div> - -The Live Preview feature of the Web IDE was intended to provide a client-side preview of static web applications. However, complex configuration steps and a narrow set of supported project types have limited its utility. With the introduction of the Web IDE Beta in GitLab 15.7, you can now connect to a full server-side runtime environment. With upcoming support for installing extensions in the Web IDE, we’ll also support more advanced workflows than those available with Live Preview. As of GitLab 15.9, Live Preview is no longer available in the Web IDE. - -### `omniauth-authentiq` gem no longer available - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389452). -</div> - -`omniauth-authentiq` is an OmniAuth strategy gem that was part of GitLab. The company providing authentication services, Authentiq, has shut down. Therefore the gem is being removed. - -### `omniauth-shibboleth` gem no longer available - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">10.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/388959). -</div> - -`omniauth-shibboleth` is an OmniAuth strategy gem that was part of GitLab. The gem has not received security updates and does not meet GitLab quality guidance criteria. This gem was originally scheduled for removal by 14.1, but it was not removed at that time. The gem is being removed now. In GitLab 16.1 or later, use the `omniauth-shibboleth-redux` gem instead. - -## Removed in 15.8 - -### CiliumNetworkPolicy within the auto deploy Helm chart is removed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/382044). -</div> - -All functionality related to the GitLab Container Network Security and Container Host Security categories was deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. The [CiliumNetworkPolicy definition](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/master/assets/auto-deploy-app/values.yaml#L175) that exists as part of the [GitLab Auto Deploy Helm chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) was not removed as scheduled in GitLab 15.0. This policy is planned to be removed in the GitLab 15.8 release. - -If you want to preserve this functionality, you can follow one of these two paths: - -1. Fork the [GitLab Auto Deploy Helm chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) into the `chart/` path within your project -1. Set `AUTO_DEPLOY_IMAGE_VERSION` and `DAST_AUTO_DEPLOY_IMAGE_VERSION` to the most recent version of the image that included the CiliumNetworkPolicy - -### Exporting and importing groups in JSON format not supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.10</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/389888). -</div> - -GitLab previously created group file exports in JSON format. In GitLab 13.10, NDJSON was added as the preferred format. - -To support transitions, importing JSON-formatted group file exports was still possible if you configured the -relevant feature flags. - -From GitLab 15.8, importing a JSON-formatted group file exports is not supported. - -### `artifacts:public` CI/CD keyword refactored - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/322454). -</div> - -The [`artifacts:public` CI/CD keyword](https://docs.gitlab.com/ee/ci/yaml/#artifactspublic) was discovered to be not working properly since GitLab 15.8 and needed to be refactored. This feature is disabled on GitLab.com, and disabled by default on self-managed instances. If an administrator for an instance running GitLab 15.8 or 15.9 enabled this feature via the `non_public_artifacts` feature flag, it is likely that artifacts created with the `public:false` setting are being treated as `public:true`. - -If you have projects that use this setting, you should delete artifacts that must not be public, or [change the visibility](https://docs.gitlab.com/ee/user/public_access.html#change-project-visibility) of affected projects to private, before updating to GitLab 15.8 or later. - -In GitLab 15.10, this feature's code was refactored. On instances with this feature enabled, new artifacts created with `public:false` are now working as expected, though still disabled by default. Avoid testing this feature with production data until it is enabled by default and made generally available. - -## Removed in 15.7 - -### File Type variable expansion in `.gitlab-ci.yml` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/29407). -</div> - -Prior to this change, variables that referenced or applied alias file variables expanded the value of the `File` type variable. For example, the file contents. This behavior was incorrect because it did not comply with typical shell variable expansion rules. A user could run an $echo command with the variable as an input parameter to leak secrets or sensitive information stored in 'File' type variables. - -In 15.7, we are removing file type variable expansion from GitLab. It is essential to check your CI pipelines to confirm if your scripts reference a file variable. If your CI job relies on the previous expansion functionality, that CI job will not work and generate an error as of 15.7. The new behavior is that variable expansion that reference or apply alias file variables expand to the file name or path of the `File` type variable, instead of its value, such as the file contents. - -### Flowdock integration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.7</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/379197). -</div> - -As of December 22, 2022, we are removing the Flowdock integration because the service was shut down on August 15, 2022. - -## Removed in 15.6 - -### NFS as Git repository storage is no longer supported - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.0</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351243). -</div> - -As of November 22, 2022, we have removed support for customers using NFS for Git repository storage. This was -originally planned for May 22, 2022, but in an effort to allow continued maturity of Gitaly Cluster, we delayed -our removal of support date until now. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support/#gitaly-and-nfs) -for further information. - -This change in support follows the development deprecation for NFS for Git repository storage that occurred in GitLab 14.0. - -Gitaly Cluster offers tremendous benefits for our customers such as: - -- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/index.html#replication-factor). -- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/index.html#strong-consistency). -- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/index.html#distributed-reads). - -We encourage customers currently using NFS for Git repositories to migrate as soon as possible by reviewing our documentation on -[migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/index.html#migrate-to-gitaly-cluster). - -## Removed in 15.4 - -### SAST analyzer consolidation and CI/CD template changes - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352554). -</div> - -We have replaced the GitLab SAST [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) for certain languages in GitLab 15.4 as part of our long-term strategy to deliver a more consistent user experience, faster scan times, and reduced CI minute usage. - -Starting from GitLab 15.4, the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) uses [Semgrep-based scanning](https://docs.gitlab.com/ee/user/application_security/sast/analyzers.html#transition-to-semgrep-based-scanning) instead of the following analyzers: - -- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) for JavaScript, TypeScript, React -- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) for Go -- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) for Python -- [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) for Java - -We will no longer make any updates to the ESLint-, Gosec-, and Bandit-based analyzers. -The SpotBugs-based analyzer will continue to be used for Groovy, Kotlin, and Scala scanning. - -We won't delete container images previously published for these analyzers, so older versions of the CI/CD template will continue to work. - -If you changed the default GitLab SAST configuration, you may need to update your configuration as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#actions-required). - -## Removed in 15.3 - -### Support for Debian 9 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -Long term service and support (LTSS) for [Debian 9 Stretch ended in July 2022](https://wiki.debian.org/LTS). Therefore, we will no longer support the Debian 9 distribution for the GitLab package. Users can upgrade to Debian 10 or Debian 11. - -### Vulnerability Report sort by State - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.0</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/360516). -</div> - -The ability to sort the Vulnerability Report by the `State` column was disabled and put behind a feature flag in GitLab 14.10 due to a refactor -of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting -by this value remains performant. Due to very low usage of the `State` column for sorting, the feature flag is instead removed in 15.3 to simplify the codebase and prevent any unwanted performance degradation. - -### Vulnerability Report sort by Tool - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">15.1</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/363138). -</div> - -The ability to sort the Vulnerability Report by the `Tool` column (scan type) was disabled and put behind a feature flag in GitLab 14.10 due to a refactor -of the underlying data model. The feature flag has remained off by default as further refactoring will be required to ensure sorting -by this value remains performant. Due to very low usage of the `Tool` column for sorting, the feature flag is instead removed in -GitLab 15.3 to simplify the codebase and prevent any unwanted performance degradation. - -## Removed in 15.2 - -### Support for older browsers - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -In GitLab 15.2, we are cleaning up and [removing old code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86003) that was specific for browsers that we no longer support. This has no impact on users if they use one of our [supported web browsers](https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers). - -Most notably, support for the following browsers has been removed: - -- Apple Safari 14 and older. -- Mozilla Firefox 78. - -The minimum supported browser versions are: - -- Apple Safari 14.1. -- Mozilla Firefox 91. -- Google Chrome 92. -- Chromium 92. -- Microsoft Edge 92. - -## Removed in 15.0 - -### API: `stale` status returned instead of `offline` or `not_connected` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347303). -</div> - -The Runner [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints have changed in 15.0. - -If a runner has not contacted the GitLab instance in more than three months, the API returns `stale` instead of `offline` or `not_connected`. -The `stale` status was introduced in 14.6. - -The `not_connected` status is no longer valid. It was replaced with `never_contacted`. Available statuses are `online`, `offline`, `stale`, and `never_contacted`. - -### Audit events for repository push events - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/337993). -</div> - -Audit events for [repository events](https://docs.gitlab.com/ee/administration/audit_events.html#removed-events) are removed as of GitLab 15.0. - -Audit events for repository events were always disabled by default and had to be manually enabled with a feature flag. -Enabling them could slow down GitLab instances by generating too many events. Therefore, they are removed. - -Please note that we will add high-volume audit events in the future as part of [streaming audit events](https://docs.gitlab.com/ee/administration/audit_event_streaming.html). An example of this is how we will send [Git fetch actions](https://gitlab.com/gitlab-org/gitlab/-/issues/343984) as a streaming audit event. If you would be interested in seeing repository push events or some other action as a streaming audit event, please reach out to us! - -### Background upload for object storage - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/26600). -</div> - -To reduce the overall complexity and maintenance burden of GitLab's [object storage feature](https://docs.gitlab.com/ee/administration/object_storage.html), support for using `background_upload` has been removed in GitLab 15.0. -By default [direct upload](https://docs.gitlab.com/ee/development/uploads/index.html#direct-upload) will be used. - -This impacts a subset of object storage providers, including but not limited to: - -- **OpenStack** Customers using OpenStack need to change their configuration to use the S3 API instead of Swift. -- **RackSpace** Customers using RackSpace-based object storage need to migrate data to a different provider. - -If your object storage provider does not support `background_upload`, please [migrate objects to a supported object storage provider](https://docs.gitlab.com/ee/administration/object_storage.html#migrate-objects-to-a-different-object-storage-provider). - -#### Encrypted S3 buckets - -Additionally, this also breaks the use of [encrypted S3 buckets](https://docs.gitlab.com/ee/administration/object_storage.html#encrypted-s3-buckets) with [storage-specific configuration form](https://docs.gitlab.com/ee/administration/object_storage.html#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form). - -If your S3 buckets have [SSE-S3 or SSE-KMS encryption enabled](https://docs.aws.amazon.com/kms/latest/developerguide/services-s3.html), please [migrate your configuration to use consolidated object storage form](https://docs.gitlab.com/ee/administration/object_storage.html#transition-to-consolidated-form) before upgrading to GitLab 15.0. Otherwise, you may start getting `ETag mismatch` errors during objects upload. - -#### 403 errors - -If you see 403 errors when uploading to object storage after -upgrading to GitLab 15.0, check that the [correct permissions](https://docs.gitlab.com/ee/administration/object_storage.html#iam-permissions) -are assigned to the bucket. Direct upload needs the ability to delete an -object (example: `s3:DeleteObject`), but background uploads do not. - -#### `remote_directory` with a path prefix - -If the object storage `remote_directory` configuration contains a slash (`/`) after the bucket (example: `gitlab/uploads`), be aware that this [was never officially supported](https://gitlab.com/gitlab-org/gitlab/-/issues/292958). -Some users found that they could specify a path prefix to the bucket. In direct upload mode, object storage uploads will fail if a slash is present in GitLab 15.0. - -If you have set a prefix, you can use a workaround to revert to background uploads: - -1. Continue to use [storage-specific configuration](https://docs.gitlab.com/ee/administration/object_storage.html#configure-each-object-type-to-define-its-own-storage-connection-storage-specific-form). -1. In Omnibus GitLab, set the `GITLAB_LEGACY_BACKGROUND_UPLOADS` to re-enable background uploads: - - ```ruby - gitlab_rails['env'] = { 'GITLAB_LEGACY_BACKGROUND_UPLOADS' => 'artifacts,external_diffs,lfs,uploads,packages,dependency_proxy,terraform_state,pages' } - ``` - -Support for prefixes was restored in GitLab 15.2 via [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/91307). -Support for setting `GITLAB_LEGACY_BACKGROUND_UPLOADS` will be removed in GitLab 15.4. - -### Container Network and Host Security - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7477). -</div> - -All functionality related to the Container Network Security and Container Host Security categories was deprecated in GitLab 14.8 and is scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies with GitLab, add the desired Helm charts in your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through GitLab [CI/CD](https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html). - -As part of this change, the following capabilities within GitLab are scheduled for removal in GitLab 15.0: - -- The **Security & Compliance > Threat Monitoring** page. -- The Network Policy security policy type, as found on the **Security & Compliance > Policies** page. -- The ability to manage integrations with the following technologies through GitLab: AppArmor, Cilium, Falco, FluentD, and Pod Security Policies. -- All APIs related to the above functionality. - -For additional context, or to provide feedback regarding this change, please reference our [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7476). - -### Container registry authentication with htpasswd - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The Container Registry supports [authentication](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#auth) with `htpasswd`. It relies on an [Apache `htpasswd` file](https://httpd.apache.org/docs/2.4/programs/htpasswd.html), with passwords hashed using `bcrypt`. - -Since it isn't used in the context of GitLab (the product), `htpasswd` authentication will be deprecated in GitLab 14.9 and removed in GitLab 15.0. - -### Custom `geo:db:*` Rake tasks are no longer available - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351945). -</div> - -In GitLab 14.8, we [deprecated the `geo:db:*` Rake tasks and replaced them with built-in tasks](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77269/diffs) after [switching the Geo tracking database to use Rails' 6 support of multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6458). -The following `geo:db:*` tasks have been removed from GitLab 15.0 and have been replaced with their corresponding `db:*:geo` tasks: - -- `geo:db:drop` -> `db:drop:geo` -- `geo:db:create` -> `db:create:geo` -- `geo:db:setup` -> `db:setup:geo` -- `geo:db:migrate` -> `db:migrate:geo` -- `geo:db:rollback` -> `db:rollback:geo` -- `geo:db:version` -> `db:version:geo` -- `geo:db:reset` -> `db:reset:geo` -- `geo:db:seed` -> `db:seed:geo` -- `geo:schema:load:geo` -> `db:schema:load:geo` -- `geo:db:schema:dump` -> `db:schema:dump:geo` -- `geo:db:migrate:up` -> `db:migrate:up:geo` -- `geo:db:migrate:down` -> `db:migrate:down:geo` -- `geo:db:migrate:redo` -> `db:migrate:redo:geo` -- `geo:db:migrate:status` -> `db:migrate:status:geo` -- `geo:db:test:prepare` -> `db:test:prepare:geo` -- `geo:db:test:load` -> `db:test:load:geo` -- `geo:db:test:purge` -> `db:test:purge:geo` - -### DS_DEFAULT_ANALYZERS environment variable - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333299). -</div> - -We are removing the `DS_DEFAULT_ANALYZERS` environment variable from Dependency Scanning on May 22, 2022 in 15.0. After this removal, this variable's value will be ignored. To configure which analyzers to run with the default configuration, you should use the `DS_EXCLUDED_ANALYZERS` variable instead. - -### Dependency Scanning default Java version changed to 17 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -For Dependency Scanning, the default version of Java that the scanner expects will be updated from 11 to 17. Java 17 is [the most up-to-date Long Term Support (LTS) version](https://en.wikipedia.org/wiki/Java_version_history). Dependency Scanning continues to support the same [range of versions (8, 11, 13, 14, 15, 16, 17)](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#supported-languages-and-package-managers), only the default version is changing. If your project uses the previous default of Java 11, be sure to [set the `DS_JAVA_VERSION` variable to match](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#configuring-specific-analyzers-used-by-dependency-scanning). Please note that consequently the default version of Gradle is now 7.3.3. - -### ELK stack logging - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346485). -</div> - -The logging features in GitLab allow users to install the ELK stack (Elasticsearch, Logstash, and Kibana) to aggregate and manage application logs. Users could search for relevant logs in GitLab directly. However, since deprecating certificate-based integration with Kubernetes clusters and GitLab Managed Apps, this feature is no longer available. For more information on the future of logging and observability, you can follow the issue for [integrating Opstrace with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). - -### Elasticsearch 6.8.x in GitLab 15.0 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/350275). -</div> - -Elasticsearch 6.8 support has been removed in GitLab 15.0. Elasticsearch 6.8 has reached [end of life](https://www.elastic.co/support/eol). -If you use Elasticsearch 6.8, **you must upgrade your Elasticsearch version to 7.x** prior to upgrading to GitLab 15.0. -You should not upgrade to Elasticsearch 8 until you have completed the GitLab 15.0 upgrade. - -View the [version requirements](https://docs.gitlab.com/ee/integration/advanced_search/elasticsearch.html#version-requirements) for details. - -### End of support for Python 3.6 in Dependency Scanning - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351503). -</div> - -For those using Dependency Scanning for Python projects, we are removing support for the default `gemnasium-python:2` image which uses Python 3.6, as well as the custom `gemnasium-python:2-python-3.9` image which uses Python 3.9. The new default image as of GitLab 15.0 will be for Python 3.9 as it is a [supported version](https://endoflife.date/python) and 3.6 [is no longer supported](https://endoflife.date/python). - -### External status check API breaking changes - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/339039). -</div> - -The [external status check API](https://docs.gitlab.com/ee/api/status_checks.html) was originally implemented to -support pass-by-default requests to mark a status check as passing. Pass-by-default requests are now removed. -Specifically, the following are removed: - -- Requests that do not contain the `status` field. -- Requests that have the `status` field set to `approved`. - -From GitLab 15.0, status checks are only set to a passing state if the `status` field is both present -and set to `passed`. Requests that: - -- Do not contain the `status` field will be rejected with a `400` error. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338827). -- Contain any value other than `passed`, such as `approved`, cause the status check to fail. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/339039). - -To align with this change, API calls to list external status checks also return the value of `passed` rather than -`approved` for status checks that have passed. - -### GitLab Serverless - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/6). -</div> - -All functionality related to GitLab Serverless was deprecated in GitLab 14.3 and is scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to explore using the following technologies with GitLab CI/CD: - -- [Serverless Framework](https://www.serverless.com) -- [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/deploying-using-gitlab.html) - -For additional context, or to provide feedback regarding this change, please reference our [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/6). - -### Gitaly nodes in virtual storage - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">13.12</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Configuring the Gitaly nodes directly in the virtual storage's root configuration object has been deprecated in GitLab 13.12 and is no longer supported in GitLab 15.0. You must move the Gitaly nodes under the `'nodes'` key as described in [the Praefect configuration](https://docs.gitlab.com/ee/administration/gitaly/praefect.html#praefect). - -### GraphQL permissions change for Package settings - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The GitLab Package stage offers a Package Registry, Container Registry, and Dependency Proxy to help you manage all of your dependencies using GitLab. Each of these product categories has a variety of settings that can be adjusted using the API. - -The permissions model for GraphQL is being updated. After 15.0, users with the Guest, Reporter, and Developer role can no longer update these settings: - -- [Package Registry settings](https://docs.gitlab.com/ee/api/graphql/reference/#packagesettings) -- [Container Registry cleanup policy](https://docs.gitlab.com/ee/api/graphql/reference/#containerexpirationpolicy) -- [Dependency Proxy time-to-live policy](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxyimagettlgrouppolicy) -- [Enabling the Dependency Proxy for your group](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxysetting) - -The issue for this removal is [GitLab-#350682](https://gitlab.com/gitlab-org/gitlab/-/issues/350682) - -### Jaeger integration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/346540). -</div> - -Tracing in GitLab is an integration with Jaeger, an open-source end-to-end distributed tracing system. GitLab users could previously navigate to their Jaeger instance to gain insight into the performance of a deployed application, tracking each function or microservice that handles a given request. Tracing in GitLab was deprecated in GitLab 14.7, and removed in 15.0. To track work on a possible replacement, see the issue for [Opstrace integration with GitLab](https://gitlab.com/groups/gitlab-org/-/epics/6976). - -### Known host required for GitLab Runner SSH executor - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28192). -</div> - -In [GitLab 14.3](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3074), we added a configuration setting in the GitLab Runner `config.toml`. This setting, [`[runners.ssh.disable_strict_host_key_checking]`](https://docs.gitlab.com/runner/executors/ssh.html#security), controls whether or not to use strict host key checking with the SSH executor. - -In GitLab 15.0, the default value for this configuration option has changed from `true` to `false`. This means that strict host key checking will be enforced when using the GitLab Runner SSH executor. - -### Legacy Geo Admin UI routes - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351345). -</div> - -In GitLab 13.0, we introduced new project and design replication details routes in the Geo Admin UI. These routes are `/admin/geo/replication/projects` and `/admin/geo/replication/designs`. We kept the legacy routes and redirected them to the new routes. These legacy routes `/admin/geo/projects` and `/admin/geo/designs` have been removed in GitLab 15.0. Please update any bookmarks or scripts that may use the legacy routes. - -### Legacy approval status names in License Compliance API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.6</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345839). -</div> - -We have now removed the deprecated legacy names for approval status of license policy (`blacklisted`, `approved`) in the API queries and responses. If you are using our License Compliance API you should stop using the `approved` and `blacklisted` query parameters, they are now `allowed` and `denied`. In 15.0 the responses will also stop using `approved` and `blacklisted` so you may need to adjust any of your custom tools. - -### Move Gitaly Cluster Praefect `database_host_no_proxy` and `database_port_no_proxy configs` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6150). -</div> - -The Gitaly Cluster configuration keys for `praefect['database_host_no_proxy']` and `praefect['database_port_no_proxy']` are replaced with `praefect['database_direct_host']` and `praefect['database_direct_port']`. - -### Move `custom_hooks_dir` setting from GitLab Shell to Gitaly - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4208). -</div> - -The [`custom_hooks_dir`](https://docs.gitlab.com/ee/administration/server_hooks.html#create-a-global-server-hook-for-all-repositories) setting is now configured in Gitaly, and is removed from GitLab Shell in GitLab 15.0. - -### OAuth implicit grant - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The OAuth implicit grant authorization flow is no longer supported. Any applications that use OAuth implicit grant must switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). - -### OAuth tokens without an expiration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab no longer supports OAuth tokens [without an expiration](https://docs.gitlab.com/ee/integration/oauth_provider.html#expiring-access-tokens). - -Any existing token without an expiration has one automatically generated and applied. - -### Optional enforcement of SSH expiration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351963). -</div> - -Disabling SSH expiration enforcement is unusual from a security perspective and could create unusual situations where an expired -key is unintentionally able to be used. Unexpected behavior in a security feature is inherently dangerous and so now we enforce -expiration on all SSH keys. - -### Optional enforcement of personal access token expiration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351962). -</div> - -Allowing expired personal access tokens to be used is unusual from a security perspective and could create unusual situations where an -expired key is unintentionally able to be used. Unexpected behavior in a security feature is inherently dangerous and so we now do not let expired personal access tokens be used. - -### Out-of-the-box SAST (SpotBugs) support for Java 8 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352549). -</div> - -The [GitLab SAST SpotBugs analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) scans [Java, Scala, Groovy, and Kotlin code](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) for security vulnerabilities. -For technical reasons, the analyzer must first compile the code before scanning. -Unless you use the [pre-compilation strategy](https://docs.gitlab.com/ee/user/application_security/sast/#pre-compilation), the analyzer attempts to automatically compile your project's code. - -In GitLab versions prior to 15.0, the analyzer image included Java 8 and Java 11 runtimes to facilitate compilation. - -As of GitLab 15.0, we've: - -- Removed Java 8 from the analyzer image to reduce the size of the image. -- Added Java 17 to the analyzer image to make it easier to compile with Java 17. -- Changed the default Java version from Java 8 to Java 17. - -If you rely on Java 8 being present in the analyzer environment, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352549#breaking-change). - -### Pipelines field from the version field - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342882). -</div> - -In GraphQL, there are two `pipelines` fields that you can use in a [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/#packagedetailstype) to get the pipelines for package versions: - -- The `versions` field's `pipelines` field. This returns all the pipelines associated with all the package's versions, which can pull an unbounded number of objects in memory and create performance concerns. -- The `pipelines` field of a specific `version`. This returns only the pipelines associated with that single package version. - -To mitigate possible performance problems, we will remove the `versions` field's `pipelines` field in GitLab 15.0. Although you will no longer be able to get all pipelines for all versions of a package, you can still get the pipelines of a single version through the remaining `pipelines` field for that version. - -### Pseudonymizer - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/219952). -</div> - -The Pseudonymizer feature is generally unused, can cause production issues with large databases, and can interfere with object storage development. -It was removed in GitLab 15.0. - -### Request profiling - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352488). -</div> - -[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/index.html) has been removed in GitLab 15.0. - -We're working on [consolidating our profiling tools](https://gitlab.com/groups/gitlab-org/-/epics/7327) and making them more easily accessible. -We [evaluated](https://gitlab.com/gitlab-org/gitlab/-/issues/350152) the use of this feature and we found that it is not widely used. -It also depends on a few third-party gems that are not actively maintained anymore, have not been updated for the latest version of Ruby, or crash frequently when profiling heavy page loads. - -For more information, check the [summary section of the deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352488#deprecation-summary). - -### Required pipeline configurations in Premium tier - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -[Required pipeline configuration](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#required-pipeline-configuration) helps to define and mandate organization-wide pipeline configurations and is a requirement at an executive and organizational level. To align better with our [pricing philosophy](https://about.gitlab.com/company/pricing/#three-tiers), this feature is removed from the Premium tier in GitLab 15.0. This feature continues to be available in the GitLab Ultimate tier. - -We recommend customers use [Compliance Pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration), also in GitLab Ultimate, as an alternative as it provides greater flexibility, allowing required pipelines to be assigned to specific compliance framework labels. - -This change also helps GitLab remain consistent in our tiering strategy with the other related Ultimate-tier features: - -- [Security policies](https://docs.gitlab.com/ee/user/application_security/policies/). -- [Compliance framework pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration). - -### Retire-JS Dependency Scanning tool - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/289830). -</div> - -We have removed support for retire.js from Dependency Scanning as of May 22, 2022 in GitLab 15.0. JavaScript scanning functionality will not be affected as it is still being covered by Gemnasium. - -If you have explicitly excluded retire.js using the `DS_EXCLUDED_ANALYZERS` variable, then you will be able to remove the reference to retire.js. If you have customized your pipeline’s Dependency Scanning configuration related to the `retire-js-dependency_scanning` job, then you will want to switch to `gemnasium-dependency_scanning`. If you have not used the `DS_EXCLUDED_ANALYZERS` to reference retire.js, or customized your template specifically for retire.js, you will not need to take any action. - -### Runner status `not_connected` API value - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347305). -</div> - -The GitLab Runner REST and GraphQL [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints -deprecated the `not_connected` status value in GitLab 14.6 and will start returning `never_contacted` in its place -starting in GitLab 15.0. - -Runners that have never contacted the GitLab instance will also return `stale` if created more than 3 months ago. - -### SAST support for .NET 2.1 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352553). -</div> - -The [GitLab SAST Security Code Scan analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan) scans .NET code for security vulnerabilities. -For technical reasons, the analyzer must first build the code to scan it. - -In GitLab versions prior to 15.0, the default analyzer image (version 2) included support for: - -- .NET 2.1 -- .NET Core 3.1 -- .NET 5.0 - -In GitLab 15.0, we've changed the default major version for this analyzer from version 2 to version 3. This change: - -- Adds [severity values for vulnerabilities](https://gitlab.com/gitlab-org/gitlab/-/issues/350408) along with [other new features and improvements](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/blob/master/CHANGELOG.md). -- Removes .NET 2.1 support. -- Adds support for .NET 6.0, Visual Studio 2019, and Visual Studio 2022. - -Version 3 was [announced in GitLab 14.6](https://about.gitlab.com/releases/2021/12/22/gitlab-14-6-released/#sast-support-for-net-6) and made available as an optional upgrade. - -If you rely on .NET 2.1 support being present in the analyzer image by default, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352553#breaking-change). - -### SUSE Linux Enterprise Server 12 SP2 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. - -### Secret Detection configuration variables - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352565). -</div> - -To make it simpler and more reliable to [customize GitLab Secret Detection](https://docs.gitlab.com/ee/user/application_security/secret_detection/#customizing-settings), we've removed some of the variables that you could previously set in your CI/CD configuration. - -The following variables previously allowed you to customize the options for historical scanning, but interacted poorly with the [GitLab-managed CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) and are now removed: - -- `SECRET_DETECTION_COMMIT_FROM` -- `SECRET_DETECTION_COMMIT_TO` -- `SECRET_DETECTION_COMMITS` -- `SECRET_DETECTION_COMMITS_FILE` - -The `SECRET_DETECTION_ENTROPY_LEVEL` previously allowed you to configure rules that only considered the entropy level of strings in your codebase, and is now removed. -This type of entropy-only rule created an unacceptable number of incorrect results (false positives). - -You can still customize the behavior of the Secret Detection analyzer using the [available CI/CD variables](https://docs.gitlab.com/ee/user/application_security/secret_detection/#available-cicd-variables). - -For further details, see [the deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352565). - -### Self-managed certificate-based integration with Kubernetes feature flagged - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). -</div> - -In 15.0 the certificate-based integration with Kubernetes will be disabled by default. - -After 15.0, you should use the [agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. The agent for Kubernetes is a more robust, secure, and reliable integration with Kubernetes. [How do I migrate to the agent?](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) - -If you need more time to migrate, you can enable the `certificate_based_clusters` [feature flag](https://docs.gitlab.com/ee/administration/feature_flags.html), which re-enables the certificate-based integration. - -In GitLab 17.0, we will [remove the feature, its related code, and the feature flag](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). GitLab will continue to fix any security or critical issues until 17.0. - -For updates and details, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). - -### Sidekiq configuration for metrics and health checks - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347509). -</div> - -In GitLab 15.0, you can no longer serve Sidekiq metrics and health checks over a single address and port. - -To improve stability, availability, and prevent data loss in edge cases, GitLab now serves -[Sidekiq metrics and health checks from two separate servers](https://gitlab.com/groups/gitlab-org/-/epics/6409). - -When you use Omnibus or Helm charts, if GitLab is configured for both servers to bind to the same address, -a configuration error occurs. -To prevent this error, choose different ports for the metrics and health check servers: - -- [Configure Sidekiq health checks](https://docs.gitlab.com/ee/administration/sidekiq.html#configure-health-checks) -- [Configure the Sidekiq metrics server](https://docs.gitlab.com/ee/administration/sidekiq.html#configure-the-sidekiq-metrics-server) - -If you installed GitLab from source, verify manually that both servers are configured to bind to separate addresses and ports. - -### Static Site Editor - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347137). -</div> - -The Static Site Editor was deprecated in GitLab 14.7 and the feature is being removed in GitLab 15.0. Incoming requests to the Static Site Editor will be redirected and open the target file to edit in the Web IDE. Current users of the Static Site Editor can view the [documentation](https://docs.gitlab.com/ee/user/project/web_ide/index.html) for more information, including how to remove the configuration files from existing projects. We will continue investing in improvements to the Markdown editing experience by [maturing the Content Editor](https://gitlab.com/groups/gitlab-org/-/epics/5401) and making it available as a way to edit content across GitLab. - -### Support for `gitaly['internal_socket_dir']` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.10</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6758). -</div> - -Gitaly introduced a new directory that holds all runtime data Gitaly requires to operate correctly. This new directory replaces the old internal socket directory, and consequentially the usage of `gitaly['internal_socket_dir']` was deprecated in favor of `gitaly['runtime_dir']`. - -### Support for legacy format of `config/database.yml` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338182). -</div> - -The syntax of [GitLab's database](https://docs.gitlab.com/omnibus/settings/database.html) -configuration located in `database.yml` has changed and the legacy format has been removed. -The legacy format supported a single PostgreSQL adapter, whereas the new format supports multiple databases. -The `main:` database needs to be defined as a first configuration item. - -This change only impacts users compiling GitLab from source, all the other installation methods handle this configuration automatically. -Instructions are available [in the source update documentation](https://docs.gitlab.com/ee/update/upgrading_from_source.html#new-configuration-options-for-databaseyml). - -### Test coverage project CI/CD setting - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -To specify a test coverage pattern, in GitLab 15.0 the -[project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-to-a-merge-request-removed) -has been removed. - -To set test coverage parsing, use the project’s `.gitlab-ci.yml` file by providing a regular expression with the -[`coverage` keyword](https://docs.gitlab.com/ee/ci/yaml/index.html#coverage). - -### The `promote-db` command is no longer available from `gitlab-ctl` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). -</div> - -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-db` which is used to promote database nodes in multi-node Geo secondary sites. The `gitlab-ctl promote-db` command has been removed in GitLab 15.0. - -### Update to the Container Registry group-level API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336912). -</div> - -In GitLab 15.0, support for the `tags` and `tags_count` parameters will be removed from the Container Registry API that [gets registry repositories from a group](../api/container_registry.md#within-a-group). - -The `GET /groups/:id/registry/repositories` endpoint will remain, but won't return any info about tags. To get the info about tags, you can use the existing `GET /registry/repositories/:id` endpoint, which will continue to support the `tags` and `tag_count` options as it does today. The latter must be called once per image repository. - -### Versions from `PackageType` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/327453). -</div> - -As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `Version` type for the basic `PackageType` type and moved it to [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype). - -In GitLab 15.0, we will completely remove `Version` from `PackageType`. - -### Vulnerability Check - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/357300). -</div> - -The vulnerability check feature was deprecated in GitLab 14.8 and is scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. - -The new security approvals feature is similar to vulnerability check. For example, both can require approvals for MRs that contain security vulnerabilities. However, security approvals improve the previous experience in several ways: - -- Users can choose who is allowed to edit security approval rules. An independent security or compliance team can therefore manage rules in a way that prevents development project maintainers from modifying the rules. -- Multiple rules can be created and chained together to allow for filtering on different severity thresholds for each scanner type. -- A two-step approval process can be enforced for any desired changes to security approval rules. -- A single set of security policies can be applied to multiple development projects to allow for ease in maintaining a single, centralized ruleset. - -### `Managed-Cluster-Applications.gitlab-ci.yml` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.0</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/333610). -</div> - -The `Managed-Cluster-Applications.gitlab-ci.yml` CI/CD template is being removed. If you need an alternative, try the [Cluster Management project template](https://gitlab.com/gitlab-org/gitlab/-/issues/333610) instead. If your are not ready to move, you can copy the [last released version](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/v14.10.1/lib/gitlab/ci/templates/Managed-Cluster-Applications.gitlab-ci.yml) of the template into your project. - -### `artifacts:reports:cobertura` keyword - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.7</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/348980). -</div> - -As of GitLab 15.0, the [`artifacts:reports:cobertura`](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscobertura-removed) -keyword has been [replaced](https://gitlab.com/gitlab-org/gitlab/-/issues/344533) by -[`artifacts:reports:coverage_report`](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscoverage_report). -Cobertura is the only supported report file, but this is the first step towards GitLab supporting other report types. - -### `defaultMergeCommitMessageWithDescription` GraphQL API field - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345451). -</div> - -The GraphQL API field `defaultMergeCommitMessageWithDescription` has been removed in GitLab 15.0. For projects with a commit message template set, it will ignore the template. - -### `dependency_proxy_for_private_groups` feature flag - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/276777). -</div> - -A feature flag was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) in GitLab 13.7 as part of the change to require authentication to use the Dependency Proxy. Before GitLab 13.7, you could use the Dependency Proxy without authentication. - -In GitLab 15.0, we will remove the feature flag, and you must always authenticate when you use the Dependency Proxy. - -### `omniauth-kerberos` gem - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.3</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The `omniauth-kerberos` gem is no longer supported. This gem has not been maintained and has very little usage. Therefore, we -removed support for this authentication method and recommend using [SPEGNO](https://en.wikipedia.org/wiki/SPNEGO) instead. You can -follow the [upgrade instructions](https://docs.gitlab.com/ee/integration/kerberos.html#upgrading-from-password-based-to-ticket-based-kerberos-sign-ins) -to upgrade from the removed integration to the new supported one. - -We are not removing Kerberos SPNEGO integration. We are removing the old password-based Kerberos. - -### `promote-to-primary-node` command from `gitlab-ctl` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.5</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). -</div> - -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-to-primary-node` which was only usable for single-node Geo sites. `gitlab-ctl promote-to-primary-node` has been removed in GitLab 15.0. - -### `push_rules_supersede_code_owners` feature flag - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/262019). -</div> - -The `push_rules_supersede_code_owners` feature flag has been removed in GitLab 15.0. From now on, push rules will supersede the `CODEOWNERS` file. Even if Code Owner approval is required, a push rule that explicitly allows a specific user to push code supersedes the Code Owners setting. - -### `type` and `types` keyword from CI/CD configuration - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.6</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The `type` and `types` CI/CD keywords is removed in GitLab 15.0, so pipelines that use these keywords fail with a syntax error. Switch to `stage` and `stages`, which have the same behavior. - -### bundler-audit Dependency Scanning tool - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.8</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/347491). -</div> - -We are removing bundler-audit from Dependency Scanning on May 22, 2022 in 15.0. After this removal, Ruby scanning functionality will not be affected as it is still being covered by Gemnasium. - -If you have explicitly excluded bundler-audit using the `DS_EXCLUDED_ANALYZERS` variable, then you will be able to remove the reference to bundler-audit. If you have customized your pipeline’s Dependency Scanning configuration related to the `bundler-audit-dependency_scanning` job, then you will want to switch to `gemnasium-dependency_scanning`. If you have not used the `DS_EXCLUDED_ANALYZERS` to reference bundler-audit or customized your template specifically for bundler-audit, you will not need to take any action. - -## Removed in 14.10 - -### Permissions change for downloading Composer dependencies - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The GitLab Composer repository can be used to push, search, fetch metadata about, and download PHP dependencies. All these actions require authentication, except for downloading dependencies. - -Downloading Composer dependencies without authentication is deprecated in GitLab 14.9, and will be removed in GitLab 15.0. Starting with GitLab 15.0, you must authenticate to download Composer dependencies. - -## Removed in 14.9 - -### Integrated error tracking disabled by default - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone">14.9</span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/353639). -</div> - -In GitLab 14.4, GitLab released an integrated error tracking backend that replaces Sentry. This feature caused database performance issues. In GitLab 14.9, integrated error tracking is removed from GitLab.com, and turned off by default in GitLab self-managed. While we explore the future development of this feature, please consider switching to the Sentry backend by [changing your error tracking to Sentry in your project settings](https://docs.gitlab.com/ee/operations/error_tracking.html#sentry-error-tracking). - -For additional background on this removal, please reference [Disable Integrated Error Tracking by Default](https://gitlab.com/groups/gitlab-org/-/epics/7580). If you have feedback please add a comment to [Feedback: Removal of Integrated Error Tracking](https://gitlab.com/gitlab-org/gitlab/-/issues/355493). - -## Removed in 14.6 - -### Limit the number of triggered pipeline to 25K in free tier - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -A large amount of triggered pipelines in a single project impacts the performance of GitLab.com. In GitLab 14.6, we are limiting the number of triggered pipelines in a single project on GitLab.com at any given moment to 25,000. This change applies to projects in the free tier only, Premium and Ultimate are not affected by this change. - -### Release CLI distributed as a generic package - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. - -## Removed in 14.3 - -### Introduced limit of 50 tags for jobs - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -GitLab values efficiency and is prioritizing reliability for [GitLab.com in FY22](https://about.gitlab.com/direction/#gitlab-hosted-first). In 14.3, GitLab CI/CD jobs must have less than 50 [tags](https://docs.gitlab.com/ee/ci/yaml/index.html#tags). If a pipeline contains a job with 50 or more tags, you will receive an error and the pipeline will not be created. - -### List project pipelines API endpoint removes `name` support in 14.3 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -In GitLab 14.3, we will remove the ability to filter by `name` in the [list project pipelines API endpoint](https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines) to improve performance. If you currently use this parameter with this endpoint, you must switch to `username`. - -### Use of legacy storage setting - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -The support for [`gitlab_pages['use_legacy_storage']` setting](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) in Omnibus installations has been removed. - -In 14.0 we removed [`domain_config_source`](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) which had been previously deprecated, and allowed users to specify disk storage. In 14.0 we added `use_legacy_storage` as a **temporary** flag to unblock upgrades, and allow us to debug issues with our users and it was deprecated and communicated for removal in 14.3. - -## Removed in 14.2 - -### Max job log file size of 100 MB - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -GitLab values efficiency for all users in our wider community of contributors, so we're always working hard to make sure the application performs at a high level with a lovable UX. - In GitLab 14.2, we have introduced a [job log file size limit](https://docs.gitlab.com/ee/administration/instance_limits.html#maximum-file-size-for-job-logs), set to 100 megabytes by default. Administrators of self-managed GitLab instances can customize this to any value. All jobs that exceed this limit are dropped and marked as failed, helping prevent performance impacts or over-use of resources. This ensures that everyone using GitLab has the best possible experience. - -## Removed in 14.1 - -### Remove support for `prometheus.listen_address` and `prometheus.enable` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -The support for `prometheus.listen_address` and `prometheus.enable` has been removed from `gitlab.yml`. Use `prometheus.enabled` and `prometheus.server_address` to set up Prometheus server that GitLab instance connects to. Refer to [our documentation](https://docs.gitlab.com/ee/install/installation.html#prometheus-server-setup) for details. - -This only affects new installations from source where users might use the old configurations. - -### Remove support for older browsers - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -</div> - -In GitLab 14.1, we are cleaning up and [removing old code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63994) that was specific for browsers that we no longer support. This has no impact on users when one of our [supported web browsers](https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers) is used. - -Most notably, support for the following browsers has been removed: - -- Apple Safari 13 and older. -- Mozilla Firefox 68. -- Pre-Chromium Microsoft Edge. - -The minimum supported browser versions are: - -- Apple Safari 13.1. -- Mozilla Firefox 78. -- Google Chrome 84. -- Chromium 84. -- Microsoft Edge 84. - -## Removed in 14.0 - -### Auto Deploy CI template v1 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/300862). -</div> - -In GitLab 14.0, we will update the [Auto Deploy](https://docs.gitlab.com/ee/topics/autodevops/stages.html#auto-deploy) CI template to the latest version. This includes new features, bug fixes, and performance improvements with a dependency on the v2 [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image). Auto Deploy CI template v1 is deprecated going forward. - -Since the v1 and v2 versions are not backward-compatible, your project might encounter an unexpected failure if you already have a deployed application. Follow the [upgrade guide](https://docs.gitlab.com/ee/topics/autodevops/upgrading_auto_deploy_dependencies.html#upgrade-guide) to upgrade your environments. You can also start using the latest template today by following the [early adoption guide](https://docs.gitlab.com/ee/topics/autodevops/upgrading_auto_deploy_dependencies.html#early-adopters). - -### Breaking changes to Terraform CI template - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/328500). -</div> - -GitLab 14.0 renews the Terraform CI template to the latest version. The new template is set up for the GitLab Managed Terraform state, with a dependency on the GitLab `terraform-images` image, to provide a good user experience around GitLab's Infrastructure-as-Code features. - -The current stable and latest templates are not compatible, and the current latest template becomes the stable template beginning with GitLab 14.0, your Terraform pipeline might encounter an unexpected failure if you run a custom `init` job. - -### Code Quality RuboCop support changed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -By default, the Code Quality feature has not provided support for Ruby 2.6+ if you're using the Code Quality template. To better support the latest versions of Ruby, the default RuboCop version is updated to add support for Ruby 2.4 through 3.0. As a result, support for Ruby 2.1, 2.2, and 2.3 is removed. You can re-enable support for older versions by [customizing your configuration](https://docs.gitlab.com/ee/ci/testing/code_quality.html#rubocop-errors). - -Relevant Issue: [Default `codeclimate-rubocop` engine does not support Ruby 2.6+](https://gitlab.com/gitlab-org/ci-cd/codequality/-/issues/28) - -### Container Scanning Engine Clair - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Clair, the default container scanning engine, was deprecated in GitLab 13.9 and is removed from GitLab 14.0 and replaced by Trivy. We advise customers who are customizing variables for their container scanning job to [follow these instructions](https://docs.gitlab.com/ee/user/application_security/container_scanning/#change-scanners) to ensure that their container scanning jobs continue to work. - -### DAST default template stages - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab 14.0, we've removed the stages defined in the current `DAST.gitlab-ci.yml` template to avoid the situation where the template overrides manual changes made by DAST users. We're making this change in response to customer issues where the stages in the template cause problems when used with customized DAST configurations. Because of this removal, `gitlab-ci.yml` configurations that do not specify a `dast` stage must be updated to include this stage. - -### DAST environment variable renaming and removal - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab 13.8 renamed multiple environment variables to support their broader usage in different workflows. In GitLab 14.0, the old variables have been permanently removed and will no longer work. Any configurations using these variables must be updated to the new variable names. Any scans using these variables in GitLab 14.0 and later will fail to be configured correctly. These variables are: - -- `DAST_AUTH_EXCLUDE_URLS` becomes `DAST_EXCLUDE_URLS`. -- `AUTH_EXCLUDE_URLS` becomes `DAST_EXCLUDE_URLS`. -- `AUTH_USERNAME` becomes `DAST_USERNAME`. -- `AUTH_PASSWORD` becomes `DAST_PASSWORD`. -- `AUTH_USERNAME_FIELD` becomes `DAST_USERNAME_FIELD`. -- `AUTH_PASSWORD_FIELD` becomes `DAST_PASSWORD_FIELD`. -- `DAST_ZAP_USE_AJAX_SPIDER` will now be `DAST_USE_AJAX_SPIDER`. -- `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` will be removed, since the feature is being removed. - -### Default Browser Performance testing job renamed in GitLab 14.0 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Browser Performance Testing has run in a job named `performance` by default. With the introduction of [Load Performance Testing](https://docs.gitlab.com/ee/ci/testing/code_quality.html) in GitLab 13.2, this naming could be confusing. To make it clear which job is running [Browser Performance Testing](https://docs.gitlab.com/ee/ci/testing/browser_performance_testing.html), the default job name is changed from `performance` to `browser_performance` in the template in GitLab 14.0. - -Relevant Issue: [Rename default Browser Performance Testing job](https://gitlab.com/gitlab-org/gitlab/-/issues/225914) - -### Default DAST spider begins crawling at target URL - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab 14.0, DAST has removed the current method of resetting the scan to the hostname when starting to spider. Prior to GitLab 14.0, the spider would not begin at the specified target path for the URL but would instead reset the URL to begin crawling at the host root. GitLab 14.0 changes the default for the new variable `DAST_SPIDER_START_AT_HOST` to `false` to better support users' intention of beginning spidering and scanning at the specified target URL, rather than the host root URL. This change has an added benefit: scans can take less time, if the specified path does not contain links to the entire site. This enables easier scanning of smaller sections of an application, rather than crawling the entire app during every scan. - -### Default branch name for new repositories now `main` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Every Git repository has an initial branch, which is named `master` by default. It's the first branch to be created automatically when you create a new repository. Future [Git versions](https://lore.kernel.org/git/pull.656.v4.git.1593009996.gitgitgadget@gmail.com/) will change the default branch name in Git from `master` to `main`. In coordination with the Git project and the broader community, [GitLab has changed the default branch name](https://gitlab.com/gitlab-org/gitlab/-/issues/223789) for new projects on both our SaaS (GitLab.com) and self-managed offerings starting with GitLab 14.0. This will not affect existing projects. - -GitLab has already introduced changes that allow you to change the default branch name both at the [instance level](https://docs.gitlab.com/ee/user/project/repository/branches/default.html#instance-level-custom-initial-branch-name) (for self-managed users) and at the [group level](https://docs.gitlab.com/ee/user/group/#use-a-custom-name-for-the-initial-branch) (for both SaaS and self-managed users). We encourage you to make use of these features to set default branch names on new projects. - -For more information, check out our [blog post](https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/). - -### Dependency Scanning - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -As mentioned in [13.9](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#deprecations-for-dependency-scanning) and [this blog post](https://about.gitlab.com/blog/2021/02/08/composition-analysis-14-deprecations-and-removals/) several removals for Dependency Scanning take effect. - -Previously, to exclude a DS analyzer, you needed to remove it from the default list of analyzers, and use that to set the `DS_DEFAULT_ANALYZERS` variable in your project’s CI template. We determined it should be easier to avoid running a particular analyzer without losing the benefit of newly added analyzers. As a result, we ask you to migrate from `DS_DEFAULT_ANALYZERS` to `DS_EXCLUDED_ANALYZERS` when it is available. Read about it in [issue #287691](https://gitlab.com/gitlab-org/gitlab/-/issues/287691). - -Previously, to prevent the Gemnasium analyzers to fetch the advisory database at runtime, you needed to set the `GEMNASIUM_DB_UPDATE` variable. However, this is not documented properly, and its naming is inconsistent with the equivalent `BUNDLER_AUDIT_UPDATE_DISABLED` variable. As a result, we ask you to migrate from `GEMNASIUM_DB_UPDATE` to `GEMNASIUM_UPDATE_DISABLED` when it is available. Read about it in [issue #215483](https://gitlab.com/gitlab-org/gitlab/-/issues/215483). - -### Deprecated GraphQL fields - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In accordance with our [GraphQL deprecation and removal process](https://docs.gitlab.com/ee/api/graphql/#deprecation-process), the following fields that were deprecated prior to 13.7 are [fully removed in 14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/267966): - -- `Mutations::Todos::MarkAllDone`, `Mutations::Todos::RestoreMany` - `:updated_ids` -- `Mutations::DastScannerProfiles::Create`, `Types::DastScannerProfileType` - `:global_id` -- `Types::SnippetType` - `:blob` -- `EE::Types::GroupType`, `EE::Types::QueryType` - `:vulnerabilities_count_by_day_and_severity` -- `DeprecatedMutations (concern**)` - `AddAwardEmoji`, `RemoveAwardEmoji`, `ToggleAwardEmoji` -- `EE::Types::DeprecatedMutations (concern***)` - `Mutations::Pipelines::RunDastScan`, `Mutations::Vulnerabilities::Dismiss`, `Mutations::Vulnerabilities::RevertToDetected` - -### DevOps Adoption API Segments - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The first release of the DevOps Adoption report had a concept of **Segments**. Segments were [quickly removed from the report](https://gitlab.com/groups/gitlab-org/-/epics/5251) because they introduced an additional layer of complexity on top of **Groups** and **Projects**. Subsequent iterations of the DevOps Adoption report focus on comparing adoption across groups rather than segments. GitLab 14.0 removes all references to **Segments** [from the GraphQL API](https://gitlab.com/gitlab-org/gitlab/-/issues/324414) and replaces them with **Enabled groups**. - -### Disk source configuration for GitLab Pages - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5993). -</div> - -GitLab Pages [API-based configuration](https://docs.gitlab.com/ee/administration/pages/#gitlab-api-based-configuration) has been available since GitLab 13.0. It replaces the unsupported `disk` source configuration removed in GitLab 14.0, which can no longer be chosen. You should stop using `disk` source configuration, and move to `gitlab` for an API-based configuration. To migrate away from the 'disk' source configuration, set `gitlab_pages['domain_config_source'] = "gitlab"` in your `/etc/gitlab/gitlab.rb` file. We recommend you migrate before updating to GitLab 14.0, to identify and troubleshoot any potential problems before upgrading. - -### Experimental prefix in Sidekiq queue selector options - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab supports a [queue selector](https://docs.gitlab.com/ee/administration/operations/extra_sidekiq_processes.html#queue-selector) to run only a subset of background jobs for a given process. When it was introduced, this option had an 'experimental' prefix (`experimental_queue_selector` in Omnibus, `experimentalQueueSelector` in Helm charts). - -As announced in the [13.6 release post](https://about.gitlab.com/releases/2020/11/22/gitlab-13-6-released/#sidekiq-cluster-queue-selector-configuration-option-has-been-renamed), the 'experimental' prefix is no longer supported. Instead, `queue_selector` for Omnibus and `queueSelector` in Helm charts should be used. - -### External Pipeline Validation Service Code Changes - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -For self-managed instances using the experimental [external pipeline validation service](https://docs.gitlab.com/ee/administration/external_pipeline_validation.html), the range of error codes GitLab accepts will be reduced. Currently, pipelines are invalidated when the validation service returns a response code from `400` to `499`. In GitLab 14.0 and later, pipelines will be invalidated for the `406: Not Accepted` response code only. - -### Geo Foreign Data Wrapper settings - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -As [announced in GitLab 13.3](https://about.gitlab.com/releases/2020/08/22/gitlab-13-3-released/#geo-foreign-data-wrapper-settings-deprecated), the following configuration settings in `/etc/gitlab/gitlab.rb` have been removed in 14.0: - -- `geo_secondary['db_fdw']` -- `geo_postgresql['fdw_external_user']` -- `geo_postgresql['fdw_external_password']` -- `gitlab-_rails['geo_migrated_local_files_clean_up_worker_cron']` - -### GitLab OAuth implicit grant - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/288516). -</div> - -GitLab is deprecating the [OAuth 2 implicit grant flow](https://docs.gitlab.com/ee/api/oauth2.html#implicit-grant-flow) as it has been removed for [OAuth 2.1](https://oauth.net/2.1/). - -Migrate your existing applications to other supported [OAuth2 flows](https://docs.gitlab.com/ee/api/oauth2.html#supported-oauth2-flows). - -### GitLab Runner helper image in GitLab.com Container Registry - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In 14.0, we are now pulling the GitLab Runner [helper image](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#helper-image) from the GitLab Container Registry instead of Docker Hub. Refer to [issue #27218](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27218) for details. - -### GitLab Runner installation to ignore the `skel` directory - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab Runner 14.0, the installation process will ignore the `skel` directory by default when creating the user home directory. Refer to [issue #4845](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4845) for details. - -### Gitaly Cluster SQL primary elector - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Now that Praefect supports a [primary election strategy](https://docs.gitlab.com/ee/administration/gitaly/praefect.html#repository-specific-primary-nodes) for each repository, we have removed the `sql` election strategy. -The `per_repository` election strategy is the new default, which is automatically used if no election strategy was specified. - -If you had configured the `sql` election strategy, you must follow the [migration instructions](https://docs.gitlab.com/ee/administration/gitaly/praefect.html#migrate-to-repository-specific-primary-gitaly-nodes) before upgrading to 14.0. - -### Global `SAST_ANALYZER_IMAGE_TAG` in SAST CI template - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/301216). -</div> - -With the maturity of GitLab Secure scanning tools, we've needed to add more granularity to our release process. Previously, GitLab shared a major version number for [all analyzers and tools](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks). This requires all tools to share a major version, and prevents the use of [semantic version numbering](https://semver.org/). In GitLab 14.0, SAST removes the `SAST_ANALYZER_IMAGE_TAG` global variable in our [managed `SAST.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) CI template, in favor of the analyzer job variable setting the `major.minor` tag in the SAST vendored template. - -Each analyzer job now has a scoped `SAST_ANALYZER_IMAGE_TAG` variable, which will be actively managed by GitLab and set to the `major` tag for the respective analyzer. To pin to a specific version, [change the variable value to the specific version tag](https://docs.gitlab.com/ee/user/application_security/sast/#pinning-to-minor-image-version). -If you override or maintain custom versions of `SAST.gitlab-ci.yml`, update your CI templates to stop referencing the global `SAST_ANALYZER_IMAGE_TAG`, and move it to a scoped analyzer job tag. We strongly encourage [inheriting and overriding our managed CI templates](https://docs.gitlab.com/ee/user/application_security/sast/#overriding-sast-jobs) to future-proof your CI templates. This change allows you to more granularly control future analyzer updates with a pinned `major.minor` version. -This deprecation and removal changes our [previously announced plan](https://about.gitlab.com/releases/2021/02/22/gitlab-13-9-released/#pin-static-analysis-analyzers-and-tools-to-minor-versions) to pin the Static Analysis tools. - -### Hardcoded `master` in CI/CD templates - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Our CI/CD templates have been updated to no longer use hard-coded references to a `master` branch. In 14.0, they all use a variable that points to your project's configured default branch instead. If your CI/CD pipeline relies on our built-in templates, verify that this change works with your current configuration. For example, if you have a `master` branch and a different default branch, the updates to the templates may cause changes to your pipeline behavior. For more information, [read the issue](https://gitlab.com/gitlab-org/gitlab/-/issues/324131). - -### Helm v2 support - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Helm v2 was [officially deprecated](https://helm.sh/blog/helm-v2-deprecation-timeline/) in November of 2020, with the `stable` repository being [de-listed from the Helm Hub](https://about.gitlab.com/blog/2020/11/09/ensure-auto-devops-work-after-helm-stable-repo/) shortly thereafter. With the release of GitLab 14.0, which will include the 5.0 release of the [GitLab Helm chart](https://docs.gitlab.com/charts/), Helm v2 will no longer be supported. - -Users of the chart should [upgrade to Helm v3](https://helm.sh/docs/topics/v2_v3_migration/) to deploy GitLab 14.0 and later. - -### Legacy DAST domain validation - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The legacy method of DAST Domain Validation for CI/CD scans was deprecated in GitLab 13.8, and is removed in GitLab 14.0. This method of domain validation only disallows scans if the `DAST_FULL_SCAN_DOMAIN_VALIDATION_REQUIRED` environment variable is set to `true` in the `gitlab-ci.yml` file, and a `Gitlab-DAST-Permission` header on the site is not set to `allow`. This two-step method required users to opt in to using the variable before they could opt out from using the header. - -For more information, see the [removal issue](https://gitlab.com/gitlab-org/gitlab/-/issues/293595). - -### Legacy feature flags - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/254324). -</div> - -Legacy feature flags became read-only in GitLab 13.4. GitLab 14.0 removes support for legacy feature flags, so you must migrate them to the [new version](https://docs.gitlab.com/ee/operations/feature_flags.html). You can do this by first taking a note (screenshot) of the legacy flag, then deleting the flag through the API or UI (you don't need to alter the code), and finally create a new Feature Flag with the same name as the legacy flag you deleted. Also, make sure the strategies and environments match the deleted flag. We created a [video tutorial](https://www.youtube.com/watch?v=CAJY2IGep7Y) to help with this migration. - -### Legacy fields from DAST report - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -As a part of the migration to a common report format for all of the Secure scanners in GitLab, DAST is making changes to the DAST JSON report. Certain legacy fields were deprecated in 13.8 and have been completely removed in 14.0. These fields are `@generated`, `@version`, `site`, and `spider`. This should not affect any normal DAST operation, but does affect users who consume the JSON report in an automated way and use these fields. Anyone affected by these changes, and needs these fields for business reasons, is encouraged to open a new GitLab issue and explain the need. - -For more information, see [the removal issue](https://gitlab.com/gitlab-org/gitlab/-/issues/33915). - -### Legacy storage - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -As [announced in GitLab 13.0](https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/#planned-removal-of-legacy-storage-in-14.0), [legacy storage](https://docs.gitlab.com/ee/administration/repository_storage_types.html#legacy-storage) has been removed in GitLab 14.0. - -### License Compliance - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In 13.0, we deprecated the License-Management CI template and renamed it License-Scanning. We have been providing backward compatibility by warning users of the old template to switch. Now in 14.0, we are completely removing the License-Management CI template. Read about it in [issue #216261](https://gitlab.com/gitlab-org/gitlab/-/issues/216261) or [this blog post](https://about.gitlab.com/blog/2021/02/08/composition-analysis-14-deprecations-and-removals/). - -### Limit projects returned in `GET /groups/:id/` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/257829). -</div> - -To improve performance, we are limiting the number of projects returned from the `GET /groups/:id/` API call to 100. A complete list of projects can still be retrieved with the `GET /groups/:id/projects` API call. - -### Make `pwsh` the default shell for newly-registered Windows Runners - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab Runner 13.2, PowerShell Core support was added to the Shell executor. In 14.0, PowerShell Core, `pwsh` is now the default shell for newly-registered Windows runners. Windows `CMD` will still be available as a shell option for Windows runners. Refer to [issue #26419](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26419) for details. - -### Migrate from `SAST_DEFAULT_ANALYZERS` to `SAST_EXCLUDED_ANALYZERS` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/229974). -</div> - -Until GitLab 13.9, if you wanted to avoid running one particular GitLab SAST analyzer, you needed to remove it from the [long string of analyzers in the `SAST.gitlab-ci.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/390afc431e7ce1ac253b35beb39f19e49c746bff/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L12) and use that to set the [`SAST_DEFAULT_ANALYZERS`](https://docs.gitlab.com/ee/user/application_security/sast/#docker-images) variable in your project's CI file. If you did this, it would exclude you from future new analyzers because this string hard codes the list of analyzers to execute. We avoid this problem by inverting this variable's logic to exclude, rather than choose default analyzers. -Beginning with 13.9, [we migrated](https://gitlab.com/gitlab-org/gitlab/-/blob/14fed7a33bfdbd4663d8928e46002a5ef3e3282c/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml#L13) to `SAST_EXCLUDED_ANALYZERS` in our `SAST.gitlab-ci.yml` file. We encourage anyone who uses a [customized SAST configuration](https://docs.gitlab.com/ee/user/application_security/sast/#customizing-the-sast-settings) in their project CI file to migrate to this new variable. If you have not overridden `SAST_DEFAULT_ANALYZERS`, no action is needed. The CI/CD variable `SAST_DEFAULT_ANALYZERS` has been removed in GitLab 14.0, which released on June 22, 2021. - -### Off peak time mode configuration for Docker Machine autoscaling - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab Runner 13.0, [issue #5069](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/5069), we introduced new timing options for the GitLab Docker Machine executor. In GitLab Runner 14.0, we have removed the old configuration option, [off peak time mode](https://docs.gitlab.com/runner/configuration/autoscale.html#off-peak-time-mode-configuration-deprecated). - -### OpenSUSE Leap 15.1 - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Support for [OpenSUSE Leap 15.1 is being deprecated](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5135). Support for 15.1 will be dropped in 14.0. We are now providing support for openSUSE Leap 15.2 packages. - -### PostgreSQL 11 support - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab 14.0 requires PostgreSQL 12 or later. It offers [significant improvements](https://www.postgresql.org/about/news/postgresql-12-released-1976/) to indexing, partitioning, and general performance benefits. - -Starting in GitLab 13.7, all new installations default to PostgreSQL version 12. From GitLab 13.8, single-node instances are automatically upgraded as well. If you aren't ready to upgrade, you can [opt out of automatic upgrades](https://docs.gitlab.com/omnibus/settings/database.html#opt-out-of-automatic-postgresql-upgrades). - -### Redundant timestamp field from DORA metrics API payload - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/325931). -</div> - -The [deployment frequency project-level API](https://docs.gitlab.com/ee/api/dora/metrics.html#list-project-deployment-frequencies) endpoint has been deprecated in favor of the [DORA 4 API](https://docs.gitlab.com/ee/api/dora/metrics.html), which consolidates all the metrics under one API with the specific metric as a required field. As a result, the timestamp field, which doesn't allow adding future extensions and causes performance issues, will be removed. With the old API, an example response would be `{ "2021-03-01": 3, "date": "2021-03-01", "value": 3 }`. The first key/value (`"2021-03-01": 3`) will be removed and replaced by the last two (`"date": "2021-03-01", "value": 3`). - -### Release description in the Tags API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/300887). -</div> - -GitLab 14.0 removes support for the release description in the Tags API. You can no longer add a release description when [creating a new tag](https://docs.gitlab.com/ee/api/tags.html#create-a-new-tag). You also can no longer [create](https://docs.gitlab.com/ee/api/tags.html#create-a-new-release) or [update](https://docs.gitlab.com/ee/api/tags.html#update-a-release) a release through the Tags API. Please migrate to use the [Releases API](https://docs.gitlab.com/ee/api/releases/#create-a-release) instead. - -### Ruby version changed in `Ruby.gitlab-ci.yml` - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -By default, the `Ruby.gitlab-ci.yml` file has included Ruby 2.5. - -To better support the latest versions of Ruby, the template is changed to use `ruby:latest`, which is currently 3.0. To better understand the changes in Ruby 3.0, please reference the [Ruby-lang.org release announcement](https://www.ruby-lang.org/en/news/2020/12/25/ruby-3-0-0-released/). - -Relevant Issue: [Updates Ruby version 2.5 to 3.0](https://gitlab.com/gitlab-org/gitlab/-/issues/329160) - -### SAST analyzer `SAST_GOSEC_CONFIG` variable - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/301215). -</div> - -With the release of [SAST Custom Rulesets](https://docs.gitlab.com/ee/user/application_security/sast/#customize-rulesets) in GitLab 13.5 we allow greater flexibility in configuration options for our Go analyzer (GoSec). As a result we no longer plan to support our less flexible [`SAST_GOSEC_CONFIG`](https://docs.gitlab.com/ee/user/application_security/sast/#analyzer-settings) analyzer setting. This variable was deprecated in GitLab 13.10. -GitLab 14.0 removes the old `SAST_GOSEC_CONFIG variable`. If you use or override `SAST_GOSEC_CONFIG` in your CI file, update your SAST CI configuration or pin to an older version of the GoSec analyzer. We strongly encourage [inheriting and overriding our managed CI templates](https://docs.gitlab.com/ee/user/application_security/sast/#overriding-sast-jobs) to future-proof your CI templates. - -### Service Templates - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Service Templates are [removed in GitLab 14.0](https://gitlab.com/groups/gitlab-org/-/epics/5672). They were used to apply identical settings to a large number of projects, but they only did so at the time of project creation. - -While they solved part of the problem, _updating_ those values later proved to be a major pain point. [Project Integration Management](https://docs.gitlab.com/ee/user/admin_area/settings/project_integration_management.html) solves this problem by enabling you to create settings at the Group or Instance level, and projects within that namespace inheriting those settings. - -### Success and failure for finished build metric conversion - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab Runner 13.5, we introduced `failed` and `success` states for a job. To support Prometheus rules, we chose to convert `success/failure` to `finished` for the metric. In 14.0, the conversion has now been removed. Refer to [issue #26900](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26900) for details. - -### Terraform template version - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](). -</div> - -As we continuously [develop GitLab's Terraform integrations](https://gitlab.com/gitlab-org/gitlab/-/issues/325312), to minimize customer disruption, we maintain two GitLab CI/CD templates for Terraform: - -- The ["latest version" template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.latest.gitlab-ci.yml), which is updated frequently between minor releases of GitLab (such as 13.10, 13.11, etc). -- The ["major version" template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml), which is updated only at major releases (such as 13.0, 14.0, etc). - -At every major release of GitLab, the "latest version" template becomes the "major version" template, inheriting the "latest template" setup. -As we have added many new features to the Terraform integration, the new setup for the "major version" template can be considered a breaking change. - -The latest template supports the [Terraform Merge Request widget](https://docs.gitlab.com/ee/user/infrastructure/iac/mr_integration.html) and -doesn't need additional setup to work with the [GitLab managed Terraform state](https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html). - -To check the new changes, see the [new "major version" template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml). - -### Ubuntu 16.04 support - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Ubuntu 16.04 [reached end-of-life in April 2021](https://ubuntu.com/about/release-cycle), and no longer receives maintenance updates. We strongly recommend users to upgrade to a newer release, such as 20.04. - -GitLab 13.12 will be the last release with Ubuntu 16.04 support. - -### Ubuntu 19.10 (Eoan Ermine) package - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -Ubuntu 19.10 (Eoan Ermine) reached end of life on Friday, July 17, 2020. In GitLab Runner 14.0, Ubuntu 19.10 (Eoan Ermine) is no longer available from our package distribution. Refer to [issue #26036](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26036) for details. - -### Unicorn in GitLab self-managed - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -[Support for Unicorn](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6078) has been removed in GitLab 14.0 in favor of Puma. [Puma has a multi-threaded architecture](https://docs.gitlab.com/ee/administration/operations/puma.html) which uses less memory than a multi-process application server like Unicorn. On GitLab.com, we saw a 40% reduction in memory consumption by using Puma. - -### WIP merge requests renamed 'draft merge requests' - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The WIP (work in progress) status for merge requests signaled to reviewers that the merge request in question wasn't ready to merge. We've renamed the WIP feature to **Draft**, a more inclusive and self-explanatory term. **Draft** clearly communicates the merge request in question isn't ready for review, and makes no assumptions about the progress being made toward it. **Draft** also reduces the cognitive load for new users, non-English speakers, and anyone unfamiliar with the WIP acronym. - -### Web Application Firewall (WAF) - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The Web Application Firewall (WAF) was deprecated in GitLab 13.6 and is removed from GitLab 14.0. The WAF had limitations inherent in the architectural design that made it difficult to meet the requirements traditionally expected of a WAF. By removing the WAF, GitLab is able to focus on improving other areas in the product where more value can be provided to users. Users who currently rely on the WAF can continue to use the free and open source [ModSecurity](https://github.com/SpiderLabs/ModSecurity) project, which is independent from GitLab. Additional details are available in the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/271276). - -### Windows Server 1903 image support - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In 14.0, we have removed Windows Server 1903. Microsoft ended support for this version on 2020-08-12. Refer to [issue #27551](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27551) for details. - -### Windows Server 1909 image support - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In 14.0, we have removed Windows Server 1909. Microsoft ended support for this version on 2021-05-11. Refer to [issue #27899](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27899) for details. - -### `/usr/lib/gitlab-runner` symlink from package - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In GitLab Runner 13.3, a symlink was added from `/user/lib/gitlab-runner/gitlab-runner` to `/usr/bin/gitlab-runner`. In 14.0, the symlink has been removed and the runner is now installed in `/usr/bin/gitlab-runner`. Refer to [issue #26651](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26651) for details. - -### `?w=1` URL parameter to ignore whitespace changes - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -To create a consistent experience for users based on their preferences, support for toggling whitespace changes via URL parameter has been removed in GitLab 14.0. - -### `CI_PROJECT_CONFIG_PATH` variable - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -The `CI_PROJECT_CONFIG_PATH` [predefined project variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) -has been removed in favor of `CI_CONFIG_PATH`, which is functionally the same. - -If you are using `CI_PROJECT_CONFIG_PATH` in your pipeline configurations, -please update them to use `CI_CONFIG_PATH` instead. - -### `FF_RESET_HELPER_IMAGE_ENTRYPOINT` feature flag - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In 14.0, we have deactivated the `FF_RESET_HELPER_IMAGE_ENTRYPOINT` feature flag. Refer to issue [#26679](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26679) for details. - -### `FF_SHELL_EXECUTOR_USE_LEGACY_PROCESS_KILL` feature flag - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -In [GitLab Runner 13.1](https://docs.gitlab.com/runner/executors/shell.html#gitlab-131-and-later), [issue #3376](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3376), we introduced `sigterm` and then `sigkill` to a process in the Shell executor. We also introduced a new feature flag, `FF_SHELL_EXECUTOR_USE_LEGACY_PROCESS_KILL`, so you can use the previous process termination sequence. In GitLab Runner 14.0, [issue #6413](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6413), the feature flag has been removed. - -### `FF_USE_GO_CLOUD_WITH_CACHE_ARCHIVER` feature flag - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab Runner 14.0 removes the `FF_USE_GO_CLOUD_WITH_CACHE_ARCHIVER` feature flag. Refer to [issue #27175](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27175) for details. - -### `secret_detection_default_branch` job - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -- To discuss this change or learn more, see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/297269). -</div> - -To ensure Secret Detection was scanning both default branches and feature branches, we introduced two separate secret detection CI jobs (`secret_detection_default_branch` and `secret_detection`) in our managed [`Secret-Detection.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) template. These two CI jobs created confusion and complexity in the CI rules logic. This deprecation moves the `rule` logic into the `script` section, which then determines how the `secret_detection` job is run (historic, on a branch, commits, etc). -If you override or maintain custom versions of `SAST.gitlab-ci.yml` or `Secret-Detection.gitlab-ci.yml`, you must update your CI templates. We strongly encourage [inheriting and overriding our managed CI templates](https://docs.gitlab.com/ee/user/application_security/secret_detection/#custom-settings-example) to future-proof your CI templates. GitLab 14.0 no longer supports the old `secret_detection_default_branch` job. - -### `trace` parameter in `jobs` API - -<div class="deprecation-notes"> -- Announced in: GitLab <span class="milestone"></span> -- This is a [breaking change](https://docs.gitlab.com/ee/development/deprecation_guidelines/). Review the details carefully before upgrading. -</div> - -GitLab Runner was updated in GitLab 13.4 to internally stop passing the `trace` parameter to the `/api/jobs/:id` endpoint. GitLab 14.0 deprecates the `trace` parameter entirely for all other requests of this endpoint. Make sure your [GitLab Runner version matches your GitLab version](https://docs.gitlab.com/runner/#gitlab-runner-versions) to ensure consistent behavior. +<!-- This redirect file can be deleted after <2023-09-28>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md index c8bed431780..d9641e18e8e 100644 --- a/doc/update/upgrading_from_source.md +++ b/doc/update/upgrading_from_source.md @@ -21,8 +21,7 @@ the [Upgrading from CE to EE](upgrading_from_ce_to_ee.md) documentation. ## Upgrading to a new major version -Major versions are reserved for backwards incompatible changes. We recommend that -you first upgrade to the latest available minor version of your current major version. +Major versions are reserved for backwards incompatible changes. You should first upgrade to the latest available minor version of your current major version. Follow the [Upgrade Recommendations](../policy/maintenance.md#upgrade-recommendations) to identify the ideal upgrade path. @@ -96,11 +95,11 @@ Download and install Go (for Linux, 64-bit): # Remove former Go installation folder sudo rm -rf /usr/local/go -curl --remote-name --location --progress-bar "https://go.dev/dl/go1.18.8.linux-amd64.tar.gz" -echo '4d854c7bad52d53470cf32f1b287a5c0c441dc6b98306dea27358e099698142a go1.18.8.linux-amd64.tar.gz' | shasum -a256 -c - && \ - sudo tar -C /usr/local -xzf go1.18.8.linux-amd64.tar.gz +curl --remote-name --location --progress-bar "https://go.dev/dl/go1.19.10.linux-amd64.tar.gz" +echo '8b045a483d3895c6edba2e90a9189262876190dbbd21756870cdd63821810677 go1.19.10.linux-amd64.tar.gz' | shasum -a256 -c - && \ + sudo tar -C /usr/local -xzf go1.19.10.linux-amd64.tar.gz sudo ln -sf /usr/local/go/bin/{go,gofmt} /usr/local/bin/ -rm go1.18.8.linux-amd64.tar.gz +rm go1.19.10.linux-amd64.tar.gz ``` ### 6. Update Git @@ -108,7 +107,7 @@ rm go1.18.8.linux-amd64.tar.gz To check you are running the minimum required Git version, see [Git versions](../install/installation.md#software-requirements). -From GitLab 13.6, we recommend you use the +From GitLab 13.6, you should use the [Git version provided by Gitaly](https://gitlab.com/gitlab-org/gitaly/-/issues/2729) that: @@ -394,11 +393,6 @@ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production If all items are green, then congratulations, the upgrade is complete! -### 17. Upgrade the product documentation - -This is an optional step. If you [installed the product documentation](../install/installation.md#install-the-product-documentation), -see how to [upgrade to a later version](../administration/docs_self_host.md#upgrade-the-product-documentation-to-a-later-version). - ## Version specific changes Upgrading versions might need some manual intervention. For more information, diff --git a/doc/user/admin_area/analytics/dev_ops_reports.md b/doc/user/admin_area/analytics/dev_ops_reports.md index 69bf4ff30f1..30ba2ad3e7b 100644 --- a/doc/user/admin_area/analytics/dev_ops_reports.md +++ b/doc/user/admin_area/analytics/dev_ops_reports.md @@ -1,74 +1,11 @@ --- -stage: Plan -group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/analytics/dev_ops_reports.md' +remove_date: '2023-10-06' --- -# DevOps Reports **(FREE SELF)** +This document was moved to [another location](../../../administration/analytics/dev_ops_reports.md). -DevOps Reports give you an overview of your entire instance's adoption of -[Concurrent DevOps](https://about.gitlab.com/topics/concurrent-devops/) -from planning to monitoring. - -To see DevOps Reports: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Analytics > DevOps Reports**. - -## DevOps Score - -> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/20976) from Conversational Development Index in GitLab 12.6. - -NOTE: -To see the DevOps score, you must activate your GitLab instance's [Service Ping](../settings/usage_statistics.md#service-ping). DevOps Score is a comparative tool, so your score data must be centrally processed by GitLab Inc. first. - -You can use the DevOps score to compare your DevOps status to other organizations. - -The DevOps Score tab displays usage of major GitLab features on your instance over -the last 30 days, averaged over the number of billable users in that time period. -You can also see the Leader usage score, calculated from top-performing instances based on -[Service Ping data](../settings/usage_statistics.md#service-ping) that GitLab has collected. -Your score is compared to the lead score of each feature and then expressed -as a percentage at the bottom of said feature. Your overall **DevOps Score** is an average of your -feature scores. - -Service Ping data is aggregated on GitLab servers for analysis. Your usage -information is **not sent** to any other GitLab instances. -If you have just started using GitLab, it might take a few weeks for data to be collected before this -feature is available. - -## DevOps Adoption **(ULTIMATE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247112) in GitLab 13.7 as a [Beta feature](../../../policy/experiment-beta-support.md#beta). -> - The Overview tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330401) in GitLab 14.1. -> - DAST and SAST metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328033) in GitLab 14.1. -> - Fuzz Testing metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/330398) in GitLab 14.2. -> - Dependency Scanning metrics [added](https://gitlab.com/gitlab-org/gitlab/-/issues/328034) in GitLab 14.2. -> - Multi-select [added](https://gitlab.com/gitlab-org/gitlab/-/issues/333586) in GitLab 14.2. -> - Overview table [added](https://gitlab.com/gitlab-org/gitlab/-/issues/335638) in GitLab 14.3. - -DevOps Adoption shows feature adoption for development, security, and operations. - -| Category | Feature | -| --- | --- | -| Development | Approvals<br>Code owners<br>Issues<br>Merge requests | -| Security | DAST<br>Dependency Scanning<br>Fuzz Testing<br>SAST | -| Operations | Deployments<br>Pipelines<br>Runners | - -You can use Group DevOps Adoption to: - -- Identify specific subgroups that are lagging in their adoption of GitLab features, so you can guide them on -their DevOps journey. -- Find subgroups that have adopted certain features, and provide guidance to other subgroups on -how to use those features. -- Verify if you are getting the return on investment that you expected from GitLab. - -## Add or remove a group - -To add or remove a subgroup from the DevOps Adoption report: - -1. Select **Add or remove groups**. -1. Select the subgroup you want to add or remove and select **Save changes**. - - +<!-- This redirect file can be deleted after <2023-10-06>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/analytics/index.md b/doc/user/admin_area/analytics/index.md index 6441cd866c8..f5849f0b876 100644 --- a/doc/user/admin_area/analytics/index.md +++ b/doc/user/admin_area/analytics/index.md @@ -1,26 +1,11 @@ --- -stage: Plan -group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/analytics/index.md' +remove_date: '2023-10-06' --- -# Instance-level analytics **(FREE SELF)** +This document was moved to [another location](../../../administration/analytics/index.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41416) in GitLab 11.2. - -Instance-level analytics provide insights into the feature and data usage of your entire instance. - -## View instance-level analytics - -Prerequisite: - -- You must have administrator access to the instance. - -To view instance-level analytics: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Analytics**, then one of the available analytics: - - - [DevOps Reports](dev_ops_reports.md): Provides an overview of your entire instance's feature usage. - - [Usage Trends](usage_trends.md): Shows how much data your instance contains, and how the data is changing. +<!-- This redirect file can be deleted after <2023-10-06>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/analytics/usage_trends.md b/doc/user/admin_area/analytics/usage_trends.md index 49e82f71a3a..12eb44d6ebc 100644 --- a/doc/user/admin_area/analytics/usage_trends.md +++ b/doc/user/admin_area/analytics/usage_trends.md @@ -1,46 +1,11 @@ --- -stage: Plan -group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/analytics/usage_trends.md' +remove_date: '2023-10-07' --- -# Usage Trends **(FREE SELF)** +This document was moved to [another location](../../../administration/analytics/usage_trends.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235754) in GitLab 13.5 behind a feature flag, disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46962) in GitLab 13.6. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/285220) from Instance Statistics to Usage Trends in GitLab 13.6. -> - It's enabled on GitLab.com. -> - It's recommended for production use. - -Usage Trends gives you an overview of how much data your instance contains, and how quickly this volume is changing over time. -Usage Trends data refreshes daily. - -## View Usage Trends - -To view Usage Trends: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Analytics > Usage Trends**. - -## Total counts - -At the top of the page, Usage Trends shows total counts for: - -- Users -- Projects -- Groups -- Issues -- Merge requests -- Pipelines - -These figures can be useful for understanding how much data your instance contains in total. - -## Past year trend charts - -Usage Trends also displays line charts that show total counts per month, over the past 12 months, -in the categories shown in [Total counts](#total-counts). - -These charts help you visualize how rapidly these records are being created on your instance. - - +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/appearance.md b/doc/user/admin_area/appearance.md index 99c2a453b07..6a3760bc62d 100644 --- a/doc/user/admin_area/appearance.md +++ b/doc/user/admin_area/appearance.md @@ -1,123 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/appearance.md' +remove_date: '2023-10-07' --- -# GitLab Appearance **(FREE SELF)** +This document was moved to [another location](../../administration/appearance.md). -Several options are available for customizing the appearance of a self-managed instance -of GitLab. To access these settings: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Appearance**. - -## Navigation bar - -By default, the navigation bar has the GitLab logo, but this can be customized with -any image desired. It is optimized for images 28px high (any width), but any image can be -used (less than 1 MB) and it is automatically resized. - -After you select and upload an image, select **Update appearance settings** at the bottom -of the page to activate it in the GitLab instance. - -NOTE: -GitLab pipeline emails also display the custom logo, unless the logo is in SVG format. If the -custom logo is in SVG format, the default logo is used instead because the SVG format is not -supported by many email clients. - -## Favicon - -By default, the favicon (used by the browser as the tab icon, as well as the CI status icon) -uses the GitLab logo. This can be customized with any icon desired. It must be a -32x32 `.png` or `.ico` image. - -After you select and upload an icon, select **Update appearance settings** at the bottom -of the page to activate it in the GitLab instance. - -## System header and footer messages - -> **Enable header and footer in emails** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344819) in GitLab 15.9. - -You can add a small header message, a small footer message, or both, to the interface -of your GitLab instance. These messages appear on all projects and pages of the -instance, including the sign in / sign up page. The default color is white text on -an orange background, but this can be customized by selecting **Customize colors**. - -Limited [Markdown](../markdown.md) is supported, such as bold, italics, and links, for -example. Other Markdown features, including lists, images, and quotes are not supported -as the header and footer messages can only be a single line. - -You can select **Enable header and footer in emails** to have the text of -the header and footer added to all emails sent by the GitLab instance. - -After you add a message, select **Update appearance settings** at the bottom of the page -to activate it in the GitLab instance. - -## Sign in / Sign up pages - -You can replace the default message on the sign in / sign up page with your own message -and logo. You can make full use of [Markdown](../markdown.md) in the description. - -The optimal size for the logo is 128 x 128 pixels, but any image can be used (below 1 MB) -and it is resized automatically. The logo image appears between the title and -the description, on the left of the sign-up page. - -After you add a message, select **Update appearance settings** at the bottom of the page -to activate it in the GitLab instance. You can also select **Sign-in page**, -to review the saved appearance settings: - -NOTE: -You can add also add a [customized help message](settings/help_page.md) below the sign in message or add [a Sign in text message](settings/sign_in_restrictions.md#sign-in-information). - -## Progressive Web App - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375708) in GitLab 15.9. - -GitLab can be installed as a [Progressive Web App](https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps) (PWA). -Use the Progressive Web App settings to customize its appearance, including its name, -description, and icon. - -### Configure the PWA settings - -To configure the PWA settings: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Appearance**. -1. Scroll to the **Progressive Web App (PWA)** section. -1. Complete the fields. - - **Icon**: If you use the standard GitLab icon, it is available in sizes 192x192 pixels, - 512x512 pixels, also as a maskable icon. If you use a custom icon, it must be in either size - 192x192 pixels, or 512x512 pixels. -1. Select **Update appearance settings**. - -## New project pages - -You can add a new project guidelines message to the **New project page** in GitLab. -You can make full use of [Markdown](../markdown.md) in the description: - -The message is displayed below the **New Project** message, on the left side -of the **New project page**. - -After you add a message, select **Update appearance settings** at the bottom of the page -to activate it in the GitLab instance. You can also select **New project page**, -which brings you to the new project page so you can review the change. - -## Libravatar - -[Libravatar](https://www.libravatar.org) is supported by GitLab for avatar images, but you must -[manually enable Libravatar support on the GitLab instance](../../administration/libravatar.md) to use the service. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index 6fe82b3ae83..4893b94ce50 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -1,120 +1,11 @@ --- -stage: Growth -group: Acquisition -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference, howto +redirect_to: '../../administration/broadcast_messages.md' +remove_date: '2023-10-07' --- -# Broadcast messages **(FREE SELF)** +This document was moved to [another location](../../administration/broadcast_messages.md). -> - Target roles [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default. -> - Theme [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83251) and background color removed in GitLab 14.10. - -GitLab can display broadcast messages to users of a GitLab instance. There are two types of broadcast messages: - -- Banners -- Notifications - -Broadcast messages can be managed using the [broadcast messages API](../../api/broadcast_messages.md). - -## Banners - -Banners are shown on the top of a page and optionally in the command line as a Git remote response. - - - -```shell -$ git push -... -remote: -remote: **Welcome** to GitLab :wave: -remote: -... -``` - -If more than one banner is active at one time, they are displayed at the top of the page in order of creation. In the command line, only the latest banner is shown. - -## Notifications - -Notifications are shown on the bottom right of a page and can contain placeholders. A placeholder is replaced with an attribute of the active user. Placeholders must be surrounded by curly braces, for example `{{name}}`. -The available placeholders are: - -- `{{email}}` -- `{{name}}` -- `{{user_id}}` -- `{{username}}` -- `{{instance_id}}` - -If the user is not signed in, user related values are empty. - - - -If more than one notification is active at one time, only the newest is shown. - -## Add a broadcast message - -To display messages to users on your GitLab instance, add a broadcast message. - -To add a broadcast message: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Messages**. -1. Add the text for the message to the **Message** field. You can style a message's content using Markdown, emoji, and the `a` and `br` HTML tags. - The `br` tag inserts a line break. The `a` HTML tag accepts `class` and `style` attributes with the following CSS properties: - - `color` - - `border` - - `background` - - `padding` - - `margin` - - `text-decoration` -1. Select a **Theme**. The default theme is `indigo`. -1. Select the **Dismissable** checkbox to enable users to dismiss the broadcast message. -1. Optional. Clear the **Git remote responses** checkbox to prevent broadcast messages from being displayed in the command line as Git remote responses. -1. Optional. Select **Target roles** to only show the broadcast message to users with the selected roles. The message displays on group, subgroup, and project pages, and does not display in Git remote responses. -1. If required, add a **Target Path** to only show the broadcast message on URLs matching that path. You can use the wildcard character `*` to match multiple URLs, for example `mygroup/myproject*`. -1. Select a date and time (UTC) for the message to start and end. -1. Select **Add broadcast message**. - -When a broadcast message expires, it no longer displays in the user interface but is still listed in the -list of broadcast messages. - -## Edit a broadcast message - -If you must make changes to a broadcast message, you can edit it. - -To edit a broadcast message: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Messages**. -1. From the list of broadcast messages, select the edit button for the message. -1. After making the required changes, select **Update broadcast message**. - -Expired messages can be made active again by changing their end date. - -## Delete a broadcast message - -If you no longer require a broadcast message, you can delete it. -You can delete a broadcast message while it's active. - -To delete a broadcast message: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Messages**. -1. From the list of broadcast messages, select the delete button for the message. - -When a broadcast message is deleted, it's removed from the list of broadcast messages. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 9bacd101c48..f38eed5de43 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -1,86 +1,11 @@ --- -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto +redirect_to: '../../administration/credentials_inventory.md' +remove_date: '2023-10-07' --- -# Credentials inventory **(ULTIMATE SELF)** +This document was moved to [another location](../../administration/credentials_inventory.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20912) in GitLab 12.6. -> - [Bot-created access tokens not displayed in personal access token list](https://gitlab.com/gitlab-org/gitlab/-/issues/351759) in GitLab 14.9. - -GitLab administrators are responsible for the overall security of their instance. To assist, GitLab -provides a Credentials inventory to keep track of all the credentials that can be used to access -their self-managed instance. - -Use Credentials inventory to see for your GitLab instance all: - -- Personal access tokens (PAT). -- Project access tokens (GitLab 14.8 and later). -- SSH keys. -- GPG keys. - -You can also [revoke](#revoke-a-users-personal-access-token) and [delete](#delete-a-users-ssh-key) and see: - -- Who they belong to. -- Their access scope. -- Their usage pattern. -- When they expire. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. -- When they were revoked. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214809) in GitLab 13.2. - -To access the Credentials inventory: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Credentials**. - -## Revoke a user's personal access token - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214811) in GitLab 13.4. - -If you see a **Revoke** button, you can revoke that user's PAT. Whether you see a **Revoke** button depends on the token state, and if an expiration date has been set. For more information, see the following table: - -| Token state | Show Revoke button? | Comments | -|-------------|---------------------|----------------------------------------------------------------------------| -| Active | Yes | Allows administrators to revoke the PAT, such as for a compromised account | -| Expired | No | Not applicable; token is already expired | -| Revoked | No | Not applicable; token is already revoked | - -When a PAT is revoked from the credentials inventory, the instance notifies the user by email. - - - -## Revoke a user's project access token - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243833) in GitLab 14.8. - -The **Revoke** button next to a project access token can be selected to revoke that particular project access token. This both: - -- Revokes the token project access token. -- Enqueues a background worker to delete the project bot user. - - - -## Delete a user's SSH key - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225248) in GitLab 13.5. - -You can **Delete** a user's SSH key by navigating to the credentials inventory's SSH Keys tab. -The instance then notifies the user. - - - -## Review existing GPG keys - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/282429) in GitLab 13.10. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292961) in GitLab 13.12. - -You can view all existing GPG in your GitLab instance by navigating to the -credentials inventory GPG Keys tab, as well as the following properties: - -- Who the GPG key belongs to. -- The ID of the GPG key. -- Whether the GPG key is [verified or unverified](../project/repository/gpg_signed_commits/index.md) - - +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/custom_project_templates.md b/doc/user/admin_area/custom_project_templates.md index 0e0acf4af57..dc773b0aaca 100644 --- a/doc/user/admin_area/custom_project_templates.md +++ b/doc/user/admin_area/custom_project_templates.md @@ -1,73 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/custom_project_templates.md' +remove_date: '2023-10-10' --- -# Custom instance-level project templates **(PREMIUM SELF)** +This document was moved to [another location](../../administration/custom_project_templates.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6860) in GitLab 11.2. - -GitLab administrators can set a group to be the source of project templates that are -selectable when a new project is created on the instance. These templates can be selected -when you go to **New project > Create from template** and select the **Instance** tab. - -Every project in the group, but not its subgroups, can be selected when a new project -is created, based on the user's access permissions: - -- Public projects can be selected by any authenticated user as a template for a new project, - if all enabled [project features](../project/settings/index.md#configure-project-visibility-features-and-permissions) - except for **GitLab Pages** and **Security and Compliance** are set to **Everyone With Access**. - The same applies to internal projects. -- Private projects can be selected only by users who are members of the projects. - -The **Metrics Dashboard** is set to **Only Project Members** when you create a new project. Make -sure you change it to **Everyone With Access** before making it a project template. - -Repository and database information that are copied over to each new project are -identical to the data exported with the [GitLab Project Import/Export](../project/settings/import_export.md). - -To set project templates at the group level, see [Custom group-level project templates](../group/custom_project_templates.md). - -## Select instance-level project template group - -To select the group to use as the source for the project templates: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Templates**. -1. Expand **Custom project templates**. -1. Select a group to use. -1. Select **Save changes**. - -Projects in subgroups of the template group are **not** included in the template list. - -## What is copied from the templates - -The entire custom instance-level project templates repository is copied, including: - -- Branches -- Commits -- Tags - -If the user: - -- Has the Owner role on the custom instance-level project templates project or is a GitLab administrator, all project settings are copied over to the new - project. -- Doesn't have the Owner role or is not a GitLab administrator, project [deploy keys](../project/deploy_keys/index.md#view-deploy-keys) and project - [webhooks](../project/integrations/webhooks.md) aren't copied over because they contain sensitive data. - -To learn more about what is migrated, see -[Items that are exported](../project/settings/import_export.md#items-that-are-exported). - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md index 72e8269e455..54d7d82af98 100644 --- a/doc/user/admin_area/diff_limits.md +++ b/doc/user/admin_area/diff_limits.md @@ -1,53 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../administration/diff_limits.md' +remove_date: '2023-10-10' --- -# Diff limits administration **(FREE SELF)** +This document was moved to [another location](../../administration/diff_limits.md). -You can set a maximum size for display of diff files (patches). - -For details about diff files, [view changes between files](../project/merge_requests/changes.md). -Read more about the [built-in limits for merge requests and diffs](../../administration/instance_limits.md#merge-requests). - -## Configure diff limits - -WARNING: -These settings are experimental. An increased maximum increases resource -consumption of your instance. Keep this in mind when adjusting the maximum. - -To speed the loading time of merge request views and branch comparison views -on your instance, you can configure three instance-level maximum values for diffs: - -| Value | Definition | Default value | Maximum value | -| ----- | ---------- | :-----------: | :-----------: | -| **Maximum diff patch size** | The total size, in bytes, of the entire diff. | 200 KB | 500 KB | -| **Maximum diff files** | The total number of files changed in a diff. | 1000 | 3000 | -| **Maximum diff lines** | The total number of lines changed in a diff. | 50,000 | 100,000 | - -When a diff reaches 10% of any of these values, the files are shown in a -collapsed view, with a link to expand the diff. Diffs that exceed any of the -set values are presented as **Too large** are cannot be expanded in the UI. - -To configure these values: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Diff limits**. -1. Enter a value for the diff limit. -1. Select **Save changes**. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/email_from_gitlab.md b/doc/user/admin_area/email_from_gitlab.md index fbdfe437719..e5194f05381 100644 --- a/doc/user/admin_area/email_from_gitlab.md +++ b/doc/user/admin_area/email_from_gitlab.md @@ -1,61 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto, reference +redirect_to: '../../administration/email_from_gitlab.md' +remove_date: '2023-10-10' --- -# Email from GitLab **(PREMIUM SELF)** +This document was moved to [another location](../../administration/email_from_gitlab.md). -GitLab provides a tool to administrators for emailing all users, or users of -a chosen group or project, right from the Admin Area. Users receive the email -at their primary email address. - -For information about email notifications originating from GitLab, read -[GitLab notification emails](../profile/notifications.md). - -## Use-cases - -- Notify your users about a new project, a new feature, or a new product launch. -- Notify your users about a new deployment, or that downtime is expected - for a particular reason. - -## Sending emails to users from GitLab - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select **Send email to users**. - -  - -1. Compose an email and choose where to send it (all users or users of a - chosen group or project). The email body only supports plain text messages. - HTML, Markdown, and other rich text formats are not supported, and is - sent as plain text to users. - -  - -NOTE: -[Starting with GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/31509), email notifications can be sent only once every 10 minutes. This helps minimize performance issues. - -## Unsubscribing from emails - -Users can choose to unsubscribe from receiving emails from GitLab by following -the unsubscribe link in the email. Unsubscribing is unauthenticated in order -to keep this feature simple. - -On unsubscribe, users receive an email notification that unsubscribe happened. -The endpoint that provides the unsubscribe option is rate-limited. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/external_users.md b/doc/user/admin_area/external_users.md index a63093deab0..ec28f109384 100644 --- a/doc/user/admin_area/external_users.md +++ b/doc/user/admin_area/external_users.md @@ -1,80 +1,11 @@ --- -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/external_users.md' +remove_date: '2023-10-10' --- -# External users **(FREE SELF)** +This document was moved to [another location](../../administration/external_users.md). -In cases where it is desired that a user has access only to some internal or -private projects, there is the option of creating **External Users**. This -feature may be useful when for example a contractor is working on a given -project and should only have access to that project. - -External users: - -- Cannot create project, groups, and snippets in their personal namespaces. -- Can only create projects (including forks), subgroups, and snippets within top-level groups to which they are explicitly granted access. -- Can only access public projects and projects to which they are explicitly granted access, - thus hiding all other internal or private ones from them (like being - logged out). -- Can only access public groups and groups to which they are explicitly granted access, - thus hiding all other internal or private ones from them (like being - logged out). -- Can only access public snippets. - -Access can be granted by adding the user as member to the project or group. -Like usual users, they receive a role in the project or group with all -the abilities that are mentioned in the [permissions table](../permissions.md#project-members-permissions). -For example, if an external user is added as Guest, and your project is internal or -private, they do not have access to the code; you need to grant the external -user access at the Reporter level or above if you want them to have access to the code. You should -always take into account the -[project's visibility and permissions settings](../project/settings/index.md#configure-project-visibility-features-and-permissions) -as well as the permission level of the user. - -NOTE: -External users still count towards a license seat. - -An administrator can flag a user as external by either of the following methods: - -- [Through the API](../../api/users.md#user-modification). -- Using the GitLab UI: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. On the left sidebar, select **Overview > Users** to create a new user or edit an existing one. - There, you can find the option to flag the user as external. - -Additionally, users can be set as external users using: - -- [SAML groups](../../integration/saml.md#external-groups). -- [LDAP groups](../../administration/auth/ldap/ldap_synchronization.md#external-groups). -- the [External providers list](../../integration/omniauth.md#create-an-external-providers-list). - -## Set a new user to external - -By default, new users are not set as external users. This behavior can be changed -by an administrator: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. - -If you change the default behavior of creating new users as external, you -have the option to narrow it down by defining a set of internal users. -The **Internal users** field allows specifying an email address regex pattern to -identify default internal users. New users whose email address matches the regex -pattern are set to internal by default rather than an external collaborator. - -The regex pattern format is in Ruby, but it needs to be convertible to JavaScript, -and the ignore case flag is set (`/regex pattern/i`). Here are some examples: - -- Use `\.internal@domain\.com$` to mark email addresses ending with - `.internal@domain.com` as internal. -- Use `^(?:(?!\.ext@domain\.com).)*$\r?` to mark users with email addresses - not including `.ext@domain.com` as internal. - -WARNING: -Be aware that this regex could lead to a -[regular expression denial of service (ReDoS) attack](https://en.wikipedia.org/wiki/ReDoS). +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/geo_sites.md b/doc/user/admin_area/geo_sites.md index 3bae90aaec8..cd0f2f5646a 100644 --- a/doc/user/admin_area/geo_sites.md +++ b/doc/user/admin_area/geo_sites.md @@ -1,119 +1,11 @@ --- -stage: Systems -group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/geo_sites.md' +remove_date: '2023-10-10' --- -# Geo sites Admin Area **(PREMIUM SELF)** +This document was moved to [another location](../../administration/geo_sites.md). -You can configure various settings for GitLab Geo sites. For more information, see -[Geo documentation](../../administration/geo/index.md). - -On either the primary or secondary site: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Geo > Sites**. - -## Common settings - -All Geo sites have the following settings: - -| Setting | Description | -| --------| ----------- | -| Primary | This marks a Geo site as **primary** site. There can be only one **primary** site. | -| Name | The unique identifier for the Geo site. It's highly recommended to use a physical location as a name. Good examples are "London Office" or "us-east-1". Avoid words like "primary", "secondary", "Geo", or "DR". This makes the failover process easier because the physical location does not change, but the Geo site role can. All nodes in a single Geo site use the same site name. Nodes use the `gitlab_rails['geo_node_name']` setting in `/etc/gitlab/gitlab.rb` to lookup their Geo site record in the PostgreSQL database. If `gitlab_rails['geo_node_name']` is not set, the node's `external_url` with trailing slash is used as fallback. The value of `Name` is case-sensitive, and most characters are allowed. | -| URL | The instance's user-facing URL. | - -The site you're currently browsing is indicated with a blue `Current` label, and -the **primary** node is listed first as `Primary site`. - -## Secondary site settings - -**Secondary** sites have a number of additional settings available: - -| Setting | Description | -|---------------------------|-------------| -| Selective synchronization | Enable Geo [selective sync](../../administration/geo/replication/configuration.md#selective-synchronization) for this **secondary** site. | -| Repository sync capacity | Number of concurrent requests this **secondary** site makes to the **primary** site when backfilling repositories. | -| File sync capacity | Number of concurrent requests this **secondary** site makes to the **primary** site when backfilling files. | - -## Geo backfill - -**Secondary** sites are notified of changes to repositories and files by the **primary** site, -and always attempt to synchronize those changes as quickly as possible. - -Backfill is the act of populating the **secondary** site with repositories and files that -existed *before* the **secondary** site was added to the database. Because there may be -extremely large numbers of repositories and files, it's not feasible to attempt to -download them all at once; so, GitLab places an upper limit on the concurrency of -these operations. - -How long the backfill takes is dependent on the maximum concurrency, but higher -values place more strain on the **primary** site. The limits are configurable. -If your **primary** site has lots of surplus capacity, -you can increase the values to complete backfill in a shorter time. If it's -under heavy load and backfill reduces its availability for standard requests, -you can decrease them. - -## Set up the internal URLs - -> Setting up internal URLs in secondary sites was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77179) in GitLab 14.7. - -You can set up a different URL for synchronization between the primary and secondary site. - -The **primary** site's Internal URL is used by **secondary** sites to contact it -(to sync repositories, for example). The name Internal URL distinguishes it from -[External URL](https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab), -which is used by users. Internal URL does not need to be a private address. - -When [Geo secondary proxying](../../administration/geo/secondary_proxy/index.md) is enabled, -the primary uses the secondary's internal URL to contact it directly. - -The internal URL defaults to external URL. To change it: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Geo > Sites**. -1. Select **Edit** on the site you want to customize. -1. Edit the internal URL. -1. Select **Save changes**. - -When enabled, the Admin Area for Geo shows replication details for each site directly -from the primary site's UI, and through the Geo secondary proxy, if enabled. - -WARNING: -We recommend using an HTTPS connection while configuring the Geo sites. To avoid -breaking communication between **primary** and **secondary** sites when using -HTTPS, customize your Internal URL to point to a load balancer with TLS -terminated at the load balancer. - -WARNING: -Starting with GitLab 13.3 and [until 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/325522), -if you use an internal URL that is not accessible to the users, the -OAuth authorization flow does not work properly, because users are redirected -to the internal URL instead of the external one. - -## Multiple secondary sites behind a load balancer - -**Secondary** sites can use identical external URLs if -a unique `name` is set for each Geo site. The `gitlab.rb` setting -`gitlab_rails['geo_node_name']` must: - -- Be set for each GitLab instance that runs `puma`, `sidekiq`, or `geo_logcursor`. -- Match a Geo site name. - -The load balancer must use sticky sessions to avoid authentication -failures and cross-site request errors. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index a59501f14ce..a729cefa7b8 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -1,440 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../administration/admin_area.md' +remove_date: '2023-10-11' --- -# GitLab Admin Area **(FREE SELF)** +This document was moved to [another location](../../administration/admin_area.md). -The Admin Area provides a web UI to manage and configure features of GitLab -self-managed instances. If you are an administrator,to access the Admin Area: - -- In GitLab 16.1 and later: on the left sidebar, expand the top-most chevron (**{chevron-down}**), then select **Admin Area**. -- In GitLab 16.0 and earlier: on the top bar, select **Main menu > Admin**. - -NOTE: -Only administrators can access the Admin Area. - -## Administering projects - -You can administer all projects in the GitLab instance from the Admin Area's Projects page. - -To access the Projects page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Projects**. -1. Select the **All**, **Private**, **Internal**, or **Public** tab to list only - projects of that criteria. - -By default, all projects are listed, in reverse order of when they were last updated. For each -project, the following information is listed: - -- Name -- Namespace -- Description -- Size, updated every 15 minutes at most - -Projects can be edited or deleted. - -To edit a project's name or description: - -1. In the Projects overview, next to the project you want to edit, select **Edit**. -1. Edit the **Project name** or **Project description**. -1. Select **Save Changes**. - -To delete a project: - -1. In the Projects overview, next to the project you want to delete, select **Delete**. - -The list of projects can be sorted by: - -- Updated date -- Last created -- Name -- Most stars -- Oldest created -- Oldest updated -- Largest repository - -A user can choose to hide or show archived projects in the list. - -In the **Filter by name** field, type the project name you want to find, and GitLab filters -them as you type. - -To filter only projects in that namespace, select from the **Namespace** dropdown list. - -You can combine the filter options. For example, to list only public projects with `score` in their name: - -1. Select the **Public** tab. -1. Enter `score` in the **Filter by name...** input box. - -## Administering users - -You can administer all users in the GitLab instance from the Admin Area's Users page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. - -To list users matching a specific criteria, select one of the following tabs on the **Users** page: - -- **Active** -- **Admins** -- **2FA Enabled** -- **2FA Disabled** -- **External** -- **[Blocked](moderate_users.md#block-a-user)** -- **[Deactivated](moderate_users.md#deactivate-a-user)** -- **Without projects** - -For each user, the following are listed: - -1. Username -1. Email address -1. Project membership count -1. Group membership count ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/276215) in GitLab 13.12) -1. Date of account creation -1. Date of last activity - -To edit a user, in the user's row, select **Edit**. To delete the user, or delete the user and their contributions, select the cog dropdown list in -that user's row, and select the desired option. - -To change the sort order: - -1. Select the sort dropdown list. -1. Select the desired order. - -By default the sort dropdown list shows **Name**. - -To search for users, enter your criteria in the search field. The user search is case -insensitive, and applies partial matching to name and username. To search for an email address, -you must provide the complete email address. - -### User impersonation - -An administrator can "impersonate" any other user, including other administrators. -This allows the administrator to "see what the user sees," and take actions on behalf of the user. -You can impersonate a user in the following ways: - -- Through the UI: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. On the left sidebar, select **Overview > Users**. - 1. From the list of users, select a user. - 1. Select **Impersonate**. -- With the API, using [impersonation tokens](../../api/rest/index.md#impersonation-tokens). - -All impersonation activities are [captured with audit events](../../administration/audit_events.md#user-impersonation). - -By default, impersonation is enabled. GitLab can be configured to [disable impersonation](../../api/rest/index.md#disable-impersonation). - - - -### User identities - -> The ability to see a user's SCIM identity was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/294608) in GitLab 15.3. - -When using authentication providers, administrators can see the identities for a user: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. From the list of users, select a user. -1. Select **Identities**. - -This list shows the user's identities, including SCIM identities. Administrators can use this information to troubleshoot SCIM-related issues and confirm -the identities being used for an account. - -### User Permission Export **(PREMIUM SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1772) in GitLab 13.8. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292436) in GitLab 13.9. - -An administrator can export user permissions for all users in the GitLab instance from the Admin Area's Users page. -The export lists direct membership the users have in groups and projects. - -The following data is included in the export: - -- Username -- Email -- Type -- Path -- Access level ([Project](../permissions.md#project-members-permissions) and [Group](../permissions.md#group-members-permissions)) -- Date of last activity ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345388) in GitLab 14.6). For a list of activities that populate this column, see the [Users API documentation](../../api/users.md#get-user-activities). - -Only the first 100,000 user accounts are exported. - - - -### Users statistics - -The **Users statistics** page provides an overview of user accounts by role. These statistics are -calculated daily, so user changes made since the last update are not reflected. - -The following totals are also included: - -- Billable users -- Blocked users -- Total users - -GitLab billing is based on the number of [**Billable users**](../../subscriptions/self_managed/index.md#billable-users). - -### Add email to user - -You must be an administrator to manually add emails to users: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Locate the user and select them. -1. Select **Edit**. -1. In **Email**, enter the new email address. This adds the new email address to the - user and sets the previous email address to be a secondary. -1. Select **Save changes**. - -## User cohorts - -The [Cohorts](user_cohorts.md) tab displays the monthly cohorts of new users and their activities over time. - -## Prevent a user from creating groups - -By default, users can create groups. To prevent a user from creating a top level group: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Locate the user and select them. -1. Select **Edit**. -1. Clear the **Can create group** checkbox. -1. Select **Save changes**. - -It is also possible to [limit which roles can create a subgroup within a group](../group/subgroups/index.md#change-who-can-create-subgroups). - -## Administering groups - -You can administer all groups in the GitLab instance from the Admin Area's Groups page. - -To access the Groups page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Groups**. - -For each group, the page displays their name, description, size, number of projects in the group, -number of members, and whether the group is private, internal, or public. To edit a group, in the group's row, select **Edit**. To delete the group, in the group's row, select **Delete**. - -To change the sort order, select the sort dropdown list and select the desired order. The default -sort order is by **Last created**. - -To search for groups by name, enter your criteria in the search field. The group search is case -insensitive, and applies partial matching. - -To [Create a new group](../group/index.md#create-a-group) select **New group**. - -## Administering topics - -- > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340920) in GitLab 14.4. -- > Merging topics [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/366884) in GitLab 15.5. - -[Topics](../project/working_with_projects.md#explore-topics) are used to categorize and find similar projects. - -You can administer all topics in the GitLab instance from the Admin Area's Topics page. - -To access the Topics page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Topics**. - -For each topic, the page displays its name and the number of projects labeled with the topic. - -To create a new topic, select **New topic**. - -To edit a topic, select **Edit** in that topic's row. - -To remove a topic, select **Remove** in that topic's row. - -To remove a topic and move all assigned projects to another topic, select **Merge topics**. - -To search for topics by name, enter your criteria in the search box. The topic search is case -insensitive and applies partial matching. - -NOTE: -The assigned topics are visible only to everyone with access to the project, -but everyone can see which topics exist on the GitLab instance. -Do not include sensitive information in the name of a topic. - -## Administering Gitaly servers - -You can list all Gitaly servers in the GitLab instance from the Admin Area's **Gitaly Servers** -page. For more details, see [Gitaly](../../administration/gitaly/index.md). - -To access the **Gitaly Servers** page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Gitaly Servers**. - -For each Gitaly server, the following details are listed: - -| Field | Description | -|----------------|-------------| -| Storage | Repository storage | -| Address | Network address on which the Gitaly server is listening | -| Server version | Gitaly version | -| Git version | Version of Git installed on the Gitaly server | -| Up to date | Indicates if the Gitaly server version is the latest version available. A green dot indicates the server is up to date. | - -## CI/CD section - -### Administering runners - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/340859) from **Overview > Runners** to **CI/CD > Runners** in GitLab 15.8. - -You can administer all runners in the GitLab instance from the Admin Area's **Runners** page. See -[GitLab Runner](https://docs.gitlab.com/runner/) for more information. - -To access the **Runners** page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Runners**. - -#### Search and filter runners - -To search runners' descriptions: - -1. In the **Search or filter results...** field, type the description of the runner you want to - find. -1. Press <kbd>Enter</kbd>. - -You can also filter runners by status, type, and tag. To filter: - -1. Select a tab or the **Search or filter results...** field. -1. Select any **Type**, or filter by **Status** or **Tags**. -1. Select or enter your search criteria. - - - -#### Bulk delete runners - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/370241) in GitLab 15.4. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/353981) in GitLab 15.5. - -You can delete multiple runners at the same time. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Runners**. -1. To the left of the runners you want to delete, select the checkbox. - To select all of the runners on the page, select the checkbox above - the list. -1. Select **Delete selected**. - -#### Runner attributes - -For each runner, the following attributes are listed: - -| Attribute | Description | -|--------------|-------------| -| Status | The status of the runner. In [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/22224), for the **Ultimate** tier, the upgrade status is available. | -| Runner details | Information about the runner, including partial token and details about the computer the runner was registered from. | -| Version | GitLab Runner version. | -| Jobs | Total number of jobs run by the runner. | -| Tags | Tags associated with the runner. | -| Last contact | Timestamp indicating when the runner last contacted the GitLab instance. | - -You can also edit, pause, or remove each runner. - -### Administering Jobs - -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/386311) from **Overview > Jobs** to **CI/CD > Jobs** in GitLab 15.8. - -You can administer all jobs in the GitLab instance from the Admin Area's Jobs page. - -To access the Jobs page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **CI/CD > Jobs**. All jobs are listed, in descending order of job ID. -1. Select the **All** tab to list all jobs. Select the **Pending**, **Running**, or **Finished** - tab to list only jobs of that status. - -For each job, the following details are listed: - -| Field | Description | -|----------|-------------| -| Status | Job status, either **passed**, **skipped**, or **failed**. | -| Job | Includes links to the job, branch, and the commit that started the job. | -| Pipeline | Includes a link to the specific pipeline. | -| Project | Name of the project, and organization, to which the job belongs. | -| Runner | Name of the CI runner assigned to execute the job. | -| Stage | Stage that the job is declared in a `.gitlab-ci.yml` file. | -| Name | Name of the job specified in a `.gitlab-ci.yml` file. | -| Timing | Duration of the job, and how long ago the job completed. | -| Coverage | Percentage of tests coverage. | - -## Monitoring section - -The following topics document the **Monitoring** section of the Admin Area. - -### System Information - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341248) in GitLab 15.2, support for relative time. "Uptime" statistic was renamed to "System started". - -The **System Info** page provides the following statistics: - -| Field | Description | -|:---------------|:--------------------------------------------------| -| CPU | Number of CPU cores available | -| Memory Usage | Memory in use, and total memory available | -| Disk Usage | Disk space in use, and total disk space available | -| System started | When the system hosting GitLab was started. In GitLab 15.1 and earlier, this was an uptime statistic. | - -These statistics are updated only when you navigate to the **System Info** page, or you refresh the page in your browser. - -### Background Jobs - -The **Background Jobs** page displays the Sidekiq dashboard. Sidekiq is used by GitLab to -perform processing in the background. - -The Sidekiq dashboard consists of the following elements: - -- A tab per jobs' status. -- A breakdown of background job statistics. -- A live graph of **Processed** and **Failed** jobs, with a selectable polling interval. -- An historical graph of **Processed** and **Failed** jobs, with a selectable time span. -- Redis statistics, including: - - Version number - - Uptime, measured in days - - Number of connections - - Current memory usage, measured in MB - - Peak memory usage, measured in MB - -### Logs - -Since GitLab 13.0, **Log** view has been removed from the Admin Area dashboard since the logging does not work in multi-node setups and could cause confusion for administrators by displaying partial information. - -For multi-node systems we recommend ingesting the logs into services like Elasticsearch and Splunk. - -| Log file | Contents | -|:------------------------|:---------| -| `application_json.log` | GitLab user activity | -| `git_json.log` | Failed GitLab interaction with Git repositories | -| `production.log` | Requests received from Puma, and the actions taken to serve those requests | -| `sidekiq.log` | Background jobs | -| `repocheck.log` | Repository activity | -| `integrations_json.log` | Activity between GitLab and integrated systems | -| `kubernetes.log` | Kubernetes activity | - -The contents of these log files can be useful when troubleshooting a problem. - -For details of these log files and their contents, see [Log system](../../administration/logs/index.md). - -The content of each log file is listed in chronological order. To minimize performance issues, a maximum 2000 lines of each log file are shown. - -### Audit Events **(PREMIUM SELF)** - -The **Audit Events** page lists changes made within the GitLab server. With this information you can control, analyze, and track every change. +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index 16721d144e5..724c9885801 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -1,28 +1,11 @@ --- -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../administration/labels.md' +remove_date: '2023-10-05' --- -# Labels administration **(FREE SELF)** +This document was moved to [another location](../../administration/labels.md). -To manage labels for the GitLab instance, select **Labels** (**{labels}**) from the Admin Area sidebar. For more details on how to manage labels, see [Labels](../project/labels.md). - -Labels created in the Admin Area are automatically added to new projects. -They are not available to new groups. -Updating or adding labels in the Admin Area does not modify labels in existing projects. - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-05>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index be6478de2a0..636f5cb2831 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -1,83 +1,11 @@ --- -stage: Fulfillment -group: Provision -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/license.md' +remove_date: '2023-10-11' --- -# Activate GitLab Enterprise Edition (EE) **(PREMIUM SELF)** +This document was moved to [another location](../../administration/license.md). -When you install a new GitLab instance without a license, only Free features -are enabled. To enable more features in GitLab Enterprise Edition (EE), activate -your instance with an activation code. - -## Activate GitLab EE - -In GitLab Enterprise Edition 14.1 and later, you need an activation code to activate -your instance. - -Prerequisite: - -- You must [purchase a subscription](https://about.gitlab.com/pricing/). -- You must be running GitLab Enterprise Edition (EE). -- You must have GitLab 14.1 or later. -- Your instance must be connected to the internet. - -To activate your instance with an activation code: - -1. Copy the activation code, a 24-character alphanumeric string, from either: - - Your subscription confirmation email. - - The [Customers Portal](https://customers.gitlab.com/customers/sign_in), on the **Manage Purchases** page. -1. Sign in to your GitLab self-managed instance. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Subscription**. -1. Paste the activation code in **Activation code**. -1. Read and accept the terms of service. -1. Select **Activate**. - -The subscription is activated. - -If you have an offline environment, -[activate GitLab EE with a license file or key](license_file.md) instead. - -If you have questions or need assistance activating your instance, -[contact GitLab Support](https://about.gitlab.com/support/#contact-support). - -When [the license expires](license_file.md#what-happens-when-your-license-expires), -some functionality is locked. - -## Verify your GitLab edition - -To verify the edition, sign in to GitLab and select -**Help** (**{question-o}**) > **Help**. The GitLab edition and version are listed -at the top of the page. - -If you are running GitLab Community Edition, you can upgrade your installation to GitLab -EE. For more details, see [Upgrading between editions](../../update/index.md#upgrading-between-editions). -If you have questions or need assistance upgrading from GitLab Community Edition (CE) to EE, -[contact GitLab Support](https://about.gitlab.com/support/#contact-support). - -## Troubleshooting - -### Cannot activate instance due to connectivity error - -This error occurs when you use an activation code to activate your instance, but your instance is unable to connect to the GitLab servers. - -You may have connectivity issues due to the following reasons: - -- **You have an offline environment**: - - Configure your setup to allow connection to GitLab servers. If connection to GitLab servers is not possible, contact your Sales Representative to request a license key. You can also contact [GitLab support](https://about.gitlab.com/support/#contact-support) if you need help finding your Sales Representative. -- **Customers Portal is not operational**: - - To check for performance or service disruptions, check the Customers Portal [status](https://status.gitlab.com/). -- **Firewall settings**: - - Check if your GitLab instance has an encrypted connection to `customers.gitlab.com` (with IP addresses 172.64.146.11 and 104.18.41.245) on port 443: - - ```shell - curl --verbose "https://customers.gitlab.com/" - ``` - - - If the curl command returns a failure, either: - - [Configure a proxy](https://docs.gitlab.com/omnibus/settings/environment-variables.html) in `gitlab.rb` to point to your server. - - Contact your network administrator to make changes to the proxy. - - If an SSL inspection appliance is used, you must add the appliance's root CA certificate to `/etc/gitlab/trusted-certs` on the server, then run `gitlab-ctl reconfigure`. -
\ No newline at end of file +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md index 12e908b4fe0..4835d656cfa 100644 --- a/doc/user/admin_area/license_file.md +++ b/doc/user/admin_area/license_file.md @@ -1,252 +1,11 @@ --- -stage: Fulfillment -group: Provision -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/license_file.md' +remove_date: '2023-10-11' --- -<!-- To promote the workflow described in license.md, this page is not included in global left nav. --> +This document was moved to [another location](../../administration/license_file.md). -# Activate GitLab EE with a license file or key - -If you receive a license file from GitLab (for example, for a trial), you can -upload it to your instance or add it during installation. The license file is -a base64-encoded ASCII text file with a `.gitlab-license` extension. - -The first time you sign in to your GitLab instance, a note with a -link to the **Add license** page should be displayed. - -Otherwise, to add your license: - -1. Sign in to GitLab as an administrator. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. In the **Add License** area, add a license by either uploading the file or entering the key. -1. Select the **Terms of Service** checkbox. -1. Select **Add license**. - -NOTE: -In GitLab 14.7.x to 14.9.x, you can add the license file with the UI. -In GitLab 14.1.x to 14.7, if you have already activated your subscription with an activation code, you cannot access **Add License** from the Admin Area. You must access **Add License** directly from the URL, `<YourGitLabURL>/admin/license/new`. - -## Activate subscription during installation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114572) in GitLab 16.0. - -To activate your subscription during installation, set the `GITLAB_ACTIVATION_CODE` environment variable with the activation code: - -```shell -export GITLAB_ACTIVATION_CODE=your_activation_code -``` - -## Add license file during installation - -If you have a license, you can also import it when you install GitLab. - -- For self-compiled installations: - - Place the `Gitlab.gitlab-license` file in the `config/` directory. - - To specify a custom location and filename for the license, set the - `GITLAB_LICENSE_FILE` environment variable with the path to the file: - - ```shell - export GITLAB_LICENSE_FILE="/path/to/license/file" - ``` - -- For Linux package installations: - - Place the `Gitlab.gitlab-license` file in the `/etc/gitlab/` directory. - - To specify a custom location and filename for the license, add this entry to `gitlab.rb`: - - ```ruby - gitlab_rails['initial_license_file'] = "/path/to/license/file" - ``` - -WARNING: -These methods only add a license at the time of installation. To renew or upgrade -a license, add the license in the **Admin Area** in the web user interface. - -## Submit license usage data - -If you use a license file or key to activate your instance in an offline environment, you must submit your license -usage data monthly. -To submit the data, [export your license usage](../../subscriptions/self_managed/index.md#export-your-license-usage) -and send it by email to the renewals service, `renewals-service@customers.gitlab.com`. - -If you don't submit your data each month after your subscription start date, an email is sent to the address -associated with your subscription and a banner displays to remind you to submit your data. The banner displays -in the **Admin Area** on the **Dashboard** and on the **Subscription** pages. You can only dismiss it until the -following month after you submit your license usage data. - -## What happens when your license expires - -Fifteen days before the license expires, a notification banner with the upcoming expiration -date displays to GitLab administrators. - -When your license expires, GitLab locks features, like Git pushes -and issue creation. Your instance becomes read-only and -an expiration message displays to all administrators. You have a 14-day grace period -before this occurs. - -To resume functionality, [renew your subscription](../../subscriptions/self_managed/index.md#renew-subscription-manually). - -If the license has been expired for more than 30 days, you must purchase a [new subscription](../../subscriptions/self_managed/index.md) to resume functionality. - -To go back to Free features, [delete all expired licenses](#remove-a-license). - -## Remove a license - -To remove a license from a self-managed instance: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Subscription**. -1. Select **Remove license**. - -Repeat these steps to remove all licenses, including those applied in the past. - -## View license details and history - -To view your license details: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Subscription**. - -You can add and view more than one license, but only the latest license in -the current date range is the active license. - -When you add a future-dated license, it doesn't take effect until its applicable date. -You can view all active subscriptions in the **Subscription history** table. - -You can also [export](../../subscriptions/self_managed/index.md) your license usage information to a CSV file. - -NOTE: -In GitLab 13.6 and earlier, a banner about an expiring license may continue to display -when you add a new license. This happens when the start date of the new license -is in the future and the expiring one is still active. -The banner disappears after the new license becomes active. - -## Troubleshooting - -### No Subscription area in the Admin Area - -You cannot add your license because there is no **Subscription** area. -This issue might occur if: - -- You're running GitLab Community Edition. Before you add your license, you - must [upgrade to Enterprise Edition](../../update/index.md#community-to-enterprise-edition). -- You're using GitLab.com. You cannot add a self-managed license to GitLab.com. - To use paid features on GitLab.com, [purchase a separate subscription](../../subscriptions/gitlab_com/index.md). - -### Users exceed license limit upon renewal - -GitLab displays a message prompting you to purchase -additional users. This issue occurs if you add a license that does not have enough -users to cover the number of users in your instance. - -To fix this issue, purchase additional seats to cover those users. -For more information, read the [licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). - -In GitLab 14.2 and later, for instances that use a license file, the following -rules apply: - -- If the users over license are less than or equal to 10% of the users in the license - file, the license is applied and you pay the overage in the next renewal. -- If the users over license are more than 10% of the users in the license file, - you cannot apply the license without purchasing more users. - -For example, if you purchase a license for 100 users, you can have 110 users when you add -your license. However, if you have 111 users, you must purchase more users before you can add -the license. - -### `Start GitLab Ultimate trial` still displays after adding license - -To fix this issue, restart [Puma or your entire GitLab instance](../../administration/restart_gitlab.md). - -### License commands in the rails console - -The following commands can be run in the [rails console](../../administration/operations/rails_console.md#starting-a-rails-console-session). - -WARNING: -Any command that changes data directly could be damaging if not run correctly, or under the right conditions. -We highly recommend running them in a test environment with a backup of the instance ready to be restored, just in case. - -#### See current license information - -```ruby -# License information (name, company, email address) -License.current.licensee - -# Plan: -License.current.plan - -# Uploaded: -License.current.created_at - -# Started: -License.current.starts_at - -# Expires at: -License.current.expires_at - -# Is this a trial license? -License.current.trial? - -# License ID for lookup on CustomersDot -License.current.license_id - -# License data in Base64-encoded ASCII format -License.current.data - -# Confirm the current billable seat count excluding guest users. This is useful for customers who use an Ultimate subscription tier where Guest seats are not counted. -User.active.without_bots.excluding_guests.count - -``` - -#### Interaction with licenses that start in the future - -```ruby -# Future license data follows the same format as current license data it just uses a different modifier for the License prefix -License.future_dated -``` - -#### Check if a project feature is available on the instance - -Features listed in [`features.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/gitlab_subscriptions/features.rb). - -```ruby -License.current.feature_available?(:jira_dev_panel_integration) -``` - -#### Check if a project feature is available in a project - -Features listed in [`features.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/gitlab_subscriptions/features.rb). - -```ruby -p = Project.find_by_full_path('<group>/<project>') -p.feature_available?(:jira_dev_panel_integration) -``` - -#### Add a license through the console - -```ruby -key = "<key>" -license = License.new(data: key) -license.save -License.current # check to make sure it applied -``` - -This is needed for example in a known edge-case with -[expired license and multiple LDAP servers](../../administration/auth/ldap/ldap-troubleshooting.md#expired-license-causes-errors-with-multiple-ldap-servers). - -#### Remove licenses - -To clean up the [License History table](../../user/admin_area/license_file.md#view-license-details-and-history): - -```ruby -TYPE = :trial? -# or :expired? - -License.select(&TYPE).each(&:destroy!) - -# or even License.all.each(&:destroy!) -``` +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/merge_requests_approvals.md b/doc/user/admin_area/merge_requests_approvals.md index 58f54c399df..de079d08d3a 100644 --- a/doc/user/admin_area/merge_requests_approvals.md +++ b/doc/user/admin_area/merge_requests_approvals.md @@ -1,43 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference, concepts +redirect_to: '../../administration/merge_requests_approvals.md' +remove_date: '2023-10-12' --- -# Merge request approvals **(PREMIUM SELF)** +This document was moved to [another location](../../administration/merge_requests_approvals.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/39060) in GitLab 12.8. - -Merge request approval rules prevent users from overriding certain settings on the project level. -When enabled at the instance level, these settings [cascade](../project/merge_requests/approvals/settings.md#settings-cascading) -and can no longer be changed: - -- In projects. -- In groups. Cascading to groups was [enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) - in GitLab 14.5. - -To enable merge request approval settings for an instance: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Push Rules**. -1. Expand **Merge request approvals**. -1. Choose the required options. -1. Select **Save changes**. - -## Available rules - -Merge request approval settings that can be set at an instance level are: - -- **Prevent approval by author**. Prevents project maintainers from allowing request authors to - merge their own merge requests. -- **Prevent approvals by users who add commits**. Prevents project maintainers from allowing users - to approve merge requests if they have submitted any commits to the source branch. -- **Prevent editing approval rules in projects and merge requests**. Prevents users from modifying - the approvers list in project settings or in individual merge requests. - -See also the following, which are affected by instance-level rules: - -- [Project merge request approval rules](../project/merge_requests/approvals/index.md). -- [Group merge request approval settings](../group/manage.md#group-merge-request-approval-settings) available in GitLab 13.9 and later. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/moderate_users.md b/doc/user/admin_area/moderate_users.md index ee6d360ac1d..3390c4791be 100644 --- a/doc/user/admin_area/moderate_users.md +++ b/doc/user/admin_area/moderate_users.md @@ -1,360 +1,11 @@ --- -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto +redirect_to: '../../administration/moderate_users.md' +remove_date: '2023-10-12' --- -# Moderate users (administration) **(FREE SELF)** +This document was moved to [another location](../../administration/moderate_users.md). -This is the administration documentation. For information about moderating users at the group level, see the [group-level documentation](../group/moderate_users.md). - -GitLab administrators can moderate user access by approving, blocking, banning, or deactivating -users. - -## Users pending approval - -A user in _pending approval_ state requires action by an administrator. A user sign up can be in a -pending approval state because an administrator has enabled any of the following options: - -- [Require administrator approval for new sign-ups](settings/sign_up_restrictions.md#require-administrator-approval-for-new-sign-ups) setting. -- [User cap](settings/sign_up_restrictions.md#user-cap). -- [Block auto-created users (OmniAuth)](../../integration/omniauth.md#configure-common-settings) -- [Block auto-created users (LDAP)](../../administration/auth/ldap/index.md#basic-configuration-settings) - -When a user registers for an account while this setting is enabled: - -- The user is placed in a **Pending approval** state. -- The user sees a message telling them their account is awaiting approval by an administrator. - -A user pending approval: - -- Is functionally identical to a [blocked](#block-a-user) user. -- Cannot sign in. -- Cannot access Git repositories or the GitLab API. -- Does not receive any notifications from GitLab. -- Does not consume a [seat](../../subscriptions/self_managed/index.md#billable-users). - -An administrator must [approve their sign up](#approve-or-reject-a-user-sign-up) to allow them to -sign in. - -### View user sign ups pending approval - -To view user sign ups pending approval: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Pending approval** tab. - -### Approve or reject a user sign up - -A user sign up pending approval can be approved or rejected from the Admin Area. - -To approve or reject a user sign up: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Pending approval** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Approve** or **Reject**. - -Approving a user: - -- Activates their account. -- Changes the user's state to active. -- Consumes a subscription [seat](../../subscriptions/self_managed/index.md#billable-users). - -## Block and unblock users - -GitLab administrators can block and unblock users. - -### Block a user - -To completely prevent access of a user to the GitLab instance, -administrators can choose to block the user. - -Users can be blocked [via an abuse report](review_abuse_reports.md#blocking-users), -by removing them in LDAP, or directly from the Admin Area. To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Block**. - -A blocked user: - -- Cannot sign in. -- Cannot access Git repositories or the API. -- Does not receive any notifications from GitLab. -- Cannot use [slash commands](../../integration/slash_commands.md). -- Does not consume a [seat](../../subscriptions/self_managed/index.md#billable-users). - -Personal projects, and group and user history of the blocked user are left intact. - -NOTE: -Users can also be blocked using the [GitLab API](../../api/users.md#block-user). - -### Unblock a user - -A blocked user can be unblocked from the Admin Area. To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Blocked** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Unblock**. - -The user's state is set to active and they consume a -[seat](../../subscriptions/self_managed/index.md#billable-users). - -NOTE: -Users can also be unblocked using the [GitLab API](../../api/users.md#unblock-user). - -The unblock option may be unavailable for LDAP users. To enable the unblock option, -the LDAP identity first needs to be deleted: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Blocked** tab. -1. Select a user. -1. Select the **Identities** tab. -1. Find the LDAP provider and select **Delete**. - -## Activate and deactivate users - -GitLab administrators can deactivate and activate users. - -### Deactivate a user - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22257) in GitLab 12.4. - -To temporarily prevent access by a GitLab user that has no recent activity, -administrators can choose to deactivate the user. - -Deactivating a user is functionally identical to [blocking a user](#block-and-unblock-users), -with the following differences: - -- It does not prohibit the user from logging back in via the UI. -- Once a deactivated user logs back into the GitLab UI, their account is set to active. - -A deactivated user: - -- Cannot access Git repositories or the API. -- Does not receive any notifications from GitLab. -- Cannot use [slash commands](../../integration/slash_commands.md). -- Does not consume a [seat](../../subscriptions/self_managed/index.md#billable-users). - -Personal projects, and group and user history of the deactivated user are left intact. - -NOTE: -Users are notified about account deactivation if -[user deactivation emails](settings/email.md#user-deactivation-emails) are enabled. - -A user can be deactivated from the Admin Area. To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Deactivate**. - -For the deactivation option to be visible to an administrator, the user: - -- Must have a state of active. -- Must be [dormant](#automatically-deactivate-dormant-users). - -NOTE: -Users can also be deactivated using the [GitLab API](../../api/users.md#deactivate-user). - -### Automatically deactivate dormant users - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320875) in GitLab 14.0. -> - Customizable time period [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336747) in GitLab 15.4 -> - The lower limit for inactive period set to 90 days [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/100793) in GitLab 15.5 - -Administrators can enable automatic deactivation of users who either: - -- Were created more than a week ago and have not signed in. -- Have no activity for a specified period of time (default and minimum is 90 days). - -To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. -1. Under **Dormant users**, check **Deactivate dormant users after a period of inactivity**. -1. Under **Days of inactivity before deactivation**, enter the number of days before deactivation. Minimum value is 90 days. -1. Select **Save changes**. - -When this feature is enabled, GitLab runs a job once a day to deactivate the dormant users. - -A maximum of 100,000 users can be deactivated per day. - -### Activate a user - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22257) in GitLab 12.4. - -A deactivated user can be activated from the Admin Area. - -To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Deactivated** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Activate**. - -The user's state is set to active and they consume a -[seat](../../subscriptions/self_managed/index.md#billable-users). - -NOTE: -A deactivated user can also activate their account themselves by logging back in via the UI. -Users can also be activated using the [GitLab API](../../api/users.md#activate-user). - -## Ban and unban users - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327353) in GitLab 14.2 [with a flag](../../administration/feature_flags.md) named `ban_user_feature_flag`. Disabled by default. -> - Ban and unban users [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/327353) in GitLab 14.8. Feature flag `ban_user_feature_flag` removed. -> - Hiding merge requests of banned users [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107836) in GitLab 15.8 [with a flag](../../administration/feature_flags.md) named `hide_merge_requests_from_banned_users`. Disabled by default. -> - Hiding comments of banned users [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112973) in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `hidden_notes`. Disabled by default. - -GitLab administrators can ban and unban users. Banned users are blocked, and their issues, merge requests, and comments are hidden. - -### Ban a user - -To block a user and hide their contributions, administrators can ban the user. - -Users can be banned using the Admin Area. To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Ban user**. - -The banned user does not consume a [seat](../../subscriptions/self_managed/index.md#billable-users). - -### Unban a user - -A banned user can be unbanned using the Admin Area. To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Banned** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Unban user**. - -The user's state is set to active and they consume a -[seat](../../subscriptions/self_managed/index.md#billable-users). - -### Delete a user - -Use the Admin Area to delete users. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Banned** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Delete user**. -1. Type the username. -1. Select **Delete user**. - -NOTE: -You can only delete a user if there are inherited or direct owners of a group. You cannot delete a user if they are the only group owner. - -You can also delete a user and their contributions, such as merge requests, issues, and groups of which they are the only group owner. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Banned** tab. -1. Optional. Select a user. -1. Select the **{settings}** **User administration** dropdown list. -1. Select **Delete user and contributions**. -1. Type the username. -1. Select **Delete user and contributions**. - -NOTE: -Before 15.1, additionally groups of which deleted user were the only owner among direct members were deleted. - -## Troubleshooting - -When moderating users, you may need to perform bulk actions on them based on certain conditions. The following rails console scripts show some examples of this. You may [start a rails console session](../../administration/operations/rails_console.md#starting-a-rails-console-session) and use scripts similar to the following: - -### Deactivate users that have no recent activity - -Administrators can deactivate users that have no recent activity. - -WARNING: -Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. - -```ruby -days_inactive = 90 -inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago) - -inactive_users.each do |user| - puts "user '#{user.username}': #{user.last_activity_on}" - user.deactivate! -end -``` - -### Block users that have no recent activity - -Administrators can block users that have no recent activity. - -WARNING: -Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. - -```ruby -days_inactive = 90 -inactive_users = User.active.where("last_activity_on <= ?", days_inactive.days.ago) - -inactive_users.each do |user| - puts "user '#{user.username}': #{user.last_activity_on}" - user.block! -end -``` - -### Block or delete users that have no projects or groups - -Administrators can block or delete users that have no projects or groups. - -WARNING: -Commands that change data can cause damage if not run correctly or under the right conditions. Always run commands in a test environment first and have a backup instance ready to restore. - -```ruby -users = User.where('id NOT IN (select distinct(user_id) from project_authorizations)') - -# How many users are removed? -users.count - -# If that count looks sane: - -# You can either block the users: -users.each { |user| user.blocked? ? nil : user.block! } - -# Or you can delete them: - # need 'current user' (your user) for auditing purposes -current_user = User.find_by(username: '<your username>') - -users.each do |user| - DeleteUserWorker.perform_async(current_user.id, user.id) -end -``` +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index f3b09c61532..bda326d4daf 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -1,146 +1,11 @@ --- -stage: Monitor -group: Respond -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/monitoring/health_check.md' +remove_date: '2023-10-12' --- -# Health Check **(FREE SELF)** +This document was moved to [another location](../../../administration/monitoring/health_check.md). -GitLab provides liveness and readiness probes to indicate service health and -reachability to required services. These probes report on the status of the -database connection, Redis connection, and access to the file system. These -endpoints [can be provided to schedulers like Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) to hold -traffic until the system is ready or restart the container as needed. - -## IP allowlist - -To access monitoring resources, the requesting client IP needs to be included in the allowlist. -For details, see [how to add IPs to the allowlist for the monitoring endpoints](../../../administration/monitoring/ip_allowlist.md). - -## Using the endpoints locally - -With default allowlist settings, the probes can be accessed from localhost using the following URLs: - -```plaintext -GET http://localhost/-/health -``` - -```plaintext -GET http://localhost/-/readiness -``` - -```plaintext -GET http://localhost/-/liveness -``` - -## Health - -Checks whether the application server is running. -It does not verify the database or other services -are running. This endpoint circumvents Rails Controllers -and is implemented as additional middleware `BasicHealthCheck` -very early into the request processing lifecycle. - -```plaintext -GET /-/health -``` - -Example request: - -```shell -curl "https://gitlab.example.com/-/health" -``` - -Example response: - -```plaintext -GitLab OK -``` - -## Readiness - -The readiness probe checks whether the GitLab instance is ready -to accept traffic via Rails Controllers. The check by default -does validate only instance-checks. - -If the `all=1` parameter is specified, the check also validates -the dependent services (Database, Redis, Gitaly etc.) -and gives a status for each. - -```plaintext -GET /-/readiness -GET /-/readiness?all=1 -``` - -Example request: - -```shell -curl "https://gitlab.example.com/-/readiness" -``` - -Example response: - -```json -{ - "master_check":[{ - "status":"failed", - "message": "unexpected Master check result: false" - }], - ... -} -``` - -On failure, the endpoint returns a `503` HTTP status code. - -This check is being exempt from Rack Attack. - -## Liveness - -WARNING: -In GitLab [12.4](https://about.gitlab.com/upcoming-releases/) -the response body of the Liveness check was changed -to match the example below. - -Checks whether the application server is running. -This probe is used to know if Rails Controllers -are not deadlocked due to a multi-threading. - -```plaintext -GET /-/liveness -``` - -Example request: - -```shell -curl "https://gitlab.example.com/-/liveness" -``` - -Example response: - -On success, the endpoint returns a `200` HTTP status code, and a response like below. - -```json -{ - "status": "ok" -} -``` - -On failure, the endpoint returns a `503` HTTP status code. - -This check is being exempt from Rack Attack. - -## Sidekiq - -Learn how to configure the [Sidekiq health checks](../../../administration/sidekiq/sidekiq_health_check.md). - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/reporting/git_abuse_rate_limit.md b/doc/user/admin_area/reporting/git_abuse_rate_limit.md index 38aaeb8eb55..0e607b03b82 100644 --- a/doc/user/admin_area/reporting/git_abuse_rate_limit.md +++ b/doc/user/admin_area/reporting/git_abuse_rate_limit.md @@ -1,49 +1,11 @@ --- -stage: Anti-Abuse -group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/reporting/git_abuse_rate_limit.md' +remove_date: '2023-10-12' --- -# Git abuse rate limit (administration) **(ULTIMATE SELF)** +This document was moved to [another location](../../../administration/reporting/git_abuse_rate_limit.md). -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `git_abuse_rate_limit_feature_flag`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/394996) in GitLab 15.11. Feature flag `git_abuse_rate_limit_feature_flag` removed. - -This is the administration documentation. For information about Git abuse rate limiting at the group level, see the [group-level documentation](../../group/reporting/git_abuse_rate_limit.md). - -Git abuse rate limiting is a feature to automatically [ban users](../moderate_users.md#ban-and-unban-users) who download, clone, or fork more than a specified number of repositories in any project in the instance in a given time frame. Banned users cannot sign in to the instance and cannot access any non-public group via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). - -Git abuse rate limiting does not apply to instance administrators, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). - -How GitLab determines a user's rate limit is under development. -GitLab team members can view more information in this confidential epic: -`https://gitlab.com/groups/gitlab-org/modelops/anti-abuse/-/epics/14`. - -## Configure Git abuse rate limiting - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Reporting**. -1. Expand **Git abuse rate limit**. -1. Update the Git abuse rate limit settings: - 1. Enter a number in the **Number of repositories** field, greater than or equal to `0` and less than or equal to `10,000`. This number specifies the maximum amount of unique repositories a user can download in the specified time period before they're banned. When set to `0`, Git abuse rate limiting is disabled. - 1. Enter a number in the **Reporting time period (seconds)** field, greater than or equal to `0` and less than or equal to `86,400` (10 days). This number specifies the time in seconds a user can download the maximum amount of repositories before they're banned. When set to `0`, Git abuse rate limiting is disabled. - 1. Optional. Exclude up to `100` users by adding them to the **Excluded users** field. Excluded users are not automatically banned. - 1. Add up to `100` users to the **Send notifications to** field. You must select at least one user. All application administrators are selected by default. - 1. Optional. Turn on the **Automatically ban users from this namespace when they exceed the specified limits** toggle to enable automatic banning. -1. Select **Save changes**. - -## Automatic ban notifications - -If automatic banning is disabled, a user is not banned automatically when they exceed the limit. However, notifications are still sent to the users listed under **Send notifications to**. You can use this setup to determine the correct values of the rate limit settings before enabling automatic banning. - -If automatic banning is enabled, an email notification is sent when a user is about to be banned, and the user is automatically banned from the GitLab instance. - -## Unban a user - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Banned** tab and search for the account you want to unban. -1. From the **User administration** dropdown list select **Unban user**. -1. On the confirmation dialog, select **Unban user**. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/reporting/ip_addr_restrictions.md b/doc/user/admin_area/reporting/ip_addr_restrictions.md index 5b749c62c30..85783640b28 100644 --- a/doc/user/admin_area/reporting/ip_addr_restrictions.md +++ b/doc/user/admin_area/reporting/ip_addr_restrictions.md @@ -1,33 +1,11 @@ --- -stage: Anti-Abuse -group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/reporting/ip_addr_restrictions.md' +remove_date: '2023-10-13' --- -# IP address restrictions **(FREE SELF)** +This document was moved to [another location](../../../administration/reporting/ip_addr_restrictions.md). -IP address restrictions help prevent malicious users hiding their activities behind multiple IP addresses. - -GitLab maintains a list of the unique IP addresses used by a user to make requests over a specified period. When the -specified limit is reached, any requests made by the user from a new IP address are rejected with a `403 Forbidden` error. - -IP addresses are cleared from the list when no further requests have been made by the user from the IP address in the specified time period. - -NOTE: -When a runner runs a CI/CD job as a particular user, the runner IP address is also stored against the user's list of -unique IP addresses. Therefore, the IP addresses per user limit should take into account the number of configured active runners. - -## Configure IP address restrictions - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Reporting**. -1. Expand **Spam and Anti-bot Protection**. -1. Update the IP address restrictions settings: - 1. Select the **Limit sign in from multiple IP addresses** checkbox to enable IP address restrictions. - 1. Enter a number in the **IP addresses per user** field, greater than or equal to `1`. This number specifies the - maximum number of unique IP addresses a user can access GitLab from in the specified time period before requests - from a new IP address are rejected. - 1. Enter a number in the **IP address expiration time** field, greater than or equal to `0`. This number specifies the - time in seconds an IP address counts towards the limit for a user, taken from the time the last request was made. -1. Select **Save changes**. +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/reporting/spamcheck.md b/doc/user/admin_area/reporting/spamcheck.md index e2508476c80..043481b6255 100644 --- a/doc/user/admin_area/reporting/spamcheck.md +++ b/doc/user/admin_area/reporting/spamcheck.md @@ -1,69 +1,11 @@ --- -stage: Systems -group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/reporting/spamcheck.md' +remove_date: '2023-10-13' --- -# Spamcheck anti-spam service **(FREE SELF)** +This document was moved to [another location](../../../administration/reporting/spamcheck.md). -> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259) in GitLab 14.8. - -WARNING: -Spamcheck is available to all tiers, but only on instances using GitLab Enterprise Edition (EE). For [licensing reasons](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259#note_726605397), it is not included in the GitLab Community Edition (CE) package. You can [migrate from CE to EE](../../../update/package/convert_to_ee.md). - -[Spamcheck](https://gitlab.com/gitlab-org/spamcheck) is an anti-spam engine -developed by GitLab originally to combat rising amount of spam in GitLab.com, -and later made public to be used in self-managed GitLab instances. - -## Enable Spamcheck - -Spamcheck is only available for package-based installations: - -1. Edit `/etc/gitlab/gitlab.rb` and enable Spamcheck: - - ```ruby - spamcheck['enable'] = true - ``` - -1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -1. Verify that the new services `spamcheck` and `spam-classifier` are - up and running: - - ```shell - sudo gitlab-ctl status - ``` - -## Configure GitLab to use Spamcheck - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Reporting**. -1. Expand **Spam and Anti-bot Protection**. -1. Update the Spam Check settings: - 1. Check the "Enable Spam Check via external API endpoint" checkbox. - 1. For **URL of the external Spam Check endpoint** use `grpc://localhost:8001`. - 1. Leave **Spam Check API key** blank. -1. Select **Save changes**. - -NOTE: -In single-node instances, Spamcheck runs over `localhost`, and hence is running -in an unauthenticated mode. If on multi-node instances where GitLab runs on one -server and Spamcheck runs on another server listening over a public endpoint, it -is recommended to enforce some sort of authentication using a reverse proxy in -front of the Spamcheck service that can be used along with an API key. One -example would be to use `JWT` authentication for this and specifying a bearer -token as the API key. -[Native authentication for Spamcheck is in the works](https://gitlab.com/gitlab-com/gl-security/engineering-and-research/automation-team/spam/spamcheck/-/issues/171). - -## Running Spamcheck over TLS - -Spamcheck service on its own cannot communicate directly over TLS with GitLab. -However, Spamcheck can be deployed behind a reverse proxy which performs TLS -termination. In such a scenario, GitLab can be made to communicate with -Spamcheck over TLS by specifying `tls://` scheme for the external Spamcheck URL -instead of `grpc://` in the Admin Area settings. +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/review_abuse_reports.md b/doc/user/admin_area/review_abuse_reports.md index 5cd63ec964c..0e01025896e 100644 --- a/doc/user/admin_area/review_abuse_reports.md +++ b/doc/user/admin_area/review_abuse_reports.md @@ -1,96 +1,11 @@ --- -stage: Anti-Abuse -group: Anti-Abuse -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference, howto +redirect_to: '../../administration/review_abuse_reports.md' +remove_date: '2023-10-11' --- -# Review abuse reports **(FREE SELF)** +This document was moved to [another location](../../administration/review_abuse_reports.md). -View and resolve abuse reports from GitLab users. - -GitLab administrators can view and [resolve](#resolving-abuse-reports) abuse -reports in the Admin Area. - -## Receive notification of abuse reports by email - -To receive notifications of new abuse reports by email: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Reporting**. -1. Expand the **Abuse reports** section. -1. Provide an email address and select **Save changes**. - -The notification email address can also be set and retrieved -[using the API](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls). - -## Reporting abuse - -To find out more about reporting abuse, see -[abuse reports user documentation](../report_abuse.md). - -## Resolving abuse reports - -To access abuse reports: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Abuse Reports**. - -There are 3 ways to resolve an abuse report, with a button for each method: - -- Remove user & report. This: - - [Deletes the reported user](../profile/account/delete_account.md) from the - instance. - - Removes the abuse report from the list. -- [Block user](#blocking-users). -- Remove report. This: - - Removes the abuse report from the list. - - Removes access restrictions for the reported user. - -The following is an example of the **Abuse Reports** page: - - - -### Blocking users - -A blocked user cannot sign in or access any repositories, but all of their data -remains. - -Blocking a user: - -- Leaves them in the abuse report list. -- Changes the **Block user** button to a disabled **Already blocked** button. - -The user is notified with the following message: - -```plaintext -Your account has been blocked. If you believe this is in error, contact a staff member. -``` - -After blocking, you can still either: - -- Remove the user and report if necessary. -- Remove the report. - -The following is an example of a blocked user listed on the **Abuse Reports** -page: - - - -NOTE: -Users can be [blocked](../../api/users.md#block-user) and -[unblocked](../../api/users.md#unblock-user) using the GitLab API. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 293decc438e..2deed61388b 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -1,399 +1,11 @@ --- -stage: Create -group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference +redirect_to: '../../../administration/settings/account_and_limit_settings.md' +remove_date: '2023-10-12' --- -# Account and limit settings **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/account_and_limit_settings.md). -## Default projects limit - -You can configure the default maximum number of projects new users can create in their -personal namespace. This limit affects only new user accounts created after you change -the setting. This setting is not retroactive for existing users, but you can separately edit -the [project limits for existing users](#projects-limit-for-a-user). - -To configure the maximum number of projects in personal namespaces for new users: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Increase or decrease that **Default projects limit** value. - -If you set **Default projects limit** to 0, users are not allowed to create projects -in their users personal namespace. However, projects can still be created in a group. - -### Projects limit for a user - -You can edit a specific user, and change the maximum number of projects this user -can create in their personal namespace: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview** > **Users**. -1. From the list of users, select a user. -1. Select **Edit**. -1. Increase or decrease the **Projects limit** value. - -## Max attachment size - -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/20061) from 10 MB to 100 MB in GitLab 15.7. - -The maximum file size for attachments in GitLab comments and replies is 100 MB. -To change the maximum attachment size: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Increase or decrease by changing the value in **Maximum attachment size (MB)**. - -If you choose a size larger than the configured value for the web server, -you may receive errors. Read the [troubleshooting section](#troubleshooting) for more -details. - -For GitLab.com repository size limits, read [accounts and limit settings](../../gitlab_com/index.md#account-and-limit-settings). - -## Max push size - -You can change the maximum push size for your instance: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Increase or decrease by changing the value in **Maximum push size (MB)**. - -For GitLab.com push size limits, read [accounts and limit settings](../../gitlab_com/index.md#account-and-limit-settings). - -NOTE: -When you [add files to a repository](../../project/repository/web_editor.md#create-a-file) -through the web UI, the maximum **attachment** size is the limiting factor, -because the [web server](../../../development/architecture.md#components) -must receive the file before GitLab can generate the commit. -Use [Git LFS](../../../topics/git/lfs/index.md) to add large files to a repository. - -## Max export size - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86124) in GitLab 15.0. - -To modify the maximum file size for exports in GitLab: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**, then expand **Account and limit**. -1. Increase or decrease by changing the value in **Maximum export size (MB)**. - -## Max import size - -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/251106) from 50 MB to unlimited in GitLab 13.8. - -To modify the maximum file size for imports in GitLab: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Increase or decrease by changing the value in **Maximum import size (MB)**. - -This setting applies only to repositories -[imported from a GitLab export file](../../project/settings/import_export.md#import-a-project-and-its-data). - -If you choose a size larger than the configured value for the web server, -you may receive errors. See the [troubleshooting section](#troubleshooting) for more -details. - -For GitLab.com repository size limits, read [accounts and limit settings](../../gitlab_com/index.md#account-and-limit-settings). - -## Personal access token prefix - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20968) in GitLab 13.7. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342327) in GitLab 14.5, a default prefix. - -You can specify a prefix for personal access tokens. You might use a prefix -to find tokens more quickly, or for use with automation tools. - -The default prefix is `glpat-` but administrators can change it. - -[Project access tokens](../../project/settings/project_access_tokens.md) and -[group access tokens](../../group/settings/group_access_tokens.md) also inherit this prefix. - -### Set a prefix - -To change the default global prefix: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. -1. Fill in the **Personal Access Token prefix** field. -1. Select **Save changes**. - -You can also configure the prefix by using the -[settings API](../../../api/settings.md). - -## Repository size limit **(PREMIUM SELF)** - -Repositories in your GitLab instance can grow quickly, especially if you are -using LFS. Their size can grow exponentially, rapidly consuming available storage. -To prevent this from happening, you can set a hard limit for your repositories' size. -This limit can be set globally, per group, or per project, with per project limits -taking the highest priority. - -There are numerous use cases where you might set up a limit for repository size. -For instance, consider the following workflow: - -1. Your team develops apps which require large files to be stored in - the application repository. -1. Although you have enabled [Git LFS](../../../topics/git/lfs/index.md#git-large-file-storage-lfs) - to your project, your storage has grown significantly. -1. Before you exceed available storage, you set up a limit of 10 GB - per repository. - -NOTE: -For GitLab.com repository size limits, read [accounts and limit settings](../../gitlab_com/index.md#account-and-limit-settings). - -### How it works - -Only a GitLab administrator can set those limits. Setting the limit to `0` means -there are no restrictions. - -These settings can be found in: - -- Each project's settings: - 1. From the Project's homepage, navigate to **Settings > General**. - 1. Fill in the **Repository size limit (MB)** field in the **Naming, topics, avatar** section. - 1. Select **Save changes**. -- Each group's settings: - 1. From the Group's homepage, navigate to **Settings > General**. - 1. Fill in the **Repository size limit (MB)** field in the **Naming, visibility** section. - 1. Select **Save changes**. -- GitLab global settings: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. Select **Settings > General**. - 1. Expand the **Account and limit** section. - 1. Fill in the **Size limit per repository (MB)** field. - 1. Select **Save changes**. - -The first push of a new project, including LFS objects, is checked for size. -If the sum of their sizes exceeds the maximum allowed repository size, the push -is rejected. - -NOTE: -The repository size limit includes repository files and LFS, but does not include artifacts, uploads, -wiki, packages, or snippets. The repository size limit applies to both private and public projects. - -For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). - -## Session duration - -### Customize the default session duration - -You can change how long users can remain signed in without activity. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Account and limit**. The set duration is in **Session duration (minutes)**. - -If [Remember me](#turn-remember-me-on-or-off) is enabled, users' sessions can remain active for an indefinite period of time. - -For details, see [cookies used for sign-in](../../profile/index.md#cookies-used-for-sign-in). - -### Turn **Remember me** on or off - -> Ability to turn the **Remember me** setting on and off [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/369133) in GitLab 16.0. - -Users can select the **Remember me** checkbox on sign-in, and their session will remain active for an indefinite period of time when accessed from that specific browser. You can turn off this setting if you need sessions to expire for security or compliance purposes. Turning off this setting will ensure users' sessions expire after the number of minutes of inactivity set when you [customize your session duration](#customize-the-default-session-duration). - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Account and limit**. -1. Select or clear the **Remember me** checkbox to turn this setting on or off. - -### Customize session duration for Git Operations when 2FA is enabled **(PREMIUM SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296669) in GitLab 13.9. -> - It's deployed behind a feature flag, disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. This feature is not ready for production use. This feature flag also affects [2FA for Git over SSH operations](../../../security/two_factor_authentication.md#2fa-for-git-over-ssh-operations). - -GitLab administrators can choose to customize the session duration (in minutes) for Git operations when 2FA is enabled. The default is 15 and this can be set to a value between 1 and 10080. - -To set a limit on how long these sessions are valid: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. -1. Fill in the **Session duration for Git operations when 2FA is enabled (minutes)** field. -1. Select **Save changes**. - -## Limit the lifetime of SSH keys **(ULTIMATE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1007) in GitLab 14.6 [with a flag](../../../administration/feature_flags.md) named `ff_limit_ssh_key_lifetime`. Disabled by default. -> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/346753) in GitLab 14.6. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/1007) in GitLab 14.7. [Feature flag `ff_limit_ssh_key_lifetime`](https://gitlab.com/gitlab-org/gitlab/-/issues/347408) removed. - -Users can optionally specify a lifetime for -[SSH keys](../../ssh.md). -This lifetime is not a requirement, and can be set to any arbitrary number of days. - -SSH keys are user credentials to access GitLab. -However, organizations with security requirements may want to enforce more protection by -requiring the regular rotation of these keys. - -### Set a lifetime - -Only a GitLab administrator can set a lifetime. Leaving it empty means -there are no restrictions. - -To set a lifetime on how long SSH keys are valid: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. -1. Fill in the **Maximum allowable lifetime for SSH keys (days)** field. -1. Select **Save changes**. - -Once a lifetime for SSH keys is set, GitLab: - -- Requires users to set an expiration date that is no later than the allowed lifetime on new - SSH keys. -- Applies the lifetime restriction to existing SSH keys. Keys with no expiry or a lifetime - greater than the maximum immediately become invalid. - -NOTE: -When a user's SSH key becomes invalid they can delete and re-add the same key again. - -## Limit the lifetime of access tokens **(ULTIMATE SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3649) in GitLab 12.6. - -Users can optionally specify a lifetime for -access tokens, this includes [personal](../../profile/personal_access_tokens.md), -[group](../../group/settings/group_access_tokens.md), and [project](../../project/settings/project_access_tokens.md) access tokens. -This lifetime is not a requirement, and can be set to any arbitrary number of days. - -Access tokens are the only tokens needed for programmatic access to GitLab. -However, organizations with security requirements may want to enforce more protection by -requiring the regular rotation of these tokens. - -### Set a lifetime - -Only a GitLab administrator can set a lifetime. Leaving it empty means -there are no restrictions. - -To set a lifetime on how long access tokens are valid: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Account and limit** section. -1. Fill in the **Maximum allowable lifetime for access tokens (days)** field. -1. Select **Save changes**. - -Once a lifetime for access tokens is set, GitLab: - -- Applies the lifetime for new personal access tokens, and require users to set an expiration date - and a date no later than the allowed lifetime. -- After three hours, revoke old tokens with no expiration date or with a lifetime longer than the - allowed lifetime. Three hours is given to allow administrators to change the allowed lifetime, - or remove it, before revocation takes place. - -## Disable user profile name changes **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24605) in GitLab 12.7. - -To maintain integrity of user details in [Audit Events](../../../administration/audit_events.md), GitLab administrators can choose to disable a user's ability to change their profile name. - -To do this: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Select the **Prevent users from changing their profile name** checkbox. - -NOTE: -When this ability is disabled, GitLab administrators can still use the -[Admin Area](../index.md#administering-users) or the -[API](../../../api/users.md#user-modification) to update usernames. - -## Prevent new users from creating top-level groups - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/367754) in GitLab 15.5. - -By default, new users can create top-level groups. GitLab administrators can prevent new users from creating top-level groups: - -- In GitLab 15.5 and later, using either: - - The GitLab UI using the steps in this section. - - The [application setting API](../../../api/settings.md#change-application-settings). -- In GitLab 15.4 and earlier, a [configuration file](../../../administration/user_settings.md#use-configuration-files-to-prevent-new-users-from-creating-top-level-groups). - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Clear the **Allow new users to create top-level groups** checkbox. - -## Set profiles of new users to private by default - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231301) in GitLab 15.8. - -By default, newly created users have a public profile. GitLab administrators can set new users to have a private profile by default: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Select the **Make new users' profiles private by default** checkbox. - -## Prevent users from deleting their accounts **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26053) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `deleting_account_disabled_for_users`. Enabled by default. - -By default, users can delete their own accounts. GitLab administrators can prevent -users from deleting their own accounts: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Account and limit**. -1. Clear the **Allows users to delete their own accounts** checkbox. - -## Troubleshooting - -### 413 Request Entity Too Large - -When attaching a file to a comment or reply in GitLab displays a `413 Request Entity Too Large` -error, the [max attachment size](#max-attachment-size) -is probably larger than the web server's allowed value. - -To increase the max attachment size to 200 MB in a -[Linux package](https://docs.gitlab.com/omnibus/) install, you may need to -add the line below to `/etc/gitlab/gitlab.rb` before increasing the max attachment size: - -```ruby -nginx['client_max_body_size'] = "200m" -``` - -### This repository has exceeded its size limit - -If you receive intermittent push errors in your [Rails exceptions log](../../../administration/logs/index.md#exceptions_jsonlog), like this: - -```plaintext -Your push has been rejected, because this repository has exceeded its size limit. -``` - -[Housekeeping](../../../administration/housekeeping.md) tasks may be causing your repository size to grow. -To resolve this problem, either of these options helps in the short- to middle-term: - -- Increase the [repository size limit](#repository-size-limit). -- [Reduce the repository size](../../project/repository/reducing_the_repo_size_using_git.md). +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 853a299cdec..72cf369146e 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -1,429 +1,11 @@ --- -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/continuous_integration.md' +remove_date: '2023-10-13' --- -# Continuous Integration and Deployment Admin Area settings **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/continuous_integration.md). -The [Admin Area](index.md) has the instance settings for Auto DevOps, runners, and -job artifacts. - -## Auto DevOps - -To enable (or disable) [Auto DevOps](../../../topics/autodevops/index.md) -for all projects: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Check (or uncheck to disable) the box that says **Default to Auto DevOps pipeline for all projects**. -1. Optionally, set up the [Auto DevOps base domain](../../../topics/autodevops/requirements.md#auto-devops-base-domain) - which is used for Auto Deploy and Auto Review Apps. -1. Select **Save changes** for the changes to take effect. - -From now on, every existing project and newly created ones that don't have a -`.gitlab-ci.yml` use the Auto DevOps pipelines. - -If you want to disable it for a specific project, you can do so in -[its settings](../../../topics/autodevops/index.md#enable-or-disable-auto-devops). - -## Enable shared runners for new projects - -You can set all new projects to have the instance's shared runners available by default. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand **Continuous Integration and Deployment**. -1. Select the **Enable shared runners for new projects** checkbox. - -Any time a new project is created, the shared runners are available. - -## Shared runners compute quota - -As an administrator you can set either a global or namespace-specific -limit on the number of [units of compute](../../../ci/pipelines/cicd_minutes.md) you can use. - -## Enable a project runner for multiple projects - -If you have already registered a [project runner](../../../ci/runners/runners_scope.md#project-runners) -you can assign that runner to other projects. - -To enable a project runner for more than one project: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. From the left sidebar, select **CI/CD > Runners**. -1. Select the runner you want to edit. -1. In the upper-right corner, select **Edit** (**{pencil}**). -1. Under **Restrict projects for this runner**, search for a project. -1. To the left of the project, select **Enable**. -1. Repeat this process for each additional project. - -## Add a message for shared runners - -To display details about the instance's shared runners in all projects' -runner settings: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand **Continuous Integration and Deployment**. -1. Enter text, including Markdown if you want, in the **Shared runner details** field. For example: - -  - -To view the rendered details: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. -1. Select **Settings > CI/CD**. -1. Expand **Runners**. - - - -## Maximum artifacts size - -The maximum size of the [job artifacts](../../../administration/job_artifacts.md) -can be set at: - -- The instance level. -- [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/21688), the project and group level. - -For the setting on GitLab.com, see [Artifacts maximum size](../../gitlab_com/index.md#gitlab-cicd). - -The value is in MB and the default is 100 MB per job. To change it at the: - -- Instance level: - - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. On the left sidebar, select **Settings > CI/CD > Continuous Integration and Deployment**. - 1. Change the value of **Maximum artifacts size (MB)**. - 1. Select **Save changes** for the changes to take effect. - -- Group level (this overrides the instance setting): - - 1. Go to the group's **Settings > CI/CD > General Pipelines**. - 1. Change the value of **Maximum artifacts size** (in MB). - 1. Select **Save changes** for the changes to take effect. - -- Project level (this overrides the instance and group settings): - - 1. Go to the project's **Settings > CI/CD > General Pipelines**. - 1. Change the value of **maximum artifacts size** (in MB). - 1. Select **Save changes** for the changes to take effect. - -NOTE: -The setting at all levels is only available to GitLab administrators. - -## Default artifacts expiration - -The default expiration time of the [job artifacts](../../../administration/job_artifacts.md) -can be set in the Admin Area of your GitLab instance. The syntax of duration is -described in [`artifacts:expire_in`](../../../ci/yaml/index.md#artifactsexpire_in) -and the default value is `30 days`. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Change the value of default expiration time. -1. Select **Save changes** for the changes to take effect. - -This setting is set per job and can be overridden in -[`.gitlab-ci.yml`](../../../ci/yaml/index.md#artifactsexpire_in). -To disable the expiration, set it to `0`. The default unit is in seconds. - -NOTE: -Any changes to this setting applies to new artifacts only. The expiration time is not -be updated for artifacts created before this setting was changed. -The administrator may need to manually search for and expire previously-created -artifacts, as described in the [troubleshooting documentation](../../../administration/job_artifacts.md#delete-job-artifacts-from-jobs-completed-before-a-specific-date). - -## Keep the latest artifacts for all jobs in the latest successful pipelines - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50889) in GitLab 13.9. - -When enabled (default), the artifacts of the most recent pipeline for each Git ref -([branches and tags](https://git-scm.com/book/en/v2/Git-Internals-Git-References)) -are locked against deletion and kept regardless of the expiry time. - -When disabled, the latest artifacts for any **new** successful or fixed pipelines -are allowed to expire. - -This setting takes precedence over the [project level setting](../../../ci/jobs/job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs). -If disabled at the instance level, you cannot enable this per-project. - -To disable the setting: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand **Continuous Integration and Deployment**. -1. Clear the **Keep the latest artifacts for all jobs in the latest successful pipelines** checkbox. -1. Select **Save changes** - -When you disable the feature, the latest artifacts do not immediately expire. -A new pipeline must run before the latest artifacts can expire and be deleted. - -NOTE: -All application settings have a [customizable cache expiry interval](../../../administration/application_settings_cache.md) which can delay the settings affect. - -## Archive jobs - -Archiving jobs is useful for reducing the CI/CD footprint on the system by removing some -of the capabilities of the jobs (metadata stored in the database needed to run the job), -but persisting the traces and artifacts for auditing purposes. - -To set the duration for which the jobs are considered as old and expired: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Continuous Integration and Deployment** section. -1. Set the value of **Archive jobs**. -1. Select **Save changes** for the changes to take effect. - -After that time passes, the jobs are archived in the background and no longer able to be -retried. Make it empty to never expire jobs. It has to be no less than 1 day, -for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. - -For the value set for GitLab.com, see [Scheduled job archiving](../../gitlab_com/index.md#gitlab-cicd). - -## Protect CI/CD variables by default - -To set all new [CI/CD variables](../../../ci/variables/index.md) as -[protected](../../../ci/variables/index.md#protect-a-cicd-variable) by default: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Select **Protect CI/CD variables by default**. - -## Maximum includes - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207270) in GitLab 16.0. - -The maximum number of [includes](../../../ci/yaml/includes.md) per pipeline can be set at the instance level. -The default is `150`. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Change the value of **Maximum includes**. -1. Select **Save changes** for the changes to take effect. - -## Default CI/CD configuration file - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18073) in GitLab 12.5. - -The default CI/CD configuration file and path for new projects can be set in the Admin Area -of your GitLab instance (`.gitlab-ci.yml` if not set): - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Input the new file and path in the **Default CI/CD configuration file** field. -1. Select **Save changes** for the changes to take effect. - -It is also possible to specify a [custom CI/CD configuration file for a specific project](../../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file). - -## Set CI/CD limits - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352175) in GitLab 14.10. - -You can configure some [CI/CD limits](../../../administration/instance_limits.md#cicd-limits) -from the Admin Area: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Continuous Integration and Deployment** section. -1. In the **CI/CD limits** section, you can set the following limits: - - **Maximum number of jobs in a single pipeline** - - **Total number of jobs in currently active pipelines** - - **Maximum number of active pipelines per project** - - **Maximum number of pipeline subscriptions to and from a project** - - **Maximum number of pipeline schedules** - - **Maximum number of DAG dependencies that a job can have** - - **Maximum number of runners registered per group** - - **Maximum number of runners registered per project** - - **Maximum number of downstream pipelines in a pipeline's hierarchy tree** - -## Enable or disable the pipeline suggestion banner - -By default, a banner displays in merge requests with no pipeline suggesting a -walkthrough on how to add one. - - - -To enable or disable the banner: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Select or clear the **Enable pipeline suggestion banner** checkbox. -1. Select **Save changes**. - -## Required pipeline configuration **(ULTIMATE SELF)** - -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/352316) from GitLab Premium to GitLab Ultimate in 15.0. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9. - -WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/389467) in GitLab 15.9 -and is planned for removal in 17.0. Use [compliance pipelines](../../group/compliance_frameworks.md#compliance-pipelines) -instead. This change is a breaking change. - -You can set a [CI/CD template](../../../ci/examples/index.md#cicd-templates) -as a required pipeline configuration for all projects on a GitLab instance. You can -use a template from: - -- The default CI/CD templates. -- A custom template stored in an [instance template repository](instance_template_repository.md). - - NOTE: - When you use a configuration defined in an instance template repository, - nested [`include:`](../../../ci/yaml/index.md#include) keywords - (including `include:file`, `include:local`, `include:remote`, and `include:template`) - [do not work](https://gitlab.com/gitlab-org/gitlab/-/issues/35345). - -The project CI/CD configuration merges into the required pipeline configuration when -a pipeline runs. The merged configuration is the same as if the required pipeline configuration -added the project configuration with the [`include` keyword](../../../ci/yaml/index.md#include). -To view a project's full merged configuration, [View full configuration](../../../ci/pipeline_editor/index.md#view-full-configuration) -in the pipeline editor. - -To select a CI/CD template for the required pipeline configuration: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Required pipeline configuration** section. -1. Select a CI/CD template from the dropdown list. -1. Select **Save changes**. - -## Package Registry configuration - -### Maven Forwarding **(PREMIUM SELF)** - -GitLab administrators can disable the forwarding of Maven requests to [Maven Central](https://search.maven.org/). - -To disable forwarding Maven requests: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Package Registry** section. -1. Clear the checkbox **Forward Maven package requests to the Maven Registry if the packages are not found in the GitLab Package Registry**. -1. Select **Save changes**. - -### npm Forwarding **(PREMIUM SELF)** - -GitLab administrators can disable the forwarding of npm requests to [npmjs.com](https://www.npmjs.com/). - -To disable it: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Package Registry** section. -1. Clear the checkbox **Forward npm package requests to the npm Registry if the packages are not found in the GitLab Package Registry**. -1. Select **Save changes**. - -### PyPI Forwarding **(PREMIUM SELF)** - -GitLab administrators can disable the forwarding of PyPI requests to [pypi.org](https://pypi.org/). - -To disable it: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Package Registry** section. -1. Clear the checkbox **Forward PyPI package requests to the PyPI Registry if the packages are not found in the GitLab Package Registry**. -1. Select **Save changes**. - -### Package file size limits - -GitLab administrators can adjust the maximum allowed file size for each package type. - -To set the maximum file size: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand the **Package Registry** section. -1. Find the package type you would like to adjust. -1. Enter the maximum file size, in bytes. -1. Select **Save size limits**. - -## Restrict runner registration by all users in an instance - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22225) in GitLab 14.1. -> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/368008) in GitLab 15.5. - -GitLab administrators can adjust who is allowed to register runners, by showing and hiding areas of the UI. - -When the registration sections are hidden in the UI, members of the project or group must contact administrators to enable runner registration in the group or project. If you plan to prevent registration, ensure users have access to the runners they need to run jobs. - -By default, all members of a project and group are able to register runners. - -To restrict all users in an instance from registering runners: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand **Runners**. -1. In the **Runner registration** section, clear the **Members of the project can register runners** and - **Members of the group can register runners** checkboxes to remove runner registration from the UI. -1. Select **Save changes**. - -NOTE: -After you disable runner registration by members of a project, the registration -token automatically rotates. The token is no longer valid and you must -use the new registration token for the project. - -## Restrict runner registration by all members in a group - -Prerequisites: - -- Runner registration must be enabled for [all users in the instance](#restrict-runner-registration-by-all-users-in-an-instance). - -GitLab administrators can adjust group permissions to restrict runner registration by group members. - -To restrict runner registration by members in a specific group: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Groups** and find your group. -1. Select **Edit**. -1. Clear the **New group runners can be registered** checkbox if you want to disable runner registration by all members in the group. If the setting is read-only, you must enable runner registration for the [instance](#restrict-runner-registration-by-all-users-in-an-instance). -1. Select **Save changes**. - -## Disable runner version management - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114041) in GitLab 15.10. - -By default, GitLab instances periodically fetch official runner version data from GitLab.com to [determine whether the runners need upgrades](../../../ci/runners/configure_runners.md#determine-which-runners-need-to-be-upgraded). - -To disable your instance fetching this data: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > CI/CD**. -1. Expand **Runners**. -1. In the **Runner version management** section, clear the **Fetch GitLab Runner release version data from GitLab.com** checkbox. -1. Select **Save changes**. - -## Troubleshooting - -### 413 Request Entity Too Large - -When build jobs fail with the following error, -increase the [maximum artifacts size](#maximum-artifacts-size). - -```plaintext -Uploading artifacts as "archive" to coordinator... too large archive <job-id> responseStatus=413 Request Entity Too Large status=413" at end of a build job on pipeline when trying to store artifacts to <object-storage>. -``` +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/deprecated_api_rate_limits.md b/doc/user/admin_area/settings/deprecated_api_rate_limits.md index e813d8fe2b1..984eeb24bd1 100644 --- a/doc/user/admin_area/settings/deprecated_api_rate_limits.md +++ b/doc/user/admin_area/settings/deprecated_api_rate_limits.md @@ -1,54 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/deprecated_api_rate_limits.md' +remove_date: '2023-10-13' --- -# Deprecated API rate limits **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/deprecated_api_rate_limits.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68645) in GitLab 14.4. - -Deprecated API endpoints are those which have been replaced with alternative -functionality, but cannot be removed without breaking backward compatibility. -Setting a restrictive rate limit on these endpoints can encourage users to -switch to the alternatives. - -## Deprecated API endpoints - -Not all deprecated API endpoints are included in this rate limit - just those -that might have a performance impact: - -- [`GET /groups/:id`](../../../api/groups.md#details-of-a-group) **without** the `with_projects=0` query parameter. - -## Define Deprecated API rate limits - -Rate limits for deprecated API endpoints are disabled by default. When enabled, they supersede -the general user and IP rate limits for requests to deprecated endpoints. You can keep any general user -and IP rate limits already in place, and increase or decrease the rate limits -for deprecated API endpoints. No other new features are provided by this override. - -Prerequisite: - -- You must have administrator access to the instance. - -To override the general user and IP rate limits for requests to deprecated API endpoints: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Deprecated API Rate Limits**. -1. Select the checkboxes for the types of rate limits you want to enable: - - **Unauthenticated API request rate limit** - - **Authenticated API request rate limit** -1. If you selected **unauthenticated**: - 1. Select the **Maximum unauthenticated API requests per period per IP**. - 1. Select the **Unauthenticated API rate limit period in seconds**. -1. If you selected **authenticated**: - 1. Select the **Maximum authenticated API requests per period per user**. - 1. Select the **Authenticated API rate limit period in seconds**. - -## Related topics - -- [Rate limits](../../../security/rate_limits.md) -- [User and IP rate limits](user_and_ip_rate_limits.md) +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 1caa1728ea4..f90aa7cc865 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -1,125 +1,11 @@ --- -type: reference -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/email.md' +remove_date: '2023-10-14' --- -# Email **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/email.md). -You can customize some of the content in emails sent from your GitLab instance. - -## Custom logo - -The logo in the header of some emails can be customized, see the [logo customization section](../appearance.md#navigation-bar). - -## Include author name in email notification email body **(PREMIUM SELF)** - -By default, GitLab overrides the email address in notification emails with the email address -of the issue, merge request, or comment author. Enable this setting to include the author's email -address in the body of the email instead. - -To include the author's email address in the email body: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Select the **Include author name in email notification email body** checkbox. -1. Select **Save changes**. - -## Enable multipart email **(PREMIUM SELF)** - -GitLab can send email in multipart format (HTML and plain text) or plain text only. - -To enable multipart email: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Select **Enable multipart email**. -1. Select **Save changes**. - -## Custom hostname for private commit emails **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/22560) in GitLab 11.5. - -This configuration option sets the email hostname for [private commit emails](../../profile/index.md#use-an-automatically-generated-private-commit-email). - By default it is set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. - -To change the hostname used in private commit emails: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Enter the desired hostname in the **Custom hostname (for private commit emails)** field. -1. Select **Save changes**. - -NOTE: -After the hostname is configured, every private commit email using the previous hostname is not -recognized by GitLab. This can directly conflict with certain [Push rules](../../project/repository/push_rules.md) such as -`Check whether author is a GitLab user` and `Check whether committer is the current authenticated user`. - -## Custom additional text **(PREMIUM SELF)** - -You can add additional text at the bottom of any email that GitLab sends. This additional text -can be used for legal, auditing, or compliance reasons, for example. - -To add additional text to emails: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Enter your text in the **Additional text** field. -1. Select **Save changes**. - -## User deactivation emails - -GitLab sends email notifications to users when their account has been deactivated. - -To disable these notifications: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Clear the **Enable user deactivation emails** checkbox. -1. Select **Save changes**. - -### Custom additional text in deactivation emails **(FREE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355964) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `deactivation_email_additional_text`. Disabled by default. -> - [Enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/111882) in GitLab 15.9. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an -administrator to [disable the feature flag](../../../administration/feature_flags.md) named -`deactivation_email_additional_text`. - -You can add additional text at the bottom of the email that GitLab sends to users when their account -is deactivated. This email text is separate from the [custom additional text](#custom-additional-text) -setting. - -To add additional text to deactivation emails: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Email**. -1. Enter your text in the **Additional text for deactivation email** field. -1. Select **Save changes**. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-14>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index c15062dd518..02b1ebadc9b 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -1,144 +1,11 @@ --- -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/external_authorization.md' +remove_date: '2023-10-14' --- -# External authorization control **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/external_authorization.md). -> [Moved](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27056) from GitLab Premium to GitLab Free in 11.10. - -In highly controlled environments, it may be necessary for access policy to be -controlled by an external service that permits access based on project -classification and user access. GitLab provides a way to check project -authorization with your own defined service. - -After the external service is configured and enabled, when a project is -accessed, a request is made to the external service with the user information -and project classification label assigned to the project. When the service -replies with a known response, the result is cached for six hours. - -If the external authorization is enabled, GitLab further blocks pages and -functionality that render cross-project data. That includes: - -- Most pages under Dashboard (Activity, Milestones, Snippets, Assigned merge - requests, Assigned issues, To-Do List). -- Under a specific group (Activity, Contribution analytics, Issues, Issue boards, - Labels, Milestones, Merge requests). -- Global and Group search are disabled. - -This is to prevent performing too many requests at once to the external -authorization service. - -Whenever access is granted or denied this is logged in a log file called -`external-policy-access-control.log`. Read more about the logs GitLab keeps in -the [Linux package documentation](https://docs.gitlab.com/omnibus/settings/logs.html). - -When using TLS Authentication with a self signed certificate, the CA certificate -needs to be trusted by the OpenSSL installation. When using GitLab installed -using the Linux package, learn to install a custom CA in the -[Linux package documentation](https://docs.gitlab.com/omnibus/settings/ssl/index.html). -Alternatively, learn where to install custom certificates by using -`openssl version -d`. - -## Configuration - -The external authorization service can be enabled by an administrator: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **External authorization**. -1. Complete the fields. -1. Select **Save changes**. - -### Allow external authorization with deploy tokens and deploy keys - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386656) in GitLab 15.9. -> - Deploy tokens no longer being able to access container or package registries [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/387721) in GitLab 16.0. - -You can set your instance to allow external authorization for Git operations with -[deploy tokens](../../project/deploy_tokens/index.md) or [deploy keys](../../project/deploy_keys/index.md). - -Prerequisites: - -- You must be using classification labels without a service URL for external authorization. - -To allow authorization with deploy tokens and keys: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **External authorization**, and: - - Leave the service URL field empty. - - Select **Allow deploy tokens and deploy keys to be used with external authorization**. -1. Select **Save changes**. - -WARNING: -If you enable external authorization, deploy tokens cannot access container or package registries. If you use deploy tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use tokens with container or package registries. - -## How it works - -When GitLab requests access, it sends a JSON POST request to the external -service with this body: - -```json -{ - "user_identifier": "jane@acme.org", - "project_classification_label": "project-label", - "user_ldap_dn": "CN=Jane Doe,CN=admin,DC=acme", - "identities": [ - { "provider": "ldap", "extern_uid": "CN=Jane Doe,CN=admin,DC=acme" }, - { "provider": "bitbucket", "extern_uid": "2435223452345" } - ] -} -``` - -The `user_ldap_dn` is optional and is only sent when the user is signed in -through LDAP. - -`identities` contains the details of all the identities associated with the -user. This is an empty array if there are no identities associated with the -user. - -When the external authorization service responds with a status code 200, the -user is granted access. When the external service responds with a status code -401 or 403, the user is denied access. In any case, the request is cached for -six hours. - -When denying access, a `reason` can be optionally specified in the JSON body: - -```json -{ - "reason": "You are not allowed access to this project." -} -``` - -Any other status code than 200, 401 or 403 also deny access to the user, but the -response isn't cached. - -If the service times out (after 500 ms), a message "External Policy Server did -not respond" is displayed. - -## Classification labels - -You can use your own classification label in the project's -**Settings > General > General project settings** page in the "Classification -label" box. When no classification label is specified on a project, the default -label defined in the [global settings](#configuration) is used. - -On all project pages, in the upper-right corner, the label appears. - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-14>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/files_api_rate_limits.md b/doc/user/admin_area/settings/files_api_rate_limits.md index 9f7a2d13b8e..c87cac2b8ac 100644 --- a/doc/user/admin_area/settings/files_api_rate_limits.md +++ b/doc/user/admin_area/settings/files_api_rate_limits.md @@ -1,51 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/files_api_rate_limits.md' +remove_date: '2023-10-14' --- -# Rate limits on Repository files API **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/files_api_rate_limits.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68561) in GitLab 14.3. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75918) in GitLab 14.6. [Feature flag `files_api_throttling`](https://gitlab.com/gitlab-org/gitlab/-/issues/338903) removed. - -The [Repository files API](../../../api/repository_files.md) enables you to -fetch, create, update, and delete files in your repository. To improve the security -and durability of your web application, you can enforce -[rate limits](../../../security/rate_limits.md) on this API. Any rate limits you -create for the Files API override the [general user and IP rate limits](user_and_ip_rate_limits.md). - -## Define Files API rate limits - -Rate limits for the Files API are disabled by default. When enabled, they supersede -the general user and IP rate limits for requests to the -[Repository files API](../../../api/repository_files.md). You can keep any general user -and IP rate limits already in place, and increase or decrease the rate limits -for the Files API. No other new features are provided by this override. - -Prerequisite: - -- You must have administrator access to the instance. - -To override the general user and IP rate limits for requests to the Repository files API: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Files API Rate Limits**. -1. Select the checkboxes for the types of rate limits you want to enable: - - **Unauthenticated API request rate limit** - - **Authenticated API request rate limit** -1. If you selected **unauthenticated**: - 1. Select the **Max unauthenticated API requests per period per IP**. - 1. Select the **Unauthenticated API rate limit period in seconds**. -1. If you selected **authenticated**: - 1. Select the **Max authenticated API requests per period per user**. - 1. Select the **Authenticated API rate limit period in seconds**. - -## Related topics - -- [Rate limits](../../../security/rate_limits.md) -- [Repository files API](../../../api/repository_files.md) -- [User and IP rate limits](user_and_ip_rate_limits.md) +<!-- This redirect file can be deleted after <2023-10-14>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/floc.md b/doc/user/admin_area/settings/floc.md index 6bd5a6dfed4..8e99005509a 100644 --- a/doc/user/admin_area/settings/floc.md +++ b/doc/user/admin_area/settings/floc.md @@ -1,42 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/floc.md' +remove_date: '2023-10-06' --- -# Federated Learning of Cohorts (FLoC) **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/floc.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60933) in GitLab 13.12. - -Federated Learning of Cohorts (FLoC) is a new feature of the Chrome browser. -It works by categorizing users into different cohorts, so that -advertisers can use this data to uniquely target and track users. For more -information, see the [FLoC repository](https://github.com/WICG/floc). - -To avoid users being tracked and categorized in any GitLab instance, FLoC is -disabled by default by sending the following header: - -```plaintext -Permissions-Policy: interest-cohort=() -``` - -To enable it: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Federated Learning of Cohorts (FLoC)**. -1. Select the **Participate in FLoC** checkbox. -1. Select **Save changes**. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-06>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/git_lfs_rate_limits.md b/doc/user/admin_area/settings/git_lfs_rate_limits.md index c9b294417ee..8d7840b804c 100644 --- a/doc/user/admin_area/settings/git_lfs_rate_limits.md +++ b/doc/user/admin_area/settings/git_lfs_rate_limits.md @@ -1,36 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/git_lfs_rate_limits.md' +remove_date: '2023-10-06' --- -# Rate limits on Git LFS **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/git_lfs_rate_limits.md). -[Git LFS (Large File Storage)](../../../topics/git/lfs/index.md) is a Git extension -for handling large files. If you use Git LFS in your repository, common Git operations -can generate many Git LFS requests. You can enforce -[general user and IP rate limits](user_and_ip_rate_limits.md), but you can also -override the general setting to enforce additional limits on Git LFS requests. This -override can improve the security and durability of your web application. Aside from -precedence, this configuration provides the same features as the general user and IP -rate limits. - -## Configure Git LFS rate limits - -Git LFS rate limits are disabled by default. If enabled and configured, these limits -supersede the [general user and IP rate limits](user_and_ip_rate_limits.md): - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Git LFS Rate Limits**. -1. Select **Enable authenticated Git LFS request rate limit**. -1. Enter a value for **Max authenticated Git LFS requests per period per user**. -1. Enter a value for **Authenticated Git LFS rate limit period in seconds**. -1. Select **Save changes**. - -## Related topics - -- [Rate limiting](../../../security/rate_limits.md) -- [User and IP rate limits](user_and_ip_rate_limits.md) +<!-- This redirect file can be deleted after <2023-10-06>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/gitaly_timeouts.md b/doc/user/admin_area/settings/gitaly_timeouts.md index d28e9ec0249..572a9c55642 100644 --- a/doc/user/admin_area/settings/gitaly_timeouts.md +++ b/doc/user/admin_area/settings/gitaly_timeouts.md @@ -1,27 +1,11 @@ --- -stage: Systems -group: Gitaly -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/gitaly_timeouts.md' +remove_date: '2023-10-07' --- -# Gitaly timeouts **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/gitaly_timeouts.md). -[Gitaly](../../../administration/gitaly/index.md) timeouts are configurable. The timeouts can be -configured to make sure that long-running Gitaly calls don't needlessly take up resources. - -To access Gitaly timeout settings: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand the **Gitaly timeouts** section. - -## Available timeouts - -The following timeouts are available. - -| Timeout | Default | Description | -|:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../../../administration/operations/puma.md#change-the-worker-timeout) that can be configured for [Puma](../../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | -| Fast | 10 seconds | Timeout for fast Gitaly operations used within requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | -| Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly within requests) but preferably not used multiple times within a request. For example, loading blobs. Timeout that should be set between Default and Fast. | +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/help_page.md b/doc/user/admin_area/settings/help_page.md index febd794b04c..38fe5c3b54c 100644 --- a/doc/user/admin_area/settings/help_page.md +++ b/doc/user/admin_area/settings/help_page.md @@ -1,111 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto +redirect_to: '../../../administration/settings/help_page.md' +remove_date: '2023-10-07' --- -# Customize the Help and sign-in page messages **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/help_page.md). -In large organizations, it is useful to have information about who to contact or where -to go for help. You can customize and display this information on the GitLab `/help` page and on -the GitLab sign-in page. - -## Add a help message to the Help page - -You can add a help message, which is shown at the top of the GitLab `/help` page (for example, -<https://gitlab.com/help>): - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sign-in and Help page**. -1. In **Additional text to show on the Help page**, enter the information you want to display on `/help`. -1. Select **Save changes**. - -You can now see the message on `/help`. - -NOTE: -By default, `/help` is visible to unauthenticated users. However, if the -[**Public** visibility level](visibility_and_access_controls.md#restrict-visibility-levels) -is restricted, `/help` is visible only to authenticated users. - -## Add a help message to the sign-in page - -You can add a help message, which is shown on the GitLab sign-in page. The message appears on the sign-in page: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sign-in and Help page**. -1. In **Additional text to show on the sign-in page**, enter the information you want to - display on the sign-in page. -1. Select **Save changes**. - -You can now see the message on the sign-in page. - -## Hide marketing-related entries from the Help page - -GitLab marketing-related entries are occasionally shown on the Help page. To hide these entries: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sign-in and Help page**. -1. Select the **Hide marketing-related entries from the Help page** checkbox. -1. Select **Save changes**. - -## Set a custom Support page URL - -You can specify a custom URL to which users are directed when they: - -- Select **Support** from the Help dropdown list. -- Select **See our website for help** on the Help page. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sign-in and Help page**. -1. In the **Support page URL** field, enter the URL. -1. Select **Save changes**. - -## Redirect `/help` pages - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/43157) in GitLab 13.5. -> - [Feature flag `help_page_documentation_redirect`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) removed in GitLab 14.4. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71737) in GitLab 14.4. - -You can redirect all `/help` links to a destination that meets the [necessary requirements](#destination-requirements). - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sign-in and Help page**. -1. In the **Documentation pages URL** field, enter the URL. -1. Select **Save changes**. - -If the "Documentation pages URL" field is empty, the GitLab instance displays a basic version of the documentation sourced from the [`doc` directory](https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc) of GitLab. - -### Destination requirements - -When redirecting `/help`, GitLab: - -- Redirects requests to the specified URL. -- Appends `ee` and the documentation path, which includes the version number, to the URL. -- Appends `.html` to the URL, and removes `.md` if necessary. - -For example, if the URL is set to `https://docs.gitlab.com`, requests for -`/help/user/admin_area/settings/help_page.md` redirect to: -`https://docs.gitlab.com/${VERSION}/ee/user/admin_area/settings/help_page.html`. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/import_export_rate_limits.md b/doc/user/admin_area/settings/import_export_rate_limits.md index 99385c77cdf..ad55f424d81 100644 --- a/doc/user/admin_area/settings/import_export_rate_limits.md +++ b/doc/user/admin_area/settings/import_export_rate_limits.md @@ -1,31 +1,11 @@ --- -stage: Manage -group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/import_export_rate_limits.md' +remove_date: '2023-10-07' --- -# Rate limits for imports and exports of project and groups **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/import_export_rate_limits.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35728) in GitLab 13.2. - -You can configure the rate limits for imports and exports of projects and groups: - -To change a rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Import and export rate limits**. -1. Change the value of any rate limit. The rate limits are per minute per user, not per IP address. - Set to `0` to disable a rate limit. - -| Limit | Default | -|-------------------------|---------| -| Project Import | 6 | -| Project Export | 6 | -| Project Export Download | 1 | -| Group Import | 6 | -| Group Export | 6 | -| Group Export Download | 1 | - -When a user exceeds a rate limit, it is logged in `auth.log`. +<!-- This redirect file can be deleted after <2023-10-07>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/incident_management_rate_limits.md b/doc/user/admin_area/settings/incident_management_rate_limits.md index 0b6e572837e..ad11d6f7f36 100644 --- a/doc/user/admin_area/settings/incident_management_rate_limits.md +++ b/doc/user/admin_area/settings/incident_management_rate_limits.md @@ -1,39 +1,11 @@ --- -type: reference -stage: Monitor -group: Respond -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/incident_management_rate_limits.md' +remove_date: '2023-10-10' --- -# Incident management rate limits **(ULTIMATE SELF)** +This document was moved to [another location](../../../administration/settings/incident_management_rate_limits.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/17859) in GitLab 12.5. - -You can limit the number of inbound alerts for [incidents](../../../operations/incident_management/incidents.md) -that can be created in a period of time. The inbound [incident management](../../../operations/incident_management/index.md) -alert limit can help prevent overloading your incident responders by reducing the -number of alerts or duplicate issues. - -As an example, if you set a limit of `10` requests every `60` seconds, -and `11` requests are sent to an [alert integration endpoint](../../../operations/incident_management/integrations.md) within one minute, -the eleventh request is blocked. Access to the endpoint is allowed again after one minute. - -This limit is: - -- Applied independently per project. -- Not applied per IP address. -- Disabled by default. - -Requests that exceed the limit are logged into `auth.log`. - -## Set a limit on inbound alerts - -To set inbound incident management alert limits: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Incident Management Limits**. -1. Select the **Enable Incident Management inbound alert limit** checkbox. -1. Optional. Input a custom value for **Maximum requests per project per rate limit period**. Default is 3600. -1. Optional. Input a custom value for **Rate limit period**. Default is 3600 seconds. +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index 632adf273c4..37112e6897f 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -1,218 +1,11 @@ --- -stage: Create -group: Source Code -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: index +redirect_to: '../../../administration/settings/index.md' +remove_date: '2023-10-10' --- -# Admin Area settings **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/index.md). -As an administrator of a GitLab self-managed instance, you can manage the behavior of your -deployment. - -The **Admin Area** is not accessible on GitLab.com, and settings can only be changed by the -GitLab.com administrators. For the settings and limits on the GitLab.com instance, -read [GitLab.com settings](../../gitlab_com/index.md). - -## Access the Admin Area - -To access the **Admin Area**: - -1. Sign in to your GitLab instance as an administrator. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings**, and the group of settings to view: - - [General](#general) - - [Geo](#geo) - - [CI/CD](#cicd) - - [Integrations](#integrations) - - [Metrics and profiling](#metrics-and-profiling) - - [Network](#network) - - [Preferences](#preferences) - - [Reporting](#reporting) - - [Repository](#repository) - - [Templates](#templates) - -### General - -The **General** settings contain: - -- [Visibility and access controls](visibility_and_access_controls.md) - Set default and - restrict visibility levels. Configure import sources and Git access protocol. -- [Account and limit](account_and_limit_settings.md) - Set projects and maximum size limits, - session duration, user options, and check feature availability for namespace plan. -- [Diff limits](../diff_limits.md) - Diff content limits. -- [Sign-up restrictions](sign_up_restrictions.md) - Configure the way a user creates a new account. -- [Sign in restrictions](sign_in_restrictions.md) - Set requirements for a user to sign in. - Enable mandatory two-factor authentication. -- [Terms of Service and Privacy Policy](terms.md) - Include a Terms of Service agreement - and Privacy Policy that all users must accept. -- [External Authentication](external_authorization.md#configuration) - External Classification Policy Authorization. -- [Web terminal](../../../administration/integration/terminal.md#limiting-websocket-connection-time) - - Set max session time for web terminal. -- [FLoC](floc.md) - Enable or disable - [Federated Learning of Cohorts (FLoC)](https://en.wikipedia.org/wiki/Federated_Learning_of_Cohorts) tracking. - -### CI/CD - -The **CI/CD** settings contain: - -- [Continuous Integration and Deployment](continuous_integration.md) - - Auto DevOps, runners and job artifacts. -- [Required pipeline configuration](continuous_integration.md#required-pipeline-configuration) - - Set an instance-wide auto included [pipeline configuration](../../../ci/yaml/index.md). - This pipeline configuration is run after the project's own configuration. -- [Package Registry](continuous_integration.md#package-registry-configuration) - - Settings related to the use and experience of using the GitLab Package Registry. Some - [risks are involved](../../packages/container_registry/reduce_container_registry_storage.md#use-with-external-container-registries) - in enabling some of these settings. - -## Security and Compliance settings - -- [License compliance settings](security_and_compliance.md#choose-package-registry-metadata-to-sync): Enable or disable synchronization of package metadata by a registry type. - -### Geo **(PREMIUM SELF)** - -The **Geo** setting contains: - -- [Geo](../../../administration/geo/index.md) - Replicate your GitLab instance to other - geographical locations. Redirects to **Admin Area > Geo > Settings** are no - longer available at **Admin Area > Settings > Geo** in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/36896). - -### Integrations - -The **Integrations** settings contain: - -- [Elasticsearch](../../../integration/advanced_search/elasticsearch.md#enable-advanced-search) - - Elasticsearch integration. Elasticsearch AWS IAM. -- [Kroki](../../../administration/integration/kroki.md#enable-kroki-in-gitlab) - - Allow rendering of diagrams in AsciiDoc and Markdown documents using [kroki.io](https://kroki.io). -- [Mailgun](../../../administration/integration/mailgun.md) - Enable your GitLab instance - to receive invite email bounce events from Mailgun, if it is your email provider. -- [PlantUML](../../../administration/integration/plantuml.md) - Allow rendering of PlantUML - diagrams in documents. -- [Slack application](../../../user/project/integrations/gitlab_slack_application.md) - - Slack integration allows you to interact with GitLab via slash commands in a chat window. - This option is only available on GitLab.com, though it may be - [available for self-managed instances in the future](https://gitlab.com/gitlab-org/gitlab/-/issues/28164). -- [Customer experience improvement and third-party offers](third_party_offers.md) - - Control the display of customer experience improvement content and third-party offers. -- [Snowplow](../../../development/internal_analytics/snowplow/index.md) - Configure the Snowplow integration. -- [Google GKE](../../project/clusters/add_gke_clusters.md) - Google GKE integration enables - you to provision GKE clusters from GitLab. -- [Amazon EKS](../../project/clusters/add_eks_clusters.md) - Amazon EKS integration enables - you to provision EKS clusters from GitLab. - -### Metrics and profiling - -The **Metrics and profiling** settings contain: - -- [Metrics - Prometheus](../../../administration/monitoring/prometheus/gitlab_metrics.md) - - Enable and configure Prometheus metrics. -- [Metrics - Grafana](../../../administration/monitoring/performance/grafana_configuration.md#integrate-with-gitlab-ui) - - Enable and configure Grafana. -- [Profiling - Performance bar](../../../administration/monitoring/performance/performance_bar.md#enable-the-performance-bar-for-non-administrators) - - Enable access to the Performance Bar for non-administrator users in a given group. -- [Usage statistics](usage_statistics.md) - Enable or disable version check and Service Ping. - -### Network - -The **Network** settings contain: - -- Performance optimization - Various settings that affect GitLab performance, including: - - [Write to `authorized_keys` file](../../../administration/operations/fast_ssh_key_lookup.md#set-up-fast-lookup). - - [Push event activities limit and bulk push events](push_event_activities_limit.md). -- [User and IP rate limits](user_and_ip_rate_limits.md) - Configure limits for web and API requests. - These rate limits can be overridden: - - [Package Registry Rate Limits](package_registry_rate_limits.md) - Configure specific - limits for Packages API requests that supersede the user and IP rate limits. - - [Git LFS Rate Limits](git_lfs_rate_limits.md) - Configure specific limits for - Git LFS requests that supersede the user and IP rate limits. - - [Files API Rate Limits](files_api_rate_limits.md) - Configure specific limits for - Files API requests that supersede the user and IP rate limits. - - [Search rate limits](../../../administration/instance_limits.md#search-rate-limit) - Configure global search request rate limits for authenticated and unauthenticated users. - - [Deprecated API Rate Limits](deprecated_api_rate_limits.md) - Configure specific limits - for deprecated API requests that supersede the user and IP rate limits. -- [Outbound requests](../../../security/webhooks.md) - Allow requests to the local network from webhooks and integrations, or deny all outbound requests. -- [Protected Paths](protected_paths.md) - Configure paths to be protected by Rack Attack. -- [Incident Management Limits](../../../operations/incident_management/index.md) - Limit the - number of inbound alerts that can be sent to a project. -- [Notes creation limit](rate_limit_on_notes_creation.md) - Set a rate limit on the note creation requests. -- [Get single user limit](rate_limit_on_users_api.md) - Set a rate limit on users API endpoint to get a user by ID. -- [Projects API rate limits for unauthenticated requests](rate_limit_on_projects_api.md) - Set a rate limit on Projects list API endpoint for unauthenticated requests. - -### Preferences - -The **Preferences** settings contain: - -- [Email](email.md) - Various email settings. -- [What's new](../../../administration/whats-new.md) - Configure **What's new** drawer and content. -- [Help page](help_page.md) - Help page text and support page URL. -- [Pages](../../../administration/pages/index.md#custom-domain-verification) - - Size and domain settings for static websites. -- [Polling interval multiplier](../../../administration/polling.md) - - Configure how frequently the GitLab UI polls for updates. -- [Gitaly timeouts](gitaly_timeouts.md) - Configure Gitaly timeouts. -- Localization: - - [Default first day of the week](../../profile/preferences.md). - - [Time tracking](../../project/time_tracking.md#limit-displayed-units-to-hours). -- [Sidekiq Job Limits](sidekiq_job_limits.md) - Limit the size of Sidekiq jobs stored in Redis. - -### Reporting - -The **Reporting** settings contain: - -- Spam and Anti-bot protection: - - Anti-spam services, such as [reCAPTCHA](../../../integration/recaptcha.md), - [Akismet](../../../integration/akismet.md), or [Spamcheck](../reporting/spamcheck.md). - - [IP address restrictions](../reporting/ip_addr_restrictions.md). -- [Abuse reports](../review_abuse_reports.md) - Set notification email for abuse reports. -- [Git abuse rate limit](../reporting/git_abuse_rate_limit.md) - Configure Git abuse rate limit settings. **(ULTIMATE SELF)** - -### Repository - -The **Repository** settings contain: - -- [Repository's custom initial branch name](../../project/repository/branches/default.md#instance-level-custom-initial-branch-name) - - Set a custom branch name for new repositories created in your instance. -- [Repository's initial default branch protection](../../project/repository/branches/default.md#instance-level-default-branch-protection) - - Configure the branch protections to apply to every repository's default branch. -- [Repository mirror](visibility_and_access_controls.md#enable-project-mirroring) - - Configure repository mirroring. -- [Repository storage](../../../administration/repository_storage_types.md) - Configure storage path settings. -- Repository maintenance: - - [Repository checks](../../../administration/repository_checks.md) - Configure - automatic Git checks on repositories. - - [Housekeeping](../../../administration/housekeeping.md). Configure automatic - Git housekeeping on repositories. - - [Inactive project deletion](../../../administration/inactive_project_deletion.md). Configure inactive - project deletion. -- [Repository static objects](../../../administration/static_objects_external_storage.md) - - Serve repository static objects (for example, archives and blobs) from an external storage (for example, a CDN). - -### Templates **(PREMIUM SELF)** - -The **Templates** settings contain: - -- [Templates](instance_template_repository.md#configuration) - Set instance-wide template repository. -- [Custom project templates](../custom_project_templates.md) - Select the custom project template source group. - -## Default first day of the week - -You can change the [Default first day of the week](../../profile/preferences.md) -for the entire GitLab instance: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Scroll to the **Localization** section, and select your desired first day of the week. - -## Default language - -You can change the [Default language](../../profile/preferences.md) -for the entire GitLab instance: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Scroll to the **Localization** section, and select your desired default language. +<!-- This redirect file can be deleted after <2023-10-10>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index a5148c41b74..752630ea922 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -1,84 +1,11 @@ --- -stage: Create -group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference +redirect_to: '../../../administration/settings/instance_template_repository.md' +remove_date: '2023-10-11' --- -# Instance template repository **(PREMIUM SELF)** +This document was moved to [another location](../../../administration/settings/instance_template_repository.md). -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52360) to behave like group-level templates in GitLab 13.9. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/321247) in GitLab 14.0. - -In hosted systems, enterprises often have a need to share their own templates -across teams. This feature allows an administrator to pick a project to be the -instance-wide collection of file templates. These templates are then exposed to -all users through the [Web Editor](../../project/repository/web_editor.md) -while the project remains secure. - -## Configuration - -To select a project to serve as the custom template repository: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Templates**. -1. Expand **Templates** -1. From the dropdown list, select the project to use as the template repository. -1. Select **Save changes**. -1. Add custom templates to the selected repository. - -After you add templates, you can use them for the entire instance. -They are available in the [Web Editor](../../project/repository/web_editor.md) -and through the [API settings](../../../api/settings.md). - -## Supported file types and locations - -Templates must be added to a specific subdirectory in the repository, -corresponding to the kind of template. The following types of custom templates -are supported: - -| Type | Directory | Extension | -| :---------------: | :-----------: | :-----------: | -| `Dockerfile` | `Dockerfile` | `.dockerfile` | -| `.gitignore` | `gitignore` | `.gitignore` | -| `.gitlab-ci.yml` | `gitlab-ci` | `.yml` | -| `LICENSE` | `LICENSE` | `.txt` | - -Each template must go in its respective subdirectory, have the correct -extension and not be empty. So, the hierarchy should look like this: - -```plaintext -|-- README.md -|-- Dockerfile - |-- custom_dockerfile.dockerfile - |-- another_dockerfile.dockerfile -|-- gitignore - |-- custom_gitignore.gitignore - |-- another_gitignore.gitignore -|-- gitlab-ci - |-- custom_gitlab-ci.yml - |-- another_gitlab-ci.yml -|-- LICENSE - |-- custom_license.txt - |-- another_license.txt -``` - -Your custom templates are displayed on the dropdown list when a new file is added through the GitLab UI: - - - -If this feature is disabled or no templates are present, -no **Custom** section displays in the selection dropdown. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/package_registry_rate_limits.md b/doc/user/admin_area/settings/package_registry_rate_limits.md index a1bc339ddd9..269864bdd49 100644 --- a/doc/user/admin_area/settings/package_registry_rate_limits.md +++ b/doc/user/admin_area/settings/package_registry_rate_limits.md @@ -1,57 +1,11 @@ --- -stage: Package -group: Package Registry -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/package_registry_rate_limits.md' +remove_date: '2023-10-11' --- -# Package Registry Rate Limits **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/package_registry_rate_limits.md). -With the [GitLab Package Registry](../../packages/package_registry/index.md), -you can use GitLab as a private or public registry for a variety of common package managers. You can -publish and share packages, which others can consume as a dependency in downstream projects through -the [Packages API](../../../api/packages.md). - -If downstream projects frequently download such dependencies, many requests are made through the -Packages API. You may therefore reach enforced [user and IP rate limits](user_and_ip_rate_limits.md). -To address this issue, you can define specific rate limits for the Packages API: - -- [Unauthenticated requests (per IP)](#enable-unauthenticated-request-rate-limit-for-packages-api). -- [Authenticated API requests (per user)](#enable-authenticated-api-request-rate-limit-for-packages-api). - -These limits are disabled by default. - -When enabled, they supersede the general user and IP rate limits for requests to -the Packages API. You can therefore keep the general user and IP rate limits, and -increase the rate limits for the Packages API. Besides this precedence, there is -no difference in functionality compared to the general user and IP rate limits. - -## Enable unauthenticated request rate limit for packages API - -To enable the unauthenticated request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Package registry rate limits**. -1. Select **Enable unauthenticated request rate limit**. - - - Optional. Update the **Maximum unauthenticated requests per rate limit period per IP** value. - Defaults to `800`. - - Optional. Update the **Unauthenticated rate limit period in seconds** value. - Defaults to `15`. - -## Enable authenticated API request rate limit for packages API - -To enable the authenticated API request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network** -1. Expand **Package registry rate limits**. -1. Select **Enable authenticated API request rate limit**. - - - Optional. Update the **Maximum authenticated API requests per rate limit period per user** value. - Defaults to `1000`. - - Optional. Update the **Authenticated API rate limit period in seconds** value. - Defaults to `15`. +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/project_integration_management.md b/doc/user/admin_area/settings/project_integration_management.md index 1bb4465020c..eff19caabbe 100644 --- a/doc/user/admin_area/settings/project_integration_management.md +++ b/doc/user/admin_area/settings/project_integration_management.md @@ -1,138 +1,11 @@ --- -stage: Manage -group: Import and Integrate -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/project_integration_management.md' +remove_date: '2023-10-11' --- -# Project integration management **(FREE)** +This document was moved to [another location](../../../administration/settings/project_integration_management.md). -Project integrations can be configured and enabled by project administrators. As a GitLab instance -administrator, you can set default configuration parameters for a given integration that all projects -can inherit and use, enabling the integration for all projects that are not already using custom -settings. - -You can update these default settings at any time, changing the settings used for all projects that -are set to use instance-level or group-level defaults. Updating the default settings also enables the integration -for all projects that didn't have it already enabled. - -Only the complete settings for an integration can be inherited. Per-field inheritance is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). - -## Manage instance-level default settings for a project integration **(FREE SELF)** - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2137) in GitLab 13.3 for project-level integrations. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6 for group-level integrations. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Integrations**. -1. Select an integration. -1. Enter configuration details and select **Save changes**. - -WARNING: -This may affect all or most of the groups and projects on your GitLab instance. Review the details -below. - -If this is the first time you are setting up instance-level settings for an integration: - -- The integration is enabled for all groups and projects that don't already have this integration configured, - if you have the **Enable integration** toggle turned on in the instance-level settings. -- Groups and projects that already have the integration configured are not affected, but can choose to use the - inherited settings at any time. - -When you make further changes to the instance defaults: - -- They are immediately applied to all groups and projects that have the integration set to use default settings. -- They are immediately applied to newer groups and projects, created after you last saved defaults for the - integration. If your instance-level default setting has the **Enable integration** toggle turned - on, the integration is automatically enabled for all such groups and projects. -- Groups and projects with custom settings selected for the integration are not immediately affected and may - choose to use the latest defaults at any time. - -Only the complete settings for an integration can be inherited. Per-field inheritance -is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow -administrators to update settings inherited by groups and projects without enabling the -integration on all non-configured groups and projects by default. - -### Remove an instance-level default setting - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Integrations**. -1. Select an integration. -1. Select **Reset** and confirm. - -Resetting an instance-level default setting removes the integration from all projects that have the integration set to use default settings. - -### View projects that override the default settings - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218252) in GitLab 14.2. - -You can view which projects in your instance use custom settings that [override the instance-level default settings](#use-custom-settings-for-a-group-or-project-integration) -for an integration. - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Integrations**. -1. Select an integration. -1. Select the **Projects using custom settings** tab. - -## Manage group-level default settings for a project integration - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6. - -1. Navigate to the group's **Settings > Integrations**. -1. Select an integration. -1. Enter configuration details and select **Save changes**. - -WARNING: -This may affect all or most of the subgroups and projects belonging to the group. Review the details below. - -If this is the first time you are setting up group-level settings for an integration: - -- The integration is enabled for all subgroups and projects belonging to the group that don't already have - this integration configured, if you have the **Enable integration** toggle turned on in the group-level - settings. -- Subgroups and projects that already have the integration configured are not affected, but can choose to use - the inherited settings at any time. - -When you make further changes to the group defaults: - -- They are immediately applied to all subgroups and projects belonging to the group that have the integration - set to use default settings. -- They are immediately applied to newer subgroups and projects, even those created after you last saved defaults for the - integration. If your group-level default setting has the **Enable integration** toggle turned on, - the integration is automatically enabled for all such subgroups and projects. - -- Subgroups and projects with custom settings selected for the integration are not immediately affected and - may choose to use the latest defaults at any time. - -Only the complete settings for an integration can be inherited. Per-field inheritance -is [planned](https://gitlab.com/groups/gitlab-org/-/epics/2137). This would allow -administrators to update settings inherited by subgroups and projects without enabling the -integration on all non-configured groups and projects by default. - -### Remove a group-level default setting - -1. Navigate to the group's **Settings > Integrations**. -1. Select an integration. -1. Select **Reset** and confirm. - -Resetting a group-level default setting removes integrations that use default settings and belong to a project or subgroup of the group. - -## Use instance-level or group-level default settings for a project integration - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2543) in GitLab 13.6 for group-level settings. - -1. Navigate to **Project > Settings > Integrations**. -1. Choose the integration you want to enable or update. -1. From the dropdown list, select **Use default settings**. -1. Ensure the toggle is set to **Enable integration**. -1. Select **Save changes**. - -## Use custom settings for a group or project integration - -1. Navigate to project or group's **Settings > Integrations**. -1. Choose the integration you want to enable or update. -1. From the dropdown list, select **Use custom settings**. -1. Ensure the toggle is set to **Enable integration** and enter all required settings. -1. Select **Save changes**. +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/protected_paths.md b/doc/user/admin_area/settings/protected_paths.md index 4080ee6a540..519d035244a 100644 --- a/doc/user/admin_area/settings/protected_paths.md +++ b/doc/user/admin_area/settings/protected_paths.md @@ -1,43 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/protected_paths.md' +remove_date: '2023-10-12' --- -# Protected paths **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/protected_paths.md). -Rate limiting is a technique that improves the security and durability of a web -application. For more details, see [Rate limits](../../../security/rate_limits.md). - -You can rate limit (protect) specified paths. For these paths, GitLab responds with HTTP status -code `429` to POST requests at protected paths that exceed 10 requests per minute per IP address. - -For example, the following are limited to a maximum 10 requests per minute: - -- User sign-in -- User sign-up (if enabled) -- User password reset - -After 10 requests, the client must wait 60 seconds before it can try again. - -See also: - -- List of paths [protected by default](../../../administration/instance_limits.md#by-protected-path). -- [User and IP rate limits](../../admin_area/settings/user_and_ip_rate_limits.md#response-headers) - for the headers returned to blocked requests. - -## Configure protected paths - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/31246) in GitLab 12.4. - -Throttling of protected paths is enabled by default and can be disabled or -customized on **Admin > Network > Protected Paths**, along with these options: - -- Maximum number of requests per period per user. -- Rate limit period in seconds. -- Paths to be protected. - - - -Requests over the rate limit are logged into `auth.log`. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/push_event_activities_limit.md b/doc/user/admin_area/settings/push_event_activities_limit.md index 56a2902f2d9..b7e059cf820 100644 --- a/doc/user/admin_area/settings/push_event_activities_limit.md +++ b/doc/user/admin_area/settings/push_event_activities_limit.md @@ -1,38 +1,11 @@ --- -stage: Create -group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference +redirect_to: '../../../administration/settings/push_event_activities_limit.md' +remove_date: '2023-10-12' --- -# Push event activities limit and bulk push events **(FREE)** +This document was moved to [another location](../../../administration/settings/push_event_activities_limit.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31007) in GitLab 12.4. - -Set the number of branches or tags to limit the number of single push events -allowed at once. If the number of events is greater than this, GitLab creates -bulk push event instead. - -For example, if 4 branches are pushed and the limit is currently set to 3, -the activity feed displays: - - - -With this feature, when a single push includes a lot of changes (for example, 1,000 -branches), only 1 bulk push event is created instead of 1,000 push -events. This helps in maintaining good system performance and preventing spam on -the activity feed. - -To modify this setting: - -- In the Admin Area: - 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). - 1. Select **Admin Area**. - 1. Select **Settings > Network**. - 1. Expand **Performance optimization**. -- Through the [Application settings API](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls) - as `push_event_activities_limit`. - -The default value is `3`, but the value can be greater than or equal to `0`. Setting this value to `0` does not disable throttling. - - +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md index 09a1e023c2b..aca30177c54 100644 --- a/doc/user/admin_area/settings/rate_limit_on_issues_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_issues_creation.md @@ -1,36 +1,11 @@ --- -type: reference -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/rate_limit_on_issues_creation.md' +remove_date: '2023-10-12' --- -# Rate limits on issue creation **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limit_on_issues_creation.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28129) in GitLab 12.10. - -This setting allows you to rate limit the requests to the issue and epic creation endpoints. -To can change its value: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Issues Rate Limits**. -1. Under **Max requests per minute**, enter the new value. -1. Select **Save changes**. - -For example, if you set a limit of 300, requests using the -[Projects::IssuesController#create](https://gitlab.com/gitlab-org/gitlab/blob/master/app/controllers/projects/issues_controller.rb) -action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. - -When using [epics](../../group/epics/index.md), epic creation shares this rate limit with issues. - - - -This limit is: - -- Applied independently per project and per user. -- Not applied per IP address. -- Disabled by default. To enable it, set the option to any value other than `0`. - -Requests over the rate limit are logged into the `auth.log` file. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limit_on_notes_creation.md b/doc/user/admin_area/settings/rate_limit_on_notes_creation.md index 59548836e78..6d5c93f8554 100644 --- a/doc/user/admin_area/settings/rate_limit_on_notes_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_notes_creation.md @@ -1,35 +1,11 @@ --- -type: reference -stage: Plan -group: Project Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/rate_limit_on_notes_creation.md' +remove_date: '2023-10-12' --- -# Rate limits on note creation **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limit_on_notes_creation.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53637) in GitLab 13.9. - -You can configure the per-user rate limit for requests to the note creation endpoint. - -To change the note creation rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Notes rate limit**. -1. In the **Maximum requests per minute** box, enter the new value. -1. Optional. In the **Users to exclude from the rate limit** box, list users allowed to exceed the limit. -1. Select **Save changes**. - -This limit is: - -- Applied independently per user. -- Not applied per IP address. - -The default value is `300`. - -Requests over the rate limit are logged into the `auth.log` file. - -For example, if you set a limit of 300, requests using the -[Projects::NotesController#create](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/notes_controller.rb) -action exceeding a rate of 300 per minute are blocked. Access to the endpoint is allowed after one minute. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md index 2d0c4405211..c469a77f7d2 100644 --- a/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md +++ b/doc/user/admin_area/settings/rate_limit_on_pipelines_creation.md @@ -1,35 +1,11 @@ --- -type: reference -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/rate_limit_on_pipelines_creation.md' +remove_date: '2023-10-12' --- -# Rate limits on pipeline creation **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limit_on_pipelines_creation.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362475) in GitLab 15.0. - -You can set a limit so that users and processes can't request more than a certain number of pipelines each minute. This limit can help save resources and improve stability. - -For example, if you set a limit of `10`, and `11` requests are sent to the [trigger API](../../../ci/triggers/index.md) within one minute, -the eleventh request is blocked. Access to the endpoint is allowed again after one minute. - -This limit is: - -- Applied to the number of pipelines created for the same combination of project, commit, and user. -- Not applied per IP address. -- Disabled by default. - -Requests that exceed the limit are logged in the `application_json.log` file. - -## Set a pipeline request limit - -To limit the number of pipeline requests: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Pipelines Rate Limits**. -1. Under **Max requests per minute**, enter a value greater than `0`. -1. Select **Save changes**. -1. Enable `ci_enforce_throttle_pipelines_creation` feature flag to enable the rate limit. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limit_on_projects_api.md b/doc/user/admin_area/settings/rate_limit_on_projects_api.md index 326dd3b4706..12577ba44b1 100644 --- a/doc/user/admin_area/settings/rate_limit_on_projects_api.md +++ b/doc/user/admin_area/settings/rate_limit_on_projects_api.md @@ -1,37 +1,11 @@ --- -type: reference -stage: Data Stores -group: Tenant Scale -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/rate_limit_on_projects_api.md' +remove_date: '2023-10-12' --- -# Rate limit on Projects API **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limit_on_projects_api.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/112283) in GitLab 15.10 with a [flag](../../../administration/feature_flags.md) named `rate_limit_for_unauthenticated_projects_api_access`. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/391922) on May 08, 2023. -> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119603) in GitLab 16.0 by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/120445) in GitLab 16.0. Feature flag `rate_limit_for_unauthenticated_projects_api_access` removed. - -You can configure the rate limit per IP address for unauthenticated requests to the [list all projects API](../../../api/projects.md#list-all-projects). - -To change the rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Projects API rate limit**. -1. In the **Maximum requests per 10 minutes per IP address** text box, enter the new value. -1. Select **Save changes**. - -The rate limit: - -- Applies per IP address. -- Doesn't apply to authenticated requests. -- Can be set to 0 to disable rate limiting. - -The default value of the rate limit is `400`. - -Requests over the rate limit are logged into the `auth.log` file. - -For example, if you set a limit of 400, unauthenticated requests to the `GET /projects` API endpoint that -exceed a rate of 400 within 10 minutes are blocked. Access to the endpoint is restored after ten minutes have elapsed. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limit_on_users_api.md b/doc/user/admin_area/settings/rate_limit_on_users_api.md index 112518131bf..80acbf21023 100644 --- a/doc/user/admin_area/settings/rate_limit_on_users_api.md +++ b/doc/user/admin_area/settings/rate_limit_on_users_api.md @@ -1,34 +1,11 @@ --- -type: reference -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/rate_limit_on_users_api.md' +remove_date: '2023-10-12' --- -# Rate limits on Users API **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limit_on_users_api.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78364) in GitLab 14.8. - -You can configure the per user rate limit for requests to [Users API](../../../api/users.md). - -To change the rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Users API rate limit**. -1. In the **Maximum requests per 10 minutes** text box, enter the new value. -1. Optional. In the **Users to exclude from the rate limit** box, list users allowed to exceed the limit. -1. Select **Save changes**. - -This limit is: - -- Applied independently per user. -- Not applied per IP address. - -The default value is `300`. - -Requests over the rate limit are logged into the `auth.log` file. - -For example, if you set a limit of 300, requests to the `GET /users/:id` API endpoint -exceeding a rate of 300 per 10 minutes are blocked. Access to the endpoint is allowed after ten minutes have elapsed. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/profile/saved_replies.md b/doc/user/admin_area/settings/rate_limits_on_git_ssh_operations.md index 1f4e4f5fa51..b60a78d1f49 100644 --- a/doc/user/profile/saved_replies.md +++ b/doc/user/admin_area/settings/rate_limits_on_git_ssh_operations.md @@ -1,11 +1,11 @@ --- -redirect_to: 'comment_templates.md' -remove_date: '2023-06-22' +redirect_to: '../../../administration/settings/rate_limits_on_git_ssh_operations.md' +remove_date: '2023-10-12' --- -This document was moved to [another location](comment_templates.md). +This document was moved to [another location](../../../administration/settings/rate_limits_on_git_ssh_operations.md). -<!-- This redirect file can be deleted after <2023-06-22>. --> +<!-- This redirect file can be deleted after <2023-10-12>. --> <!-- Redirects that point to other docs in the same project expire in three months. --> <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md b/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md index 78e65f7ba7b..5cfee536a58 100644 --- a/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md +++ b/doc/user/admin_area/settings/rate_limits_on_raw_endpoints.md @@ -1,29 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/rate_limits_on_raw_endpoints.md' +remove_date: '2023-10-12' --- -# Rate limits on raw endpoints **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/rate_limits_on_raw_endpoints.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30635) in GitLab 12.2. - -This setting defaults to `300` requests per minute, and allows you to rate limit the requests to raw endpoints: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **Performance optimization**. - -For example, requests over `300` per minute to `https://gitlab.com/gitlab-org/gitlab-foss/raw/master/app/controllers/application_controller.rb` are blocked. Access to the raw file is released after 1 minute. - - - -This limit is: - -- Applied independently per project, per file path. -- Not applied per IP address. -- Active by default. To disable, set the option to `0`. - -Requests over the rate limit are logged into `auth.log`. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/scim_setup.md b/doc/user/admin_area/settings/scim_setup.md index 0471dffe442..4ebd8a84f8a 100644 --- a/doc/user/admin_area/settings/scim_setup.md +++ b/doc/user/admin_area/settings/scim_setup.md @@ -1,43 +1,11 @@ --- -type: reference, howto -stage: Manage -group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/scim_setup.md' +remove_date: '2023-10-12' --- -# Configure SCIM for self-managed GitLab instances **(PREMIUM SELF)** +This document was moved to [another location](../../../administration/settings/scim_setup.md). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8902) in GitLab 15.8. - -You can use the open standard System for Cross-domain Identity Management (SCIM) to automatically: - -- Create users. -- Block users. - -The [internal GitLab SCIM API](../../../development/internal_api/index.md#instance-scim-api) implements part of [the RFC7644 protocol](https://www.rfc-editor.org/rfc/rfc7644). - -If you are a GitLab.com user, see [configuring SCIM for GitLab.com groups](../../../user/group/saml_sso/scim_setup.md). - -## Configure GitLab - -Prerequisites: - -- Configure [SAML single sign-on](../../../integration/saml.md). - -To configure GitLab SCIM: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **SCIM Token** section and select **Generate a SCIM token**. -1. For configuration of your identity provider, save the: - - Token from the **Your SCIM token** field. - - URL from the **SCIM API endpoint URL** field. - -## Remove access - -Removing or deactivating a user on the identity provider blocks the user on -the GitLab instance, while the SCIM identity remains linked to the GitLab user. - -To update the user SCIM identity, use the -[internal GitLab SCIM API](../../../development/internal_api/index.md#update-a-single-scim-provisioned-user-1). +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/security_and_compliance.md b/doc/user/admin_area/settings/security_and_compliance.md index 54fd04f6521..8c1e514c575 100644 --- a/doc/user/admin_area/settings/security_and_compliance.md +++ b/doc/user/admin_area/settings/security_and_compliance.md @@ -1,25 +1,11 @@ --- -stage: Secure -group: Composition Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: howto +redirect_to: '../../../administration/settings/security_and_compliance.md' +remove_date: '2023-10-13' --- -# Security and Compliance Admin Area settings **(ULTIMATE SELF)** +This document was moved to [another location](../../../administration/settings/security_and_compliance.md). -The settings for package metadata synchronization are located in the [Admin Area](index.md). - -## Choose package registry metadata to sync - -WARNING: -The full package metadata sync can add up to 30 GB to the PostgreSQL database. Ensure you have provisioned enough disk space for the database before enabling this feature. -We are actively working on reducing this data size in [epic 10415](https://gitlab.com/groups/gitlab-org/-/epics/10415). - -To choose the packages you want to synchronize with the GitLab License Database for [License Compliance](../../compliance/license_scanning_of_cyclonedx_files/index.md): - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Security and Compliance**. -1. Expand **License Compliance**. -1. Select or clear checkboxes for the package registries that you want to sync. -1. Select **Save changes**. +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/sidekiq_job_limits.md b/doc/user/admin_area/settings/sidekiq_job_limits.md index 08c3ced4c4e..81be26ec8e0 100644 --- a/doc/user/admin_area/settings/sidekiq_job_limits.md +++ b/doc/user/admin_area/settings/sidekiq_job_limits.md @@ -1,35 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/sidekiq_job_limits.md' +remove_date: '2023-10-13' --- -# Sidekiq job size limits **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/sidekiq_job_limits.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68982) in GitLab 14.3. - -[Sidekiq](../../../administration/sidekiq/index.md) jobs get stored in -Redis. To avoid excessive memory for Redis, we: - -- Compress job arguments before storing them in Redis. -- Reject jobs that exceed the specified threshold limit after compression. - -To access Sidekiq job size limits: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Sidekiq job size limits**. -1. Adjust the compression threshold or size limit. The compression can - be disabled by selecting the **Track** mode. - -## Available settings - -| Setting | Default | Description | -|-------------------------------------------|------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Limiting mode | Compress | This mode compresses the jobs at the specified threshold and rejects them if they exceed the specified limit after compression. | -| Sidekiq job compression threshold (bytes) | 100 000 (100 KB) | When the size of arguments exceeds this threshold, they are compressed before being stored in Redis. | -| Sidekiq job size limit (bytes) | 0 | The jobs exceeding this size after compression are rejected. This avoids excessive memory usage in Redis leading to instability. Setting it to 0 prevents rejecting jobs. | - -After changing these values, [restart Sidekiq](../../../administration/restart_gitlab.md). +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/sign_in_restrictions.md b/doc/user/admin_area/settings/sign_in_restrictions.md index 3b79e55f998..c3957ed97eb 100644 --- a/doc/user/admin_area/settings/sign_in_restrictions.md +++ b/doc/user/admin_area/settings/sign_in_restrictions.md @@ -1,201 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/sign_in_restrictions.md' +remove_date: '2023-10-13' --- -# Sign-in restrictions **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/sign_in_restrictions.md). -You can use **Sign-in restrictions** to customize authentication restrictions for web interfaces as well as Git over HTTP(S). - -## Settings - -To access sign-in restriction settings: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Sign-in restrictions** section. - -## Password authentication enabled - -You can restrict the password authentication for web interface and Git over HTTP(S): - -- **Web interface**: When this feature is disabled, the **Standard** sign-in tab - is removed and an [external authentication provider](../../../administration/auth/index.md) - must be used. -- **Git over HTTP(S)**: When this feature is disabled, a [Personal Access Token](../../profile/personal_access_tokens.md) - or LDAP password must be used to authenticate. - -In the event of an external authentication provider outage, use the [GitLab Rails console](../../../administration/operations/rails_console.md) to [re-enable the standard web sign-in form](#re-enable-standard-web-sign-in-form-in-rails-console). This configuration can also be changed over the [Application settings REST API](../../../api/settings.md#change-application-settings) while authenticating with an administrator account's personal access token. - -## Admin Mode - -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2158) in GitLab 13.10. - -If you're an administrator, you might want to work in GitLab without administrator access. -You could either create a separate user account that does not have -administrator access or use Admin Mode. - -With Admin Mode, your account does not have administrator access by default. -You can continue to access groups and projects you're a member of. However, for administrative tasks, -you must authenticate (except for [certain features](#limitations-of-admin-mode)). - -When Admin Mode is enabled, it applies to all administrators on the instance. - -When Admin Mode is enabled for an instance, administrators: - -- Are allowed to access group and projects for which they are members. -- Cannot access the **Admin Area**. - -### Enable Admin Mode for your instance - -Administrators can enable Admin Mode though the API, the Rails console, or the UI. - -#### Use the API to enable Admin Mode - -Make the following request to your instance endpoint: - -```shell -curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab.example.com>/api/v4/application/settings?admin_mode=true" -``` - -Replace `<gitlab.example.com>` with your instance URL. - -For more information, see the [list of settings that can be accessed through API calls](../../../api/settings.md). - -#### Use the Rails console to enable Admin Mode - -Open the [Rails console](../../../administration/operations/rails_console.md) and run the following: - -```ruby -::Gitlab::CurrentSettings.update!(admin_mode: true) -``` - -#### Use the UI to enable Admin Mode - -To enable Admin Mode through the UI: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-in restrictions**. -1. In the **Admin Mode** section, select the **Require additional authentication for administrative tasks** checkbox. - -### Turn on Admin Mode for your session - -To turn on Admin Mode for your current session and access potentially dangerous resources: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Enter Admin Mode**. -1. Try to access any part of the UI with `/admin` in the URL (which requires administrator access). - -When Admin Mode status is disabled or turned off, administrators cannot access resources unless -they've been explicitly granted access. For example, administrators get a `404` error -if they try to open a private group or project, unless they are members of that group or project. - -2FA should be enabled for administrators. 2FA, OmniAuth providers, and LDAP -authentication are supported by Admin Mode. Admin Mode status is stored in the current user session and remains active until either: - -- It is explicitly disabled. -- It is disabled automatically after six hours. - -### Turn off Admin Mode for your session - -To turn off Admin Mode for your current session: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Leave Admin Mode**. - -### Limitations of Admin Mode - -Admin Mode times out after six hours, and you cannot change this timeout limit. - -The following access methods are **not** protected by Admin Mode: - -- Git client access (SSH using public keys or HTTPS using Personal Access Tokens). -- API access using a Personal Access Token. - -In other words, administrators who are otherwise limited by Admin Mode can still use -Git clients, and access RESTful API endpoints as administrators, without additional -authentication steps. - -We may address these limitations in the future. For more information see the following epic: -[Admin Mode for GitLab Administrators](https://gitlab.com/groups/gitlab-org/-/epics/2158). - -Also, when GitLab Geo is enabled, you can't view the replication status of projects and designs while -on a secondary node. A fix is proposed when projects ([issue 367926](https://gitlab.com/gitlab-org/gitlab/-/issues/367926)) and designs ([issue 355660](https://gitlab.com/gitlab-org/gitlab/-/issues/355660)) move to the new Geo framework. - -### Troubleshooting Admin Mode - -If necessary, you can disable **Admin Mode** as an administrator by using one of these two methods: - -- **API**: - - ```shell - curl --request PUT --header "PRIVATE-TOKEN:$ADMIN_TOKEN" "<gitlab-url>/api/v4/application/settings?admin_mode=false" - ``` - -- [**Rails console**](../../../administration/operations/rails_console.md#starting-a-rails-console-session): - - ```ruby - ::Gitlab::CurrentSettings.update!(admin_mode: false) - ``` - -## Two-factor authentication - -When this feature is enabled, all users must use the [two-factor authentication](../../profile/account/two_factor_authentication.md). - -After the two-factor authentication is configured as mandatory, users are allowed -to skip forced configuration of two-factor authentication for the configurable grace -period in hours. - - - -## Email notification for unknown sign-ins - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218457) in GitLab 13.2. - -When enabled, GitLab notifies users of sign-ins from unknown IP addresses or devices. For more information, -see [Email notification for unknown sign-ins](../../profile/notifications.md#notifications-for-unknown-sign-ins). - - - -## Sign-in information - -All users that are not logged in are redirected to the page represented by the configured -**Home page URL** if value is not empty. - -All users are redirected to the page represented by the configured **Sign-out page URL** -after sign out if value is not empty. - -In the **Sign-in restrictions** section, scroll to the **Sign-in text** field. You can add a -custom message for your users in Markdown format. - -For example, if you include the following information in the noted text box: - -```markdown -# Custom sign-in text - -To access this text box: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Sign-in restrictions** section. -``` - -Your users see the **Custom sign-in text** when they navigate to the sign-in screen for your -GitLab instance. - -## Troubleshooting - -### Re-enable standard web sign-in form in rails console - -Re-enable the standard username and password-based sign-in form if it was disabled as a [Sign-in restriction](#password-authentication-enabled). - -You can use this method through the [rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session) when a configured external authentication provider (through SSO or an LDAP configuration) is facing an outage and direct sign-in access to GitLab is required. - -```ruby -Gitlab::CurrentSettings.update!(password_authentication_enabled_for_web: true) -``` +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index 0dcca5e7518..553caa9ff0d 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -1,206 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/sign_up_restrictions.md' +remove_date: '2023-10-13' --- -# Sign-up restrictions **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/sign_up_restrictions.md). -You can enforce the following restrictions on sign ups: - -- Disable new sign ups. -- Require administrator approval for new sign ups. -- Require user email confirmation. -- Allow or deny sign ups using specific email domains. - -## Disable new sign ups - -By default, any user visiting your GitLab domain can sign up for an account. For customers running -public-facing GitLab instances, we **highly** recommend that you consider disabling new sign ups if -you do not expect public users to sign up for an account. - -To disable sign ups: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Clear the **Sign-up enabled** checkbox, then select **Save changes**. - -## Require administrator approval for new sign ups - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4491) in GitLab 13.5. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/267568) in GitLab 13.6. - -When this setting is enabled, any user visiting your GitLab domain and signing up for a new account using the registration form -must be explicitly [approved](../moderate_users.md#approve-or-reject-a-user-sign-up) by an -administrator before they can start using their account. In GitLab 13.6 and later, this setting is -enabled by default for new GitLab instances. It is only applicable if sign ups are enabled. - -To require administrator approval for new sign ups: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Select the **Require admin approval for new sign-ups** checkbox, then select **Save changes**. - -In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/273258), if an administrator disables this setting, the users in pending approval state are -automatically approved in a background job. - -NOTE: -This setting doesn't apply to LDAP or OmniAuth users. To enforce approvals for new users -signing up using OmniAuth or LDAP, set `block_auto_created_users` to `true` in the -[OmniAuth configuration](../../../integration/omniauth.md#configure-common-settings) or -[LDAP configuration](../../../administration/auth/ldap/index.md#basic-configuration-settings). -A [user cap](#user-cap) can also be used to enforce approvals for new users. - -## Confirm user email - -> - Soft email confirmation [introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/47003) in GitLab 12.2 [with a flag](../../../operations/feature_flags.md) named `soft_email_confirmation`. -> - Soft email confirmation [changed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107302/diffs) from a feature flag to an application setting in GitLab 15.9. - -You can send confirmation emails during sign up and require that users confirm -their email address before they are allowed to sign in. - -For example, to enforce confirmation of the email address used for new sign ups: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Under **Email confirmation settings**, select **Hard**. - -The following settings are available: - -- **Hard** - Send a confirmation email during sign up. New users must confirm their email address before they can log in. -- **Soft** - Send a confirmation email during sign up. New users can log in immediately, but must confirm their email in three days. Unconfirmed accounts are deleted. -- **Off** - New users can sign up without confirming their email address. - -## User cap - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4315) in GitLab 13.7. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292600) in GitLab 13.9. - -When the number of billable users reaches the user cap, any user who is added or requests access must be -[approved](../moderate_users.md#approve-or-reject-a-user-sign-up) by an administrator before they can start using -their account. - -If an administrator [increases](#set-the-user-cap-number) or [removes](#remove-the-user-cap) the -user cap, the users in pending approval state are automatically approved in a background job. - -NOTE: -The amount of billable users [is updated once a day](../../../subscriptions/self_managed/index.md#billable-users). -This means the user cap might apply only retrospectively after the cap has already been exceeded. -To ensure the cap is enabled immediately, set it to a low value below the current number of -billable users, for example: `1`. - -On instances that use LDAP or OmniAuth, enabling and disabling -[administrator approval for new sign ups](#require-administrator-approval-for-new-sign-ups) -involves changing the Rails configuration, and may require downtime. -User cap can be used instead. As noted above, set the cap to value that ensures it is enforced immediately. - -### Set the user cap number - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Enter a number in **User cap**. -1. Select **Save changes**. - -New user sign ups are subject to the user cap restriction. - -## Remove the user cap - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Remove the number from **User cap**. -1. Select **Save changes**. - -New users sign ups are not subject to the user cap restriction. Users in pending approval state are -automatically approved in a background job. - -## Minimum password length limit - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20661) in GitLab 12.6 - -You can [change](../../../security/password_length_limits.md#modify-minimum-password-length) -the minimum number of characters a user must have in their password using the GitLab UI. - -### Password complexity requirements **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354965) in GitLab 15.2. - -By default, the only requirement for user passwords is [minimum password length](#minimum-password-length-limit). -You can add additional complexity requirements. Changes to password complexity requirements apply to new passwords: - -- For new users that sign up. -- For existing users that reset their password. - -Existing passwords are unaffected. To change password complexity requirements: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. Under **Minimum password length (number of characters)**, select additional password complexity requirements. You can require numbers, uppercase letters, lowercase letters, - and symbols. -1. Select **Save changes**. - -## Allow or deny sign ups using specific email domains - -You can specify an inclusive or exclusive list of email domains which can be used for user sign up. - -These restrictions are only applied during sign up from an external user. An administrator can add a -user through the administrator panel with a disallowed domain. The users can also change their -email addresses to disallowed domains after sign up. - -### Allowlist email domains - -You can restrict users only to sign up using email addresses matching the given -domains list. - -### Denylist email domains - -You can block users from signing up when using an email addresses of specific domains. This can -reduce the risk of malicious users creating spam accounts with disposable email addresses. - -### Create email domain allowlist or denylist - -To create an email domain allowlist or denylist: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Sign-up restrictions**. -1. For the allowlist, you must enter the list manually. For the denylist, you can enter the list - manually or upload a `.txt` file that contains list entries. - - Both the allowlist and denylist accept wildcards. For example, you can use -`*.company.com` to accept every `company.com` subdomain, or `*.io` to block all -domains ending in `.io`. Domains must be separated by a whitespace, -semicolon, comma, or a new line. - -  - -## Set up LDAP user filter - -You can limit GitLab access to a subset of the LDAP users on your LDAP server. - -See the [documentation on setting up an LDAP user filter](../../../administration/auth/ldap/index.md#set-up-ldap-user-filter) for more information. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/slack_app.md b/doc/user/admin_area/settings/slack_app.md new file mode 100644 index 00000000000..5aae85081ed --- /dev/null +++ b/doc/user/admin_area/settings/slack_app.md @@ -0,0 +1,109 @@ +--- +stage: Manage +group: Import and Integrate +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# GitLab for Slack app administration **(FREE SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358872) for self-managed instances in GitLab 16.2. + +NOTE: +This page contains information about administering the GitLab for Slack app for self-managed instances. For user documentation, see [GitLab for Slack app](../../../user/project/integrations/gitlab_slack_application.md). + +The GitLab for Slack app distributed through the Slack App Directory only works with GitLab.com. +On self-managed GitLab, you can create your own copy of the GitLab for Slack app from a [manifest file](https://api.slack.com/reference/manifests#creating_apps) and configure your instance. + +The app is a private one-time copy installed in your Slack workspace only and not distributed through the Slack App Directory. To have the [GitLab for Slack app](../../../user/project/integrations/gitlab_slack_application.md) on your self-managed instance, you must enable the integration. + +## Create a GitLab for Slack app + +Prerequisite: + +- You must be at least a [Slack workspace administrator](https://slack.com/help/articles/360018112273-Types-of-roles-in-Slack). + +To create a GitLab for Slack app: + +- **In GitLab**: + + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Settings > General**. + 1. Expand **GitLab for Slack app**. + 1. Select **Create Slack app**. + +You're then redirected to Slack for the next steps. + +- **In Slack**: + + 1. Select the Slack workspace to create the app in, then select **Next**. + 1. Slack displays a summary of the app for review. To view the complete manifest, select **Edit Configurations**. To go back to the review summary, select **Next**. + 1. Select **Create**. + 1. Select **Got it** to close the dialog. + 1. Select **Install to Workspace**. + +## Configure the settings + +After you've [created a GitLab for Slack app](#create-a-gitlab-for-slack-app), you can configure the settings in GitLab: + +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **Settings > General**. +1. Expand **GitLab for Slack app**. +1. Select the **Enable GitLab for Slack app** checkbox. +1. Enter the details of your GitLab for Slack app: + 1. Go to [Slack API](https://api.slack.com/apps). + 1. Search for and select **GitLab (\<your host name\>)**. + 1. Scroll to **App Credentials**. +1. Select **Save changes**. + +### Test your configuration + +To test your GitLab for Slack app configuration: + +1. Enter the `/gitlab help` slash command into a channel in your Slack workspace. +1. Press <kbd>Enter</kbd>. + +You should see a list of available Slash commands. + +To use Slash commands for a project, configure the [GitLab for Slack app](../../../user/project/integrations/gitlab_slack_application.md) for the project. + +## Update the GitLab for Slack app + +When GitLab releases new features for the GitLab for Slack app, you might have to manually update your copy to use the new features. + +To update your copy of the GitLab for Slack app: + +- **In GitLab**: + + 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). + 1. Select **Admin Area**. + 1. On the left sidebar, select **Settings > General**. + 1. Expand **GitLab for Slack app**. + 1. Select **Download latest manifest file** to download `slack_manifest.json`. + +- **In Slack**: + + 1. Go to [Slack API](https://api.slack.com/apps). + 1. Search for and select **GitLab (\<your host name\>)**. + 1. On the left sidebar, select **App Manifest**. + 1. Select the **JSON** tab to switch to a JSON view of the manifest. + 1. Copy the contents of the `slack_manifest.json` file you've downloaded from GitLab. + 1. Paste the contents into the JSON viewer to replace any existing contents. + 1. Select **Save Changes**. + +## Connectivity requirements + +To enable the GitLab for Slack app functionality, your network must allow inbound and outbound connections between GitLab and Slack. + +- For [Slack notifications](../../../user/project/integrations/gitlab_slack_application.md#slack-notifications), the GitLab instance must be able to send requests to `https://slack.com`. +- For [Slash commands](../../../user/project/integrations/gitlab_slack_application.md#slash-commands) and other features, the GitLab instance must be able to receive requests from `https://slack.com`. + +## Troubleshooting + +### Slash commands return `/gitlab failed with the error "dispatch_failed"` in Slack + +Slash commands might return `/gitlab failed with the error "dispatch_failed"` in Slack. To resolve this issue, ensure: + +- The GitLab for Slack app is properly [configured](#configure-the-settings), and the **Enable GitLab for Slack app** checkbox is selected. +- Your GitLab instance [allows requests to and from Slack](#connectivity-requirements). diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md index 8563f778292..444eeeb15ea 100644 --- a/doc/user/admin_area/settings/terms.md +++ b/doc/user/admin_area/settings/terms.md @@ -1,49 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/terms.md' +remove_date: '2023-10-13' --- -# Terms of Service and Privacy Policy **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/terms.md). -An administrator can enforce acceptance of a terms of service and privacy policy. -When this option is enabled, new and existing users must accept the terms. - -When enabled, you can view the Terms of Service at the `-/users/terms` page on the instance, -for example `https://gitlab.example.com/-/users/terms`. - -## Enforce a Terms of Service and Privacy Policy - -To enforce acceptance of a Terms of Service and Privacy Policy: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Terms of Service and Privacy Policy** section. -1. Check the **All users must accept the Terms of Service and Privacy Policy to access GitLab** checkbox. -1. Input the text of the **Terms of Service and Privacy Policy**. You can use [Markdown](../../markdown.md) - in this text box. -1. Select **Save changes**. - -For each update to the terms, a new version is stored. When a user accepts or declines the terms, -GitLab records which version they accepted or declined. - -Existing users must accept the terms on their next GitLab interaction. -If an authenticated user declines the terms, they are signed out. - -When enabled, it adds a mandatory checkbox to the sign up page for new users: - - - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/terraform_limits.md b/doc/user/admin_area/settings/terraform_limits.md index 0e620bb84ce..8fed7589bb7 100644 --- a/doc/user/admin_area/settings/terraform_limits.md +++ b/doc/user/admin_area/settings/terraform_limits.md @@ -1,28 +1,11 @@ --- -stage: Deploy -group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/terraform_limits.md' +remove_date: '2023-10-13' --- -# Terraform limits **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/terraform_limits.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352951) in GitLab 15.7. - -You can limit the total storage of [Terraform state files](../../../administration/terraform_state.md). -The limit applies to each individual -state file version, and is checked whenever a new version is created. - -To add a storage limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Preferences**. -1. Expand **Terraform limits**. -1. Adjust the size limit. - -## Available settings - -| Setting | Default | Description | -|------------------------------------|---------|---------------------------------------------------------------------------------------------------------------------------------------------------------| -| Terraform state size limit (bytes) | 0 | Terraform state files that exceed this size are not saved, and associated Terraform operations are rejected. Set to 0 to allow files of unlimited size. | +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index 39e2275f411..54c5b36bbc0 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -1,38 +1,11 @@ --- -stage: Data Stores -group: Tenant Scale -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/third_party_offers.md' +remove_date: '2023-10-13' --- -# Customer experience improvement and third-party offers **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/third_party_offers.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/20379) in GitLab 11.1. - -Within GitLab, we inform users of available third-party offers they might find valuable in order -to enhance the development of their projects. An example is the Google Cloud Platform free credit -for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). - -Furthermore, we use content to improve customer experience. An example are the personalization -questions when creating a group. - -To toggle the display of customer experience improvement content and third-party offers: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand **Customer experience improvement and third-party offers**. -1. Select **Do not display content for customer experience improvement and offers from third parties**. -1. Select **Save changes**. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index ed0c8d21931..5b2afd3ad90 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -1,230 +1,11 @@ --- -stage: Analytics -group: Analytics Instrumentation -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../../administration/settings/usage_statistics.md' +remove_date: '2023-10-13' --- -# Usage statistics **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/usage_statistics.md). -GitLab Inc. periodically collects information about your instance in order -to perform various actions. - -All usage statistics are [opt-out](#enable-or-disable-usage-statistics). - -## Service Ping - -Service Ping is a process that collects and sends a weekly payload to GitLab Inc. -For more information, see the [Service Ping guide](../../../development/internal_analytics/service_ping/index.md). When Service Ping is enabled, GitLab gathers data from other instances and enables certain [instance-level analytics features](../analytics/index.md) -that are dependent on Service Ping. - -### Why enable Service Ping? - -The main purpose of Service Ping is to build a better GitLab. We collect data about how GitLab is used -to understand feature or stage adoption and usage. This data gives an insight into how GitLab adds -value and helps our team understand the reasons why people use GitLab, and with this knowledge we're able to make better product decisions. - -There are several other benefits to enabling Service Ping: - -- Analyze the users' activities over time of your GitLab installation. -- A [DevOps Score](../analytics/dev_ops_reports.md#devops-score) to give you an overview of your entire instance's adoption of concurrent DevOps from planning to monitoring. -- More proactive support (assuming that our [Customer Success Managers (CSMs)](https://handbook.gitlab.com/job-families/sales/customer-success-management/) and support organization used the data to deliver more value). -- Insight and advice into how to get the most value out of your investment in GitLab. -- Reports that show how you compare against other similar organizations (anonymized), with specific advice and recommendations on how to improve your DevOps processes. -- Participation in our [Registration Features Program](#registration-features-program) to receive free paid features. - -## Registration Features Program - -> Introduced in GitLab 14.1. - -In GitLab versions 14.1 and later, GitLab Free customers with a self-managed instance running -GitLab Enterprise Edition can receive paid features by registering with GitLab and sending us -activity data through Service Ping. Features introduced here do not remove the feature from its paid -tier. Users can continue to access the features in a paid tier without sharing usage data. - -### Features available in 14.1 and later - -- [Email from GitLab](../email_from_gitlab.md). - -### Features available in 14.4 and later - -- [Repository size limit](../settings/account_and_limit_settings.md#repository-size-limit). -- [Group access restriction by IP address](../../group/access_and_permissions.md#restrict-group-access-by-ip-address). - -### Features available in 16.0 and later - -- [View description change history](../../../user/discussions/index.md#view-description-change-history). -- [Maintenance mode](../../../administration/maintenance_mode/index.md). -- [Configurable issue boards](../../project/issue_board.md#configurable-issue-boards). -- [Coverage-guided fuzz testing](../../application_security/coverage_fuzzing/index.md). -- [Password complexity requirements](../../../user/admin_area/settings/sign_up_restrictions.md#password-complexity-requirements). - -NOTE: -Registration is not yet required for participation, but may be added in a future milestone. - -### Enable registration features - -1. Sign in as a user with administrator access. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Metrics and profiling**. -1. Expand the **Usage statistics** section. -1. If not enabled, select the **Enable Service Ping** checkbox. -1. Select the **Enable Registration Features** checkbox. -1. Select **Save changes**. - -## Version check - -If enabled, version check informs you if a new version is available and the -importance of it through a status. The status displays on the help pages (`/help`) -for all authenticated users, and on the Admin Area pages. The statuses are: - -- Green: You are running the latest version of GitLab. -- Orange: An updated version of GitLab is available. -- Red: The version of GitLab you are running is vulnerable. You should install - the latest version with security fixes as soon as possible. - - - -GitLab Inc. collects your instance's version and hostname (through the HTTP -referer) as part of the version check. No other information is collected. - -This information is used, among other things, to identify to which versions -patches must be backported, making sure active GitLab instances remain -secure. - -If you [disable version check](#enable-or-disable-usage-statistics), this information isn't collected. - -### Request flow example - -The following example shows a basic request/response flow between a -self-managed GitLab instance and the GitLab Version Application: - -```mermaid -sequenceDiagram - participant GitLab instance - participant Version Application - GitLab instance->>Version Application: Is there a version update? - loop Version Check - Version Application->>Version Application: Record version info - end - Version Application->>GitLab instance: Response (PNG/SVG) -``` - -## Configure your network - -To send usage statistics to GitLab Inc., you must allow network traffic from your -GitLab instance to the host `version.gitlab.com` on port `443`. - -If your GitLab instance is behind a proxy, set the appropriate -[proxy configuration variables](https://docs.gitlab.com/omnibus/settings/environment-variables.html). - -## Enable or disable usage statistics - -To enable or disable Service Ping and version check: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Metrics and profiling**. -1. Expand **Usage statistics**. -1. Select or clear the **Enable version check** and **Enable Service Ping** checkboxes. -1. Select **Save changes**. - -NOTE: -Service Ping settings only control whether the data is being shared with GitLab, or used only internally. -Even if you disable Service Ping, the `gitlab_service_ping_worker` background job still periodically generates a Service Ping payload for your instance. -The payload is available in the [Service Usage data](#manually-upload-service-ping-payload) admin section. - -## Disable usage statistics with the configuration file - -NOTE: -The method to disable Service Ping in the GitLab configuration file does not work in -GitLab versions 9.3 to 13.12.3. For more information about how to disable it, see [troubleshooting](../../../development/internal_analytics/service_ping/troubleshooting.md#cannot-disable-service-ping-with-the-configuration-file). - -To disable Service Ping and prevent it from being configured in the future through -the Admin Area: - -**For installations using the Linux package:** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['usage_ping_enabled'] = false - ``` - -1. Reconfigure GitLab: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -**For installations from source:** - -1. Edit `/home/git/gitlab/config/gitlab.yml`: - - ```yaml - production: &base - # ... - gitlab: - # ... - usage_ping_enabled: false - ``` - -1. Restart GitLab: - - ```shell - sudo service gitlab restart - ``` - -## View the Service Ping payload - -You can view the exact JSON payload sent to GitLab Inc. in the Admin Area. To view the payload: - -1. Sign in as a user with administrator access. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Metrics and profiling**. -1. Expand the **Usage statistics** section. -1. Select **Preview payload**. - -For an example payload, see [Example Service Ping payload](../../../development/internal_analytics/service_ping/index.md#example-service-ping-payload). - -## Manually upload Service Ping payload - -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7388) in GitLab 14.8 with a flag named `admin_application_settings_service_usage_data_center`. Disabled by default. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83265) in GitLab 14.10. - -You can upload the Service Ping payload to GitLab even if your instance doesn't have internet access, -or if the Service Ping [cron job](../../../development/internal_analytics/service_ping/index.md#how-service-ping-works) is not enabled. - -To upload the payload manually: - -1. Sign in as a user with administrator access. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Service** usage data. -1. Select **Download payload**. -1. Save the JSON file. -1. Visit [Service usage data center](https://version.gitlab.com/usage_data/new). -1. Select **Choose file** and choose the file from p5. -1. Select **Upload**. - -The uploaded file is encrypted and sent using secure HTTPS protocol. HTTPS creates a secure -communication channel between web browser and the server, and protects transmitted data against man-in-the-middle attacks. - -If there are problems with the manual upload: - -1. Open a confidential issue in the [security fork of version app project](https://gitlab.com/gitlab-org/security/version.gitlab.com). -1. Attach the JSON payload if possible. -1. Tag `@gitlab-org/analytics-section/analytics-instrumentation` who will triage the issue. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-13>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md index d145c351f3e..fae47358879 100644 --- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -1,240 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments -type: reference +redirect_to: '../../../administration/settings/user_and_ip_rate_limits.md' +remove_date: '2023-10-14' --- -# User and IP rate limits **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/user_and_ip_rate_limits.md). -Rate limiting is a common technique used to improve the security and durability -of a web application. For more details, see -[Rate limits](../../../security/rate_limits.md). - -The following limits are disabled by default: - -- [Unauthenticated API requests (per IP)](#enable-unauthenticated-api-request-rate-limit). -- [Unauthenticated web requests (per IP)](#enable-unauthenticated-web-request-rate-limit). -- [Authenticated API requests (per user)](#enable-authenticated-api-request-rate-limit). -- [Authenticated web requests (per user)](#enable-authenticated-web-request-rate-limit). - -NOTE: -By default, all Git operations are first tried unauthenticated. Because of this, HTTP Git operations -may trigger the rate limits configured for unauthenticated requests. - -NOTE: -[In GitLab 14.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/344807), -the rate limits for API requests don't affect requests made by the frontend, as these are always -counted as web traffic. - -## Enable unauthenticated API request rate limit - -To enable the unauthenticated request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **User and IP rate limits**. -1. Select **Enable unauthenticated API request rate limit**. - - - Optional. Update the **Maximum unauthenticated API requests per rate limit period per IP** value. - Defaults to `3600`. - - Optional. Update the **Unauthenticated rate limit period in seconds** value. - Defaults to `3600`. - -## Enable unauthenticated web request rate limit - -To enable the unauthenticated request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **User and IP rate limits**. -1. Select **Enable unauthenticated web request rate limit**. - - - Optional. Update the **Maximum unauthenticated web requests per rate limit period per IP** value. - Defaults to `3600`. - - Optional. Update the **Unauthenticated rate limit period in seconds** value. - Defaults to `3600`. - -## Enable authenticated API request rate limit - -To enable the authenticated API request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **User and IP rate limits**. -1. Select **Enable authenticated API request rate limit**. - - - Optional. Update the **Maximum authenticated API requests per rate limit period per user** value. - Defaults to `7200`. - - Optional. Update the **Authenticated API rate limit period in seconds** value. - Defaults to `3600`. - -## Enable authenticated web request rate limit - -To enable the unauthenticated request rate limit: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **User and IP rate limits**. -1. Select **Enable authenticated web request rate limit**. - - - Optional. Update the **Maximum authenticated web requests per rate limit period per user** value. - Defaults to `7200`. - - Optional. Update the **Authenticated web rate limit period in seconds** value. - Defaults to `3600`. - -## Use a custom rate limit response - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50693) in GitLab 13.8. - -A request that exceeds a rate limit returns a `429` response code and a -plain-text body, which by default is `Retry later`. - -To use a custom response: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > Network**. -1. Expand **User and IP rate limits**. -1. In the **Plain-text response to send to clients that hit a rate limit** text box, - add the plain-text response message. - -## Response headers - -> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/731) in GitLab 13.8, the `RateLimit` headers. `Retry-After` was introduced in an earlier version. - -When a client exceeds the associated rate limit, the following requests are -blocked. The server may respond with rate-limiting information allowing the -requester to retry after a specific period of time. These information are -attached into the response headers. - -| Header | Example | Description | -|:----------------------|:--------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the Admin Area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | -| `RateLimit-Name` | `throttle_authenticated_web` | Name of the throttle blocking the requests. | -| `RateLimit-Observed` | `67` | Number of requests associated to the client in the time window. | -| `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Observed`. | -| `RateLimit-Reset` | `1609844400` | [Unix time](https://en.wikipedia.org/wiki/Unix_time)-formatted time when the request quota is reset. | -| `RateLimit-ResetTime` | `Tue, 05 Jan 2021 11:00:00 GMT` | [RFC2616](https://www.rfc-editor.org/rfc/rfc2616#section-3.3.1)-formatted date and time when the request quota is reset. | -| `Retry-After` | `30` | Remaining duration **in seconds** until the quota is reset. This is a [standard HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After). | - -## Use an HTTP header to bypass rate limiting - -> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/622) in GitLab 13.6. - -Depending on the needs of your organization, you may want to enable rate limiting -but have some requests bypass the rate limiter. - -You can do this by marking requests that should bypass the rate limiter with a custom -header. You must do this somewhere in a load balancer or reverse proxy in front of -GitLab. For example: - -1. Pick a name for your bypass header. For example, `Gitlab-Bypass-Rate-Limiting`. -1. Configure your load balancer to set `Gitlab-Bypass-Rate-Limiting: 1` on requests - that should bypass GitLab rate limiting. -1. Configure your load balancer to either: - - Erase `Gitlab-Bypass-Rate-Limiting`. - - Set `Gitlab-Bypass-Rate-Limiting` to a value other than `1` on all requests that - should be affected by rate limiting. -1. Set the environment variable `GITLAB_THROTTLE_BYPASS_HEADER`. - - For [Linux package installations](https://docs.gitlab.com/omnibus/settings/environment-variables.html), - set `'GITLAB_THROTTLE_BYPASS_HEADER' => 'Gitlab-Bypass-Rate-Limiting'` in `gitlab_rails['env']`. - - For source installations, set `export GITLAB_THROTTLE_BYPASS_HEADER=Gitlab-Bypass-Rate-Limiting` - in `/etc/default/gitlab`. - -It is important that your load balancer erases or overwrites the bypass -header on all incoming traffic. Otherwise, you must trust your -users to not set that header and bypass the GitLab rate limiter. - -The bypass works only if the header is set to `1`. - -Requests that bypassed the rate limiter because of the bypass header -are marked with `"throttle_safelist":"throttle_bypass_header"` in -[`production_json.log`](../../../administration/logs/index.md#production_jsonlog). - -To disable the bypass mechanism, make sure the environment variable -`GITLAB_THROTTLE_BYPASS_HEADER` is unset or empty. - -## Allow specific users to bypass authenticated request rate limiting - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49127) in GitLab 13.7. - -Similarly to the bypass header described above, it is possible to allow -a certain set of users to bypass the rate limiter. This only applies -to authenticated requests: with unauthenticated requests, by definition -GitLab does not know who the user is. - -The allowlist is configured as a comma-separated list of user IDs in -the `GITLAB_THROTTLE_USER_ALLOWLIST` environment variable. If you want -users 1, 53 and 217 to bypass the authenticated request rate limiter, -the allowlist configuration would be `1,53,217`. - -- For [Linux package installations](https://docs.gitlab.com/omnibus/settings/environment-variables.html), - set `'GITLAB_THROTTLE_USER_ALLOWLIST' => '1,53,217'` in `gitlab_rails['env']`. -- For source installations, set `export GITLAB_THROTTLE_USER_ALLOWLIST=1,53,217` - in `/etc/default/gitlab`. - -Requests that bypassed the rate limiter because of the user allowlist -are marked with `"throttle_safelist":"throttle_user_allowlist"` in -[`production_json.log`](../../../administration/logs/index.md#production_jsonlog). - -At application startup, the allowlist is logged in [`auth.log`](../../../administration/logs/index.md#authlog). - -## Try out throttling settings before enforcing them - -> [Introduced](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/629) in GitLab 13.6. - -You can try out throttling settings by setting the `GITLAB_THROTTLE_DRY_RUN` environment variable to -a comma-separated list of throttle names. - -The possible names are: - -- `throttle_unauthenticated` - - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/335300) in GitLab 14.3. Use `throttle_unauthenticated_api` or `throttle_unauthenticated_web` instead. - `throttle_unauthenticated` is still supported and selects both of them. -- `throttle_unauthenticated_api` -- `throttle_unauthenticated_web` -- `throttle_authenticated_api` -- `throttle_authenticated_web` -- `throttle_unauthenticated_protected_paths` -- `throttle_authenticated_protected_paths_api` -- `throttle_authenticated_protected_paths_web` -- `throttle_unauthenticated_packages_api` -- `throttle_authenticated_packages_api` -- `throttle_authenticated_git_lfs` -- `throttle_unauthenticated_files_api` -- `throttle_authenticated_files_api` -- `throttle_unauthenticated_deprecated_api` -- `throttle_authenticated_deprecated_api` - -For example, to try out throttles for all authenticated requests to -non-protected paths can be done by setting -`GITLAB_THROTTLE_DRY_RUN='throttle_authenticated_web,throttle_authenticated_api'`. - -To enable dry run mode for all throttles, the variable can be set to `*`. - -Setting a throttle to dry run mode logs a message to the -[`auth.log`](../../../administration/logs/index.md#authlog) when it would hit the limit, while letting the -request continue. The log message contains an `env` field set to `track`. The `matched` -field contains the name of throttle that was hit. - -It is important to set the environment variable **before** enabling -the rate limiting in the settings. The settings in the Admin Area -take effect immediately, while setting the environment variable -requires a restart of all the Puma processes. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-14>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index dd53efaf518..c9ff105f8c9 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -1,363 +1,11 @@ --- -stage: Create -group: Source Code -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" -type: reference +redirect_to: '../../../administration/settings/visibility_and_access_controls.md' +remove_date: '2023-10-14' --- -# Control access and visibility **(FREE SELF)** +This document was moved to [another location](../../../administration/settings/visibility_and_access_controls.md). -GitLab enables users with administrator access to enforce -specific controls on branches, projects, snippets, groups, and more. - -To access the visibility and access control options: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. - -## Define which roles can create projects - -Instance-level protections for project creation define which roles can -[add projects to a group](../../group/index.md#specify-who-can-add-projects-to-a-group) -on the instance. To alter which roles have permission to create projects: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. For **Default project creation protection**, select the desired roles: - - No one. - - Maintainers. - - Developers and Maintainers. -1. Select **Save changes**. - -## Restrict project deletion to administrators **(PREMIUM SELF)** - -> User interface [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. - -By default both administrators and anyone with the **Owner** role can delete a project. To restrict project deletion to only administrators: - -1. Sign in to GitLab as a user with administrator access. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Scroll to: - - (GitLab 15.1 and later) **Allowed to delete projects**, and select **Administrators**. - - (GitLab 15.0 and earlier) **Default project deletion protection** and select **Only admins can delete project**. -1. Select **Save changes**. - -## Deletion protection **(PREMIUM SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255449) in GitLab 14.2 for groups created after August 12, 2021. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) from default delayed project deletion in GitLab 15.1. -> - [Enabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/89466) in GitLab 15.1. -> - [Disabled for projects in personal namespaces](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95495) in GitLab 15.3. -> - [Removed option to delete immediately](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) in GitLab 15.11 [with a flag](../../../administration/feature_flags.md) named `always_perform_delayed_deletion`. Disabled by default. -> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. - -Instance-level protection against accidental deletion of groups and projects. - -### Retention period - -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. - -Groups and projects remain restorable within a defined retention period. By default this is 7 days but it can be changed. -Setting the retention period to `0` means that groups and project are removed immediately and cannot be restored. - -In GitLab 15.1 and later, the retention period must be between `1` and `90`. If the retention period was `0` before the 15.1 update, -then it gets automatically changed to `1` while also disabling deletion protection the next time any application setting is changed. - -### Delayed project deletion - -> - User interface [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. -> - Enabled delayed deletion by default and removed the option to delete immediately [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/393622) and [on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119606) in GitLab 16.0. - -To configure delayed project deletion: - -1. Sign in to GitLab as a user with administrator access. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Scroll to: - - (In GitLab 15.11 and later with `always_perform_delayed_deletion` feature flag enabled, or GitLab 16.0 and later) **Deletion protection** and set the retention period to a value between `1` and `90`. - - (GitLab 15.1 and later) **Deletion protection** and select keep deleted groups and projects, and select a retention period. - - (GitLab 15.0 and earlier) **Default delayed project protection** and select **Enable delayed project deletion by - default for newly-created groups.** Then set a retention period in **Default deletion delay**. -1. Select **Save changes**. - -Deletion protection is not available for projects only (without being also being enabled for groups). - -In GitLab 15.1, and later this setting is enforced on groups when disabled and it cannot be overridden. - -### Delayed group deletion - -> - User interface [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352960) in GitLab 15.1. -> - [Changed to default behavior](https://gitlab.com/gitlab-org/gitlab/-/issues/389557) on the Premium and Ultimate tier in GitLab 16.0. - -Groups remain restorable if the retention period is `1` or more days. - -In GitLab 15.1 and later, delayed group deletion can be enabled by setting **Deletion projection** to **Keep deleted**. -In GitLab 15.11 and later with the `always_perform_delayed_deletion` feature flag enabled, or in GitLab 16.0 and later: - -- The **Keep deleted** option is removed. -- Delayed group deletion is the default. - -### Override defaults and delete immediately - -Alternatively, projects that are marked for removal can be deleted immediately. To do so: - -1. [Restore the project](../../project/settings/index.md#restore-a-project). -1. Delete the project as described in the - [Administering Projects page](../../admin_area/index.md#administering-projects). - -## Configure project visibility defaults - -To set the default [visibility levels for new projects](../../public_access.md): - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select the desired default project visibility: - - **Private** - Project access must be granted explicitly to each user. If this - project is part of a group, access is granted to members of the group. - - **Internal** - The project can be accessed by any authenticated user except external users. - - **Public** - The project can be accessed without any authentication. -1. Select **Save changes**. - -## Configure snippet visibility defaults - -To set the default visibility levels for new [snippets](../../snippets.md): - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select the desired default snippet visibility. -1. Select **Save changes**. - -For more details on snippet visibility, read -[Project visibility](../../public_access.md). - -## Configure group visibility defaults - -To set the default visibility levels for new groups: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select the desired default group visibility: - - **Private** - The group and its projects can only be viewed by members. - - **Internal** - The group and any internal projects can be viewed by any authenticated user except external users. - - **Public** - The group and any public projects can be viewed without any authentication. -1. Select **Save changes**. - -For more details on group visibility, see -[Group visibility](../../group/index.md#group-visibility). - -## Restrict visibility levels - -When restricting visibility levels, consider how these restrictions interact -with permissions for subgroups and projects that inherit their visibility from -the item you're changing. - -To restrict visibility levels for groups, projects, snippets, and selected pages: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. In the **Restricted visibility levels** section, select the desired visibility levels to restrict. - - If you restrict the **Public** level: - - Only administrators are able to create public groups, projects, and snippets. - - User profiles are only visible to authenticated users through the Web interface. - - User attributes through the GraphQL API are: - - Not visible in [GitLab 15.1 and later](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/88020). - - Only visible to authenticated users between [GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/33195) and GitLab 15.0. - - If you restrict the **Internal** level: - - Only administrators are able to create internal groups, projects, and snippets. - - If you restrict the **Private** level: - - Only administrators are able to create private groups, projects, and snippets. -1. Select **Save changes**. - -For more details on project visibility, see -[Project visibility](../../public_access.md). - -## Configure allowed import sources - -Before you can import projects from other systems, you must enable the -[import source](../../gitlab_com/index.md#default-import-sources) for that system. - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select each of **Import sources** to allow. -1. Select **Save changes**. - -## Enable project export - -To enable the export of -[projects and their data](../../project/settings/import_export.md#export-a-project-and-its-data): - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Scroll to **Project export**. -1. Select the **Enabled** checkbox. -1. Select **Save changes**. - -## Enable migration of groups and projects by direct transfer - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/383268) in GitLab 15.8. - -You can enable migration of groups by direct transfer using the UI. - -To enable migration of groups by direct transfer: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Scroll to **Allow migrating GitLab groups and projects by direct transfer**. -1. Select the **Enabled** checkbox. -1. Select **Save changes**. - -The same setting -[is available](../../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls) in the API as the -`bulk_import_enabled` attribute. - -## Configure enabled Git access protocols - -With GitLab access restrictions, you can select the protocols users can use to -communicate with GitLab. Disabling an access protocol does not block port access to the -server itself. The ports used for the protocol, SSH or HTTP(S), are still accessible. -The GitLab restrictions apply at the application level. - -To specify the enabled Git access protocols: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select the desired Git access protocols: - - Both SSH and HTTP(S) - - Only SSH - - Only HTTP(S) -1. Select **Save changes**. - -When both SSH and HTTP(S) are enabled, users can choose either protocol. -If only one protocol is enabled: - -- The project page shows only the allowed protocol's URL, with no option to - change it. -- GitLab shows a tooltip when you hover over the protocol for the URL, if user action - (such as adding a SSH key or setting a password) is required: - -  - -GitLab only allows Git actions for the protocols you select. - -WARNING: -GitLab versions [10.7 and later](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/18021), -allow the HTTP(S) protocol for Git clone or fetch requests done by GitLab Runner -from CI/CD jobs, even if you select **Only SSH**. - -## Customize Git clone URL for HTTP(S) - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18422) in GitLab 12.4. - -You can customize project Git clone URLs for HTTP(S), which affects the clone -panel: - -For example, if: - -- Your GitLab instance is at `https://example.com`, then project clone URLs are like - `https://example.com/foo/bar.git`. -- You want clone URLs that look like `https://git.example.com/gitlab/foo/bar.git` instead, - you can set this setting to `https://git.example.com/gitlab/`. - - - -To specify a custom Git clone URL for HTTP(S): - -1. Enter a root URL for **Custom Git clone URL for HTTP(S)**. -1. Select **Save changes**. - -NOTE: -SSH clone URLs can be customized in `gitlab.rb` by setting `gitlab_rails['gitlab_ssh_host']` and -other related settings. - -## Configure defaults for RSA, DSA, ECDSA, ED25519, ECDSA_SK, ED25519_SK SSH keys - -These options specify the permitted types and lengths for SSH keys. - -To specify a restriction for each key type: - -1. Select the desired option from the dropdown list. -1. Select **Save changes**. - -For more details, see [SSH key restrictions](../../../security/ssh_keys_restrictions.md). - -## Enable project mirroring - -This option is enabled by default. By disabling it, both -[pull mirroring](../../project/repository/mirror/pull.md) and [push mirroring](../../project/repository/mirror/push.md) no longer -work in every repository. They can only be re-enabled by an administrator user on a per-project basis. - - - -## Configure globally-allowed IP address ranges - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87579) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md) named `group_ip_restrictions_allow_global`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) in GitLab 15.4. [Feature flag `group_ip_restrictions_allow_global`](https://gitlab.com/gitlab-org/gitlab/-/issues/366445) removed. - -Administrators can set IP address ranges to be combined with [group-level IP restrictions](../../group/access_and_permissions.md#restrict-group-access-by-ip-address). -Use globally-allowed IP addresses to allow aspects of the GitLab installation to work even when group-level IP address -restrictions are set. - -For example, if the GitLab Pages daemon runs on the `10.0.0.0/24` range, you can specify that range as globally-allowed. -This means GitLab Pages can still fetch artifacts from pipelines even if group-level IP address restrictions don't -include the `10.0.0.0/24` range. - -To add a IP address range to the group-level allowlist: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. In **Globally-allowed IP ranges**, provide a list of IP address ranges. This list: - - Has no limit on the number of IP address ranges. - - Has a size limit of 1 GB. - - Applies to both SSH or HTTP authorized IP address ranges. You cannot split - this list by type of authorization. -1. Select **Save changes**. - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, for example `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +<!-- This redirect file can be deleted after <2023-10-14>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/admin_area/user_cohorts.md b/doc/user/admin_area/user_cohorts.md index 6f2798f437c..b0b4facd7db 100644 --- a/doc/user/admin_area/user_cohorts.md +++ b/doc/user/admin_area/user_cohorts.md @@ -1,40 +1,11 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../../administration/user_cohorts.md' +remove_date: '2023-10-11' --- -# Cohorts **(FREE SELF)** +This document was moved to [another location](../../administration/user_cohorts.md). -You can analyze your users' GitLab activities over time. - -How do you interpret the user cohorts table? Let's review an example with the -following user cohorts: - - - -For the cohort of March 2020, three users were added to this server and have -been active since this month. One month later (April 2020), two users are still -active. Five months later (August 2020), one user from this cohort is still -active, or 33% of the original cohort of three that joined in March. - -The **Inactive users** column shows the number of users who were added during -the month, but who never had any activity in the instance. - -How do we measure the activity of users? GitLab considers a user active if: - -- The user signs in. -- The user has Git activity (whether push or pull). -- The user visits pages related to dashboards, projects, issues, or merge - requests ([introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/54947) in GitLab 11.8). -- The user uses the API. -- The user uses the GraphQL API. - -## View user cohorts - -To view user cohorts: - -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Admin Area**. -1. Select **Overview > Users**. -1. Select the **Cohorts** tab. +<!-- This redirect file can be deleted after <2023-10-11>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/ai_features.md b/doc/user/ai_features.md index d5f7201556e..b9bcaec8b57 100644 --- a/doc/user/ai_features.md +++ b/doc/user/ai_features.md @@ -46,20 +46,14 @@ The following feature is in Beta: ## Experiment AI features -[Experiment features](../policy/experiment-beta-support.md#experiment) will soon require -[Experiment features to be enabled](group/manage.md#enable-experiment-features). - -## Third-party AI features - -Third-party AI features require [third-party AI services to be enabled](group/manage.md#enable-third-party-ai-features). - -For Experiment third-party AI features, [Experiment features must be enabled](group/manage.md#enable-experiment-features) as well. +[Experiment](../policy/experiment-beta-support.md#experiment) AI features require +[Experiment features to be enabled](group/manage.md#enable-experiment-features) as well as [third-party AI services to be enabled](group/manage.md#enable-third-party-ai-features). ### Explain Selected Code in the Web UI **(ULTIMATE SAAS)** > Introduced in GitLab 15.11 as an [Experiment](../policy/experiment-beta-support.md#experiment) on GitLab.com. -This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. +This AI feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by Google's Codey for Code Chat (codechat-bison). GitLab can help you get up to speed faster if you: @@ -70,16 +64,32 @@ By using a large language model, GitLab can explain the code in natural language Prerequisites: -- The project must be a public project on GitLab.com. +Additional prerequisites [beyond the two above](#experiment-ai-features). + +- The project must be on GitLab.com. - You must have the GitLab Ultimate subscription tier. -- You must be a member of the project. +- You must be a member of the project with sufficient permissions to view the repository. To explain your code: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select any file in your project that contains code. +1. On the file, select the lines that you want to have explained. +1. On the left side, select the question mark (**{question}**). You might have to scroll to the first line of your selection to view it. This sends the selected code, together with a prompt, to provide an explanation to the large language model. +1. A drawer is displayed on the right side of the page. Wait a moment for the explanation to be generated. +1. Provide feedback about how satisfied you are with the explanation, so we can improve the results. + +You can also have code explained in the context of a merge request. To explain +code in a merge request: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. On the left sidebar, select **Code > Merge requests**, then select your merge request. 1. On the secondary menu, select **Changes**. -1. Go to the file, and select the lines that you want to have explained. +1. On the file you would like explained, select the three dots (**{ellipsis_v}**) and select **View File @ $SHA**. + + A separate browser tab opens and shows the full file with the latest changes. + +1. On the new tab, select the lines that you want to have explained. 1. On the left side, select the question mark (**{question}**). You might have to scroll to the first line of your selection to view it. This sends the selected code, together with a prompt, to provide an explanation to the large language model. 1. A drawer is displayed on the right side of the page. Wait a moment for the explanation to be generated. 1. Provide feedback about how satisfied you are with the explanation, so we can improve the results. @@ -120,20 +130,23 @@ Review the drawer on the right-hand side of your screen. We cannot guarantee that the large language model produces results that are correct. Use the explanation with caution. -### GitLab Chat **(ULTIMATE SAAS)** +### GitLab Duo Chat **(ULTIMATE SAAS)** > Introduced in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). This feature is an [Experiment](../policy/experiment-beta-support.md) on GitLab.com that is powered by OpenAI's GPT-3. It requires the [group-level third-party AI features setting](group/manage.md#enable-third-party-ai-features) to be enabled. -Getting help has never been easier. If you have a question about how the GitLab product works, you can ask product how-to questions and get AI generated support from GitLab Chat. +Getting help has never been easier. If you have a question about how the GitLab product works, you can ask product how-to questions and get AI generated support from GitLab Duo Chat. 1. In the lower-left corner, select the Help icon. -1. Select **Ask in GitLab Chat**. A drawer opens on the right side of your screen. +1. Select **Ask in GitLab Duo Chat**. A drawer opens on the right side of your screen. 1. Enter your question in the chat input box and press **Enter** or select **Send**. It may take a few seconds for the interactive AI chat to search the product documentation and produce an answer. To give feedback, select the **Give Feedback** link. +NOTE: +Only the last 50 messages in the chat history are retained. The chat history expires 3 days after last use. + ### Summarize merge request changes **(ULTIMATE SAAS)** > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10400) in GitLab 16.0 as an [Experiment](../policy/experiment-beta-support.md#experiment). diff --git a/doc/user/analytics/analytics_dashboards.md b/doc/user/analytics/analytics_dashboards.md index e29e02c867f..9d2c91b6bc8 100644 --- a/doc/user/analytics/analytics_dashboards.md +++ b/doc/user/analytics/analytics_dashboards.md @@ -9,21 +9,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w > Introduced in GitLab 15.9 as an [Experiment](../../policy/experiment-beta-support.md#experiment) feature [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `combined_analytics_dashboards`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `combined_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. -## Dashboards +Analytics dashboards help you visualize the collected data. +You can use built-in dashboards or create your own with custom visualizations. -Each project can have an unlimited number of dashboards, only limited by the instances [repository size limits](../project/repository/reducing_the_repo_size_using_git.md#storage-limits). -These dashboards are defined using the GitLab YAML schema, and stored in the `.gitlab/analytics/dashboards/` directory of a project repository. -The dashboard file name and containing directory should be the same, for example `my_dashboard/my_dashboard.yaml`. For more information see [defining a dashboard](#define-a-dashboard). -Each dashboard can reference one or more [visualizations](#define-a-chart-visualization), which are shared across dashboards. - -Project maintainers can enforce approval rules on dashboard changes using features such as [code owners](../project/codeowners/index.md) and [approval rules](../project/merge_requests/approvals/rules.md). -Your dashboard files are versioned in source control with the rest of a project's code. - -### Data sources +## Data sources A data source is a connection to a database or collection of data which can be used by your dashboard filters and visualizations to query and retrieve results. @@ -32,15 +25,96 @@ The following data sources are configured for analytics dashboards: - [Product analytics](../product_analytics/index.md) -### View project dashboards +## Built-in dashboards + +To help you get started with analytics, GitLab provides two built-in dashboards with predefined visualizations: + +- **Audience**, which displays metrics related to traffic, such as number of users and sessions. +- **Behavior**, which displays metrics related to user activity, such as number of page views and events. + +These dashboards are labeled **By GitLab**, and you cannot edit them. +Instead, you can create a custom dashboard with a similar style. + +## Custom dashboards -To view a list of dashboards for a project: +With custom dashboards, you can design and create visualizations for the metrics that are most relevant to your use case. +You can create custom dashboards with the dashboard designer. + +- Each project can have an unlimited number of dashboards. +The only limitation might be the [repository size limit](../project/repository/reducing_the_repo_size_using_git.md#storage-limits). +- Each dashboard can reference one or more [visualizations](#define-a-chart-visualization). +- Visualizations are shared across dashboards. + +Project maintainers can enforce approval rules on dashboard changes with features such as [code owners](../project/codeowners/index.md) and [approval rules](../project/merge_requests/approvals/rules.md). +Your dashboard files are versioned in source control with the rest of a project's code. + +## Dashboard designer + +> Introduced in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +NOTE: +This feature does not work in conjunction with the `product_analytics_snowplow_support` feature flag. + +You can use the dashboard designer to: + +- Create custom dashboards. +- Rename custom dashboards. +- Add visualizations to new and existing custom dashboards. +- Resize or move panels in custom dashboards. + +## View project dashboards + +To view a list of dashboards (both built-in and custom) for a project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Analyze > Dashboards**. 1. From the list of available dashboards, select the dashboard you want to view. -### Define a dashboard +## Change the location of dashboards + +You can change the location of your project or group dashboards. + +### Group dashboards + +NOTE: +This feature will be connected to group-level dashboards as part of [issue #411572](https://gitlab.com/gitlab-org/gitlab/-/issues/411572). + +To change the location of a group's dashboards: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find the project you want to store your dashboard files in. + The project must belong to the group for which you create the dashboards. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Settings > General**. +1. Expand **Analytics**. +1. In the **Analytics Dashboards** section, select your dashboard files project. +1. Select **Save changes**. + +### Project dashboards + +Dashboards are usually defined in the project where the analytics data is being retrieved from. +However, you can also have a separate project for dashboards. +This is recommended if you want to enforce specific access rules to the dashboard definitions or share dashboards across multiple projects. + +NOTE: +You can share dashboards only between projects that are located in the same group. + +To change the location of project dashboards: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project, + or select **Create new...** (**{plus}**) and **New project/repository** + to create the project to store your dashboard files. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and find the analytics project. +1. Select **Settings > General**. +1. Expand **Analytics**. +1. In the **Analytics Dashboards** section, select your dashboard files project. +1. Select **Save changes**. + +## Define a dashboard To define a dashboard: @@ -67,7 +141,7 @@ and one visualization (line chart) that applies to all dashboards, the file stru │ └── example_line_chart.yaml ``` -### Define a chart visualization +## Define a chart visualization You can define different charts, and add visualization options to some of them: @@ -91,65 +165,7 @@ create a `line_chart.yaml` file with the following required fields: - data - options -### Change the location of project dashboards - -Dashboards are usually defined in the project where analytics data is being retrieved. -However, you can also have a separate project for dashboards. -This is recommended if you want to enforce specific access rules to the dashboard definitions or share dashboards across multiple projects. - -NOTE: -You can share dashboards only between projects that are located in the same group. - -To change the location of project dashboards: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project, - or select **Create new...** (**{plus}**) and **New project/repository** - to create the project to store your dashboard files. -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and find the project you want to use the dashboards for. -1. Select **Settings > General**. -1. Expand **Analytics**. -1. In the **Analytics Dashboards** section, select the project that contains the dashboard files. -1. Select **Save changes**. - -### Change the location of group dashboards - -NOTE: -This feature will be connected to group-level dashboards in [issue 411572](https://gitlab.com/gitlab-org/gitlab/-/issues/411572). - -If you want to use dashboards for a group, you must store the dashboard files in a project that belongs to that group. -You can change the source project of a group's dashboards at any time. - -To change the location of a group's dashboards: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Settings > General**. -1. Expand **Analytics**. -1. In the **Analytics Dashboards** section, select the project that contains the dashboard files. -1. Select **Save changes**. - -## Dashboards designer - -> Introduced in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `combined_analytics_dashboards_editor`. -On GitLab.com, this feature is not available. -This feature is not ready for production use. - -NOTE: -This feature does not work in conjunction with the `product_analytics_snowplow_support` feature flag. - -You can use the dashboards designer to: - -- Create custom dashboards -- Rename custom dashboards -- Add visualizations to new and existing custom dashboards -- Resize or move panels within custom dashboards - -You cannot edit the built-in dashboards labeled as `By GitLab`. -To edit these dashboards you should create a new custom dashboard which uses the same visualizations. - -### Create a custom dashboard +## Create a custom dashboard To create a custom dashboard: @@ -161,9 +177,9 @@ To create a custom dashboard: 1. Optional. Drag or resize the selected panel how you prefer. 1. Select **Save**. -### Edit a custom dashboard +## Edit a custom dashboard -You can edit your custom dashboard's title and add or resize visualizations within the dashboard designer. +You can edit your custom dashboard's title and add or resize visualizations in the dashboard designer. To edit an existing custom dashboard: @@ -175,3 +191,20 @@ To edit an existing custom dashboard: 1. Optional. From the **Add visualizations** list on the right, select other visualizations to add to the dashboard. 1. Optional. In the dashboard, select a panel and drag or resize it how you prefer. 1. Select **Save**. + +## Troubleshooting + +### `Something went wrong while loading the dashboard.` + +If the dashboard displays a global error message that data could not be loaded, first try reloading the page. If the error persists: + +- Check that your configurations match the [JSON schema](#define-a-dashboard) defined in `ee/app/validators/json_schemas/analytics_dashboard.json`. +- For product analytics, check your [admin and project settings](../product_analytics/index.md#project-level-settings), and make sure they are set up correctly. + +### Dashboard panel error + +If a dashboard panel displays an error message: + +- Check your [Cube query](../product_analytics/index.md#product-analytics-dashboards) and [visualization](../analytics/analytics_dashboards.md#define-a-chart-visualization) +configurations, and make sure they are set up correctly. +- For [product analytics](../product_analytics/index.md), also check that your visualization's Cube query is valid. diff --git a/doc/user/analytics/dora_metrics.md b/doc/user/analytics/dora_metrics.md index 6b7b1d87843..bdfeffcec05 100644 --- a/doc/user/analytics/dora_metrics.md +++ b/doc/user/analytics/dora_metrics.md @@ -43,7 +43,7 @@ High deployment frequency means you can get feedback sooner and iterate faster t ### How deployment frequency is calculated In GitLab, Deployment frequency is measured by the average number of deployments per day to a given environment, based on the deployment's end time (its `finished_at` property). -GitLab calculates the deployment frequency from the number of finished deployments on the given day. +GitLab calculates the deployment frequency from the number of finished deployments on the given day. Only successful deployments (`Deployment.statuses = success`) are counted. The calculation takes into account the production `environment tier` or the environments named `production/prod`. The environment must be part of the production deployment tier for its deployment information to appear on the graphs. diff --git a/doc/user/analytics/index.md b/doc/user/analytics/index.md index e78f55911e6..c057a8b193d 100644 --- a/doc/user/analytics/index.md +++ b/doc/user/analytics/index.md @@ -27,7 +27,7 @@ GitLab provides several analytics features at the group level. Some of these fea - [Issue](../group/issues_analytics/index.md) - [Productivity](productivity_analytics.md) - [Repositories](../group/repositories_analytics/index.md) -- [Value Stream](../group/value_stream_analytics/index.md) +- [Value Stream Management Analytics](value_stream_analytics.md), and [Value Stream Management Dashboard](value_streams_dashboard.md) ## Project-level analytics @@ -43,7 +43,7 @@ You can use GitLab to review analytics at the project level. Some of these featu - [Merge Request](merge_request_analytics.md), enabled with the `project_merge_request_analytics` [feature flag](../../development/feature_flags/index.md#enabling-a-feature-flag-locally-in-development) - [Repository](repository_analytics.md) -- [Value Stream](value_stream_analytics.md) +- [Value Stream Management Analytics](value_stream_analytics.md), and [Value Stream Management Dashboard](value_streams_dashboard.md) ### Remove project analytics from the left sidebar diff --git a/doc/user/analytics/value_streams_dashboard.md b/doc/user/analytics/value_streams_dashboard.md index 9b332d78060..a853263d20f 100644 --- a/doc/user/analytics/value_streams_dashboard.md +++ b/doc/user/analytics/value_streams_dashboard.md @@ -11,15 +11,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/392734) in GitLab 16.0. Feature flag `group_analytics_dashboards_page` removed. You can leave feedback on dashboard bugs or functionality in [issue 381787](https://gitlab.com/gitlab-org/gitlab/-/issues/381787). +For more information, see also the [Value Stream Management category direction page](https://about.gitlab.com/direction/plan/value_stream_management/). -The Value Streams Dashboard is a customizable dashboard that enables decision-makers to identify trends, patterns, and opportunities for digital transformation improvements. -This page is a work in progress, and we're updating the information as we add more features. -For more information, see the [Value Stream Management category direction page](https://about.gitlab.com/direction/plan/value_stream_management/). +The Value Streams Dashboard is a customizable dashboard you can use to identify trends, patterns, and opportunities for digital transformation improvements. -## Initial use case - -Our initial use case is focused on providing the ability to compare software delivery metrics. -This comparison can help decision-makers understand whether projects and groups are improving. +With the Value Streams Dashboard, you can compare software delivery metrics. +This comparison can help you understand whether projects and groups are improving. The Value Streams Dashboard includes the following metrics: @@ -57,7 +54,7 @@ To view the value streams dashboard: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. 1. Select **Analyze > Value stream analytics**. -1. Below the **Filter results** text box, in the **Key metrics** row, select **Value Streams Dashboard / DORA**. +1. Below the **Filter results** text box, in the **Lifecycle metrics** row, select **Value Streams Dashboard / DORA**. 1. Optional. To open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL (for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). @@ -79,7 +76,8 @@ For example, the parameter `query=gitlab-org/gitlab-ui,gitlab-org/plan-stage` di ### Using YAML configuration -To change the default content of the page, you need to create a YAML configuration file in a project of your choice. Query parameters can still be used to override the YAML configuration. +To customize the default content of the page, you need to create a YAML configuration file in a project of your choice. In this file you can define various settings and parameters, such as title, description, and number of panels and labels filters. The file is schema-driven and managed with version control systems like Git. This enables tracking and maintaining a history of configuration changes, reverting to previous versions if necessary, and collaborating effectively with team members. +Query parameters can still be used to override the YAML configuration. First, you need to set up the project. @@ -110,12 +108,25 @@ description: 'Custom description' # title - Change the title of the panel. [optional] # data.namespace - The Group or Project path to use for the chart panel. # data.exclude_metrics - Hide rows by metric ID from the chart panel. +# data.filter_labels - +# Only show results for data that matches the queried label(s). If multiple labels are provided, +# only a single label needs to match for the data to be included in the results. +# Compatible metrics (other metrics will be automatically excluded): +# * lead_time +# * cycle_time +# * issues +# * issues_completed +# * merge_request_throughput +# (This option is dependant on the `vsd_graphql_dora_and_flow_metrics` feature.) panels: - title: 'My Custom Project' data: namespace: group/my-custom-project - data: namespace: group/another-project + filter_labels: + - in_development + - in_review - title: 'My Custom Group' data: namespace: group/my-custom-group @@ -134,6 +145,9 @@ panels: namespace: my-group ``` +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview of editing label filters in the configuration file, see [GitLab Value Streams Dashboard - Label filters demo](https://www.youtube.com/watch?v=4qDAHCxCfik). + ## Dashboard metrics and drill-down reports | Metric | Description | Drill-down report | Documentation page | ID | @@ -142,8 +156,8 @@ panels: | Lead time for changes | The time to successfully deliver a commit into production. This metric reflects the efficiency of CI/CD pipelines. | [Lead time tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=lead-time) | [Lead time for changes](dora_metrics.md#lead-time-for-changes) | `lead_time_for_changes` | | Time to restore service | The time it takes an organization to recover from a failure in production. | [Time to restore service tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=time-to-restore-service) | [Time to restore service](dora_metrics.md#time-to-restore-service) | `time_to_restore_service` | | Change failure rate | Percentage of deployments that cause an incident in production. | [Change failure rate tab](https://gitlab.com/groups/gitlab-org/-/analytics/ci_cd?tab=change-failure-rate) | [Change failure rate](dora_metrics.md#change-failure-rate) | `change_failure_rate` | -| Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | `lead_time` | -| Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#key-metrics) | `cycle_time` | +| Lead time | Median time from issue created to issue closed. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#lifecycle-metrics) | `lead_time` | +| Cycle time | Median time from the earliest commit of a linked issue's merge request to when that issue is closed. | [VSA overview](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [View the lead time and cycle time for issues](../group/value_stream_analytics/index.md#lifecycle-metrics) | `cycle_time` | | New issues | Number of new issues created. | [Issue Analytics](https://gitlab.com/groups/gitlab-org/-/issues_analytics) | Issue analytics [for projects](issue_analytics.md) and [for groups](../../user/group/issues_analytics/index.md) | `issues` | | Closed issues | Number of issues closed by month. | [Value Stream Analytics](https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics) | [Value Stream Analytics](../group/value_stream_analytics/index.md) | `issues_completed` | | Number of deploys | Total number of deploys to production. | [Merge Request Analytics](https://gitlab.com/gitlab-org/gitlab/-/analytics/merge_request_analytics) | [Merge request analytics](merge_request_analytics.md) | `deploys` | diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 28e72816a99..e8feb0f4a59 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -2032,7 +2032,7 @@ Follow these steps to view details of a fuzzing fault: 1. You can view faults in a project, or a merge request: - - In a project, go to the project's **Security and Compliance > Vulnerability Report** + - In a project, go to the project's **Secure > Vulnerability report** page. This page shows all vulnerabilities from the default branch only. - In a merge request, go the merge request's **Security** section and select the **Expand** button. API Fuzzing faults are available in a section labeled @@ -2452,7 +2452,7 @@ See the following documentation sections for assistance: See [Performance Tuning and Testing Speed](#performance-tuning-and-testing-speed) -### Error waiting for API Fuzzing 'http://127.0.0.1:5000' to become available +### Error: `Error waiting for API Fuzzing 'http://127.0.0.1:5000' to become available` A bug exists in versions of the API Fuzzing analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the API Fuzzing analyzer. diff --git a/doc/user/application_security/comparison_dependency_and_container_scanning.md b/doc/user/application_security/comparison_dependency_and_container_scanning.md new file mode 100644 index 00000000000..d02d94b7a3e --- /dev/null +++ b/doc/user/application_security/comparison_dependency_and_container_scanning.md @@ -0,0 +1,34 @@ +--- +stage: Secure +group: Composition Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Dependency Scanning compared to Container Scanning + +GitLab offers both [Dependency Scanning](dependency_scanning/index.md) and +[Container Scanning](container_scanning/index.md) to ensure coverage for all of these +dependency types. To cover as much of your risk area as possible, we encourage you to use all of our +security scanning tools: + +- Dependency Scanning analyzes your project and tells you which software dependencies, + including upstream dependencies, have been included in your project, and what known + risks the dependencies contain. +- Container Scanning analyzes your containers and tells you about known risks in the operating + system's (OS) packages. + +The following table summarizes which types of dependencies each scanning tool can detect: + +| Feature | Dependency Scanning | Container Scanning | +|----------------------------------------------------------------------------------------------|---------------------|----------------------------------------------| +| Identify the manifest, lock file, or static file that introduced the dependency | **{check-circle}** | **{dotted-circle}** | +| Development dependencies | **{check-circle}** | **{dotted-circle}** | +| Dependencies in a lock file committed to your repository | **{check-circle}** | **{check-circle}** <sup>1</sup> | +| Binaries built by Go | **{dotted-circle}** | **{check-circle}** <sup>2</sup> <sup>3</sup> | +| Dynamically-linked language-specific dependencies installed by the Operating System | **{dotted-circle}** | **{check-circle}** <sup>3</sup> | +| Operating system dependencies | **{dotted-circle}** | **{check-circle}** | +| Language-specific dependencies installed on the operating system (not built by your project) | **{dotted-circle}** | **{check-circle}** | + +1. Lock file must be present in the image to be detected. +1. Binary file must be present in the image to be detected. +1. Only when using Trivy. diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 042ed0190c4..791a73bfdc2 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -32,7 +32,8 @@ you wrote yourself. GitLab offers both Container Scanning and [Dependency Scanning](../dependency_scanning/index.md) to ensure coverage for all of these dependency types. To cover as much of your risk area as -possible, we encourage you to use all of our security scanners. +possible, we encourage you to use all of our security scanners. For a comparison of these features, see +[Dependency Scanning compared to Container Scanning](../comparison_dependency_and_container_scanning.md). ## Overview @@ -240,8 +241,7 @@ When you enable this feature, you may see [duplicate findings](../terminology/in in the [Vulnerability Report](../vulnerability_report/index.md) if [Dependency Scanning](../dependency_scanning/index.md) is enabled for your project. This happens because GitLab can't automatically deduplicate findings -across different types of scanning tools. Reference [this comparison](../dependency_scanning/index.md#dependency-scanning-compared-to-container-scanning) -between GitLab Dependency Scanning and Container Scanning for more details on which types of dependencies are likely to be duplicated. +across different types of scanning tools. To understand which types of dependencies are likely to be duplicated, see [Dependency Scanning compared to Container Scanning](../comparison_dependency_and_container_scanning.md). #### Available CI/CD variables @@ -328,7 +328,7 @@ To enable Container Scanning in a project, create a merge request from the Secur page: 1. In the project where you want to enable Container Scanning, go to - **Security and Compliance > Security configuration**. + **Secure > Security configuration**. 1. In the **Container Scanning** row, select **Configure with a merge request**. This automatically creates a merge request with the changes necessary to enable Container Scanning. diff --git a/doc/user/application_security/dast/browser_based.md b/doc/user/application_security/dast/browser_based.md index 7b263e5817d..0338555598c 100644 --- a/doc/user/application_security/dast/browser_based.md +++ b/doc/user/application_security/dast/browser_based.md @@ -189,7 +189,7 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_BROWSER_MAX_ACTIONS` | number | `10000` | The maximum number of actions that the crawler performs. For example, selecting a link, or filling a form. | | `DAST_BROWSER_MAX_DEPTH` | number | `10` | The maximum number of chained actions that the crawler takes. For example, `Click -> Form Fill -> Click` is a depth of three. | | `DAST_BROWSER_MAX_RESPONSE_SIZE_MB` | number | `15` | The maximum size of a HTTP response body. Responses with bodies larger than this are blocked by the browser. Defaults to 10 MB. | -| `DAST_BROWSER_NAVIGATION_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis after a navigation completes. | +| `DAST_BROWSER_NAVIGATION_STABILITY_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `7s` | The maximum amount of time to wait for a browser to consider a page loaded and ready for analysis after a navigation completes. Defaults to `800ms`.| | `DAST_BROWSER_NAVIGATION_TIMEOUT` | [Duration string](https://pkg.go.dev/time#ParseDuration) | `15s` | The maximum amount of time to wait for a browser to navigate from one page to another. | | `DAST_BROWSER_NUMBER_OF_BROWSERS` | number | `3` | The maximum number of concurrent browser instances to use. For shared runners on GitLab.com, we recommended a maximum of three. Private runners with more resources may benefit from a higher number, but are likely to produce little benefit after five to seven instances. | | `DAST_BROWSER_PAGE_LOADING_SELECTOR` | selector | `css:#page-is-loading` | Selector that when is no longer visible on the page, indicates to the analyzer that the page has finished loading and the scan can continue. Cannot be used with `DAST_BROWSER_PAGE_READY_SELECTOR`. | @@ -212,6 +212,75 @@ For authentication CI/CD variables, see [Authentication](authentication.md). | `DAST_WEBSITE` | URL | `https://example.com` | The URL of the website to scan. | | `SECURE_ANALYZERS_PREFIX` | URL | `registry.organization.com` | Set the Docker registry base address from which to download the analyzer. | +## Managing scope + +Scope controls what URLs DAST follows when crawling the target application. Properly managed scope minimizes scan run time while ensuring only the target application is checked for vulnerabilities. + +### Types of scope + +There are three types of scope: + +- in scope +- out of scope +- excluded from scope + +#### In scope + +DAST follows in-scope URLs and searches the DOM for subsequent actions to perform to continue the crawl. +Recorded in-scope HTTP messages are passively checked for vulnerabilities and used to build attacks when running a full scan. + +#### Out of scope + +DAST follows out-of-scope URLs for non-document content types such as image, stylesheet, font, script, or AJAX request. +[Authentication](#scope-works-differently-during-authentication) aside, DAST does not follow out-of-scope URLs for full page loads, such as when clicking a link to an external website. +Except for passive checks that search for information leaks, recorded HTTP messages for out-of-scope URLs are not checked for vulnerabilities. + +#### Excluded from scope + +DAST does not follow excluded-from-scope URLs. Except for passive checks that search for information leaks, recorded HTTP messages for excluded-from-scope URLs are not checked for vulnerabilities. + +### Scope works differently during authentication + +Many target applications have an authentication process that depends on external websites, such as when using an identity access management provider for single sign on (SSO). +To ensure that DAST can authenticate with these providers, DAST follows out-of-scope URLs for full page loads during authentication. DAST does not follow excluded-from-scope URLs. + +### How DAST blocks HTTP requests + +DAST instructs the browser to make the HTTP request as usual when blocking a request due to scope rules. The request is subsequently intercepted and rejected with the reason `BlockedByClient`. +This approach allows DAST to record the HTTP request while ensuring it never reaches the target server. Passive checks such as [200.1](checks/200.1.md) use these recorded requests to verify information sent to external hosts. + +### How to configure scope + +By default, URLs matching the host of the target application are considered in-scope. All other hosts are considered out-of-scope. + +Scope is configured using the following variables: + +- Use `DAST_BROWSER_ALLOWED_HOSTS` to add in-scope hosts. +- Use `DAST_BROWSER_IGNORED_HOSTS` to add to out-of-scope hosts. +- Use `DAST_BROWSER_EXCLUDED_HOSTS` to add to excluded-from-scope hosts. +- Use `DAST_EXCLUDE_URLS` to set specific URLs to be excluded-from-scope. + +Rules: + +- Excluding a host is given priority over ignoring a host, which is given priority over allowing a host. +- Configuring scope for a host does not configure scope for the subdomains of that host. +- Configuring scope for a host does not configure scope for all ports on that host. + +The following could be a typical configuration: + +```yaml +include: + - template: DAST.gitlab-ci.yml + +dast: + variables: + DAST_WEBSITE: "https://my.site.com" # my.site.com URLs are considered in-scope by default + DAST_BROWSER_ALLOWED_HOSTS: "api.site.com:8443" # include the API as part of the scan + DAST_BROWSER_IGNORED_HOSTS: "analytics.site.com" # explicitly disregard analytics from the scan + DAST_BROWSER_EXCLUDED_HOSTS: "ads.site.com" # don't visit any URLs on the ads subdomain + DAST_EXCLUDE_URLS: "https://my.site.com/user/logout" # don't visit this URL +``` + ## Vulnerability detection Vulnerability detection is gradually being migrated from the default Zed Attack Proxy (ZAP) solution diff --git a/doc/user/application_security/dast/browser_based_troubleshooting.md b/doc/user/application_security/dast/browser_based_troubleshooting.md index 6cc80bcfbc3..f659001e7c5 100644 --- a/doc/user/application_security/dast/browser_based_troubleshooting.md +++ b/doc/user/application_security/dast/browser_based_troubleshooting.md @@ -100,7 +100,7 @@ For example, the following log entry has level `INFO`, is part of the `CRAWL` lo Logs are sent either to file or to console (the CI/CD job log). You can configure each destination to accept different logs using the environment variables `DAST_BROWSER_LOG` for console logs and `DAST_BROWSER_FILE_LOG` for file logs. -In the following example, the file log defaults to `DEBUG` level, the console log defaults to `INFO` level and logs the `AUTH` module at `DEBUG` level. +For example: ```yaml include: @@ -108,9 +108,10 @@ include: dast: variables: - DAST_BROWSER_LOG: "auth:debug" - DAST_BROWSER_FILE_LOG: "loglevel:debug" - DAST_BROWSER_FILE_LOG_PATH: "/zap/wrk/dast-scan.log" + DAST_BROWSER_SCAN: "true" + DAST_BROWSER_LOG: "auth:debug" # console log defaults to INFO level, logs AUTH module at DEBUG + DAST_BROWSER_FILE_LOG: "loglevel:debug,cache:warn" # file log defaults to DEBUG level, logs CACHE module at WARN + DAST_BROWSER_FILE_LOG_PATH: "$CI_PROJECT_DIR/dast-scan.log" # Save the file log in the project directory so it can be recognized as an artifact artifacts: paths: - dast-scan.log diff --git a/doc/user/application_security/dast/checks/16.9.md b/doc/user/application_security/dast/checks/16.9.md index b0ba502b578..c63a620794e 100644 --- a/doc/user/application_security/dast/checks/16.9.md +++ b/doc/user/application_security/dast/checks/16.9.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description A `Content-Security-Policy-Report-Only` (CSPRO) was identified on the target site. CSP-Report-Only headers -aid in determining how to implement a `Content-Security-Policy` that does not disrupt normal use of the target +aid in determining how to implement a `Content-Security-Policy` that does not disrupt use of the target site. ## Remediation diff --git a/doc/user/application_security/dast/proxy-based.md b/doc/user/application_security/dast/proxy-based.md index 499efd3f60d..0eec04bfeff 100644 --- a/doc/user/application_security/dast/proxy-based.md +++ b/doc/user/application_security/dast/proxy-based.md @@ -461,132 +461,95 @@ The DAST job does not require the project's repository to be present when runnin ## On-demand scans -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.2. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/218465) in GitLab 13.3. -> - The saved scans feature was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5100) in GitLab 13.9. -> - The option to select a branch was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4847) in GitLab 13.10. -> - DAST branch selection [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/322672) in GitLab 13.11. -> - Auditing for DAST profile management was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. +> Auditing for DAST profile management [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217872) in GitLab 14.1. An on-demand DAST scan runs outside the DevOps life cycle. Changes in your repository don't trigger -the scan. You must either start it manually, or schedule it to run. - -An on-demand DAST scan: - -- Can run a specific combination of a [site profile](#site-profile) and a - [scanner profile](#scanner-profile). -- Is associated with your project's default branch. -- Is saved on creation so it can be run later. +the scan. You must either start it manually, or schedule it to run. For on-demand DAST scans, +a [site profile](#site-profile) defines **what** is to be scanned, and a +[scanner profile](#scanner-profile) defines **how** the application is to be scanned. An on-demand scan can be run in active or passive mode: -- _Passive mode_ is the default and runs a ZAP Baseline Scan. -- _Active mode_ runs a ZAP Full Scan which is potentially harmful to the site being scanned. To - minimize the risk of accidental damage, running an active scan requires a [validated site profile](#site-profile-validation). +- **Passive mode**: The default mode, which runs a ZAP Baseline Scan. +- **Active mode**: Runs a ZAP Full Scan which is potentially harmful to the site being scanned. To + minimize the risk of accidental damage, running an active scan requires a + [validated site profile](#site-profile-validation). ### View on-demand DAST scans -To view on-demand scans, from your project's home page, go to **Security & Compliance > On-demand -scans** in the left sidebar. +To view on-demand scans: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. +1. Select **Secure > On-demand scans**. On-demand scans are grouped by their status. The scan library contains all available on-demand scans. -From the **On-demand scans** page you can: - -- [Run](#run-an-on-demand-dast-scan) an on-demand scan. -- View the results of an on-demand scan. -- Cancel (**{cancel}**) a pending or running on-demand scan. -- Retry (**{retry}**) a scan that failed, or succeeded with warnings. -- [Edit](#edit-an-on-demand-scan) (**{pencil}**) an on-demand scan's settings. -- [Delete](#delete-an-on-demand-scan) an on-demand scan. - ### Run an on-demand DAST scan Prerequisites: - You must have permission to run an on-demand DAST scan against a protected branch. The default - branch is automatically protected. For more information, read + branch is automatically protected. For more information, see [Pipeline security on protected branches](../../../ci/pipelines/index.md#pipeline-security-on-protected-branches). -- A [scanner profile](#create-a-scanner-profile). -- A [site profile](#create-a-site-profile). -- If you are running an active scan the site profile must have been [validated](#validate-a-site-profile). - -You can run an on-demand scan immediately, once at a scheduled date and time or at a specified -frequency: - -- Every day -- Every week -- Every month -- Every 3 months -- Every 6 months -- Every year - -To run an on-demand scan immediately, either: - -- [Create and run an on-demand scan immediately](#create-and-run-an-on-demand-scan-immediately). -- [Run a previously saved on-demand scan](#run-a-saved-on-demand-scan). - -To run an on-demand scan either at a scheduled date or frequency, read -[Schedule an on-demand scan](#schedule-an-on-demand-scan). - -#### Create and run an on-demand scan immediately - -1. From your project's home page, go to **Security & Compliance > On-demand Scans** in the left - sidebar. -1. Select **New scan**. -1. Complete the **Scan name** and **Description** fields. -1. In GitLab 13.10 and later, select the desired branch from the **Branch** dropdown list. -1. In **Scanner profile**, select a scanner profile from the dropdown list. -1. In **Site profile**, select a site profile from the dropdown list. -1. To run the on-demand scan immediately, select **Save and run scan**. Otherwise, select - **Save scan** to [run](#run-a-saved-on-demand-scan) it later. - -The on-demand DAST scan runs and the project's dashboard shows the results. -#### Run a saved on-demand scan - -To run a saved on-demand scan: +To run an existing on-demand scan: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Secure > On-demand scans**. 1. Select the **Scan library** tab. 1. In the scan's row, select **Run scan**. - If the branch saved in the scan no longer exists, you must first - [edit the scan](#edit-an-on-demand-scan), select a new branch, and save the edited scan. + If the branch saved in the scan no longer exists, you must: + + 1. [Edit the scan](#edit-an-on-demand-scan). + 1. Select a new branch. + 1. Save the edited scan. The on-demand DAST scan runs, and the project's dashboard shows the results. -#### Schedule an on-demand scan +#### Create an on-demand scan > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.3. [Deployed behind the `dast_on_demand_scans_scheduler` flag](../../../administration/feature_flags.md), disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.4. > - [Feature flag `dast_on_demand_scans_scheduler` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/328749) in GitLab 14.5. -To schedule a scan: +After you create an on-demand scan, you can: -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +- Run it immediately. +- Save it to be run in the future. +- Schedule it to be run at a specified schedule. + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. 1. Select **Secure > On-demand scans**. 1. Select **New scan**. -1. Complete the **Scan name** and **Description** text boxes. -1. In GitLab 13.10 and later, from the **Branch** dropdown list, select the desired branch. -1. In the **Scanner profile** section, from the dropdown list, select a scanner profile. -1. In the **Site profile** section, from the dropdown list, select a site profile. -1. Select **Schedule scan**. -1. In the **Start time** section, select a time zone, date, and time. -1. From the **Repeats** dropdown list, select your desired frequency: - - To run the scan once, select **Never**. - - For a recurring scan, select any other option. -1. To run the on-demand scan immediately, select **Save and run scan**. To [run](#run-a-saved-on-demand-scan) it according to the schedule you set, select - **Save scan**. +1. Complete the **Scan name** and **Description** fields. +1. In the **Branch** dropdown list, select the desired branch. +1. Select **Select scanner profile** or **Change scanner profile** to open the drawer, and either: + - Select a scanner profile from the drawer, **or** + - Select **New profile**, create a [scanner profile](#scanner-profile), then select **Save profile**. +1. Select **Select site profile** or **Change site profile** to open the drawer, and either: + - Select a site profile from the **Site profile library** drawer, or + - Select **New profile** create a [site profile](#site-profile), then select **Save profile**. +1. To run the on-demand scan: + + - Immediately, select **Save and run scan**. + - In the future, select **Save scan**. + - On a schedule: + + - Turn on the **Enable scan schedule** toggle. + - Complete the schedule fields. + - Select **Save scan**. + +The on-demand DAST scan runs as specified and the project's dashboard shows the results. ### View details of an on-demand scan To view details of an on-demand scan: -1. From your project's home page, go to **Security & Compliance > On-demand scans**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > On-demand scans**. 1. Select the **Scan library** tab. 1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**. @@ -594,17 +557,19 @@ To view details of an on-demand scan: To edit an on-demand scan: -1. From your project's home page, go to **Security & Compliance > On-demand scans**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > On-demand scans**. 1. Select the **Scan library** tab. 1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Edit**. -1. Edit the form. +1. Edit the saved scan's details. 1. Select **Save scan**. ### Delete an on-demand scan To delete an on-demand scan: -1. From your project's home page, go to **Security & Compliance > On-demand scans**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > On-demand scans**. 1. Select the **Scan library** tab. 1. In the saved scan's row select **More actions** (**{ellipsis_v}**), then select **Delete**. 1. On the confirmation dialog, select **Delete**. @@ -645,66 +610,63 @@ This data can only be read and decrypted with a valid secrets file. ### Site profile validation -> - Site profile validation [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/233020) in GitLab 13.8. -> - Meta tag validation [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6460) in GitLab 14.2. +> Meta tag validation [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6460) in GitLab 14.2. Site profile validation reduces the risk of running an active scan against the wrong website. A site -must be validated before an active scan can run against it. The site validation methods are as -follows: +must be validated before an active scan can run against it. Each of the site validation methods are +equivalent in functionality, so use whichever is most suitable: -- _Text file validation_ requires a text file be uploaded to the target site. The text file is +- **Text file validation**: Requires a text file be uploaded to the target site. The text file is allocated a name and content that is unique to the project. The validation process checks the file's content. -- _Header validation_ requires the header `Gitlab-On-Demand-DAST` be added to the target site, +- **Header validation**: Requires the header `Gitlab-On-Demand-DAST` be added to the target site, with a value unique to the project. The validation process checks that the header is present, and checks its value. -- _Meta tag validation_ requires the meta tag named `gitlab-dast-validation` be added to the target site, - with a value unique to the project. Make sure it's added to the `<head>` section of the page. The validation process checks that the meta tag is present, and - checks its value. - -All these methods are equivalent in functionality. Use whichever is feasible. - -In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/324990), site profile -validation happens in a CI job using the [GitLab Runner](../../../ci/runners/index.md). +- **Meta tag validation**: Requires the meta tag named `gitlab-dast-validation` be added to the + target site, with a value unique to the project. Make sure it's added to the `<head>` section of + the page. The validation process checks that the meta tag is present, and checks its value. ### Create a site profile To create a site profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. Select **Manage** in the **DAST Profiles** row. -1. Select **New > Site Profile**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select **New > Site profile**. 1. Complete the fields then select **Save profile**. -The site profile is created. +The site profile is saved, for use in an on-demand scan. ### Edit a site profile -If a site profile is linked to a security policy, a user cannot edit the profile from this page. See -[Scan execution policies](../policies/scan-execution-policies.md) -for more information. +NOTE: +If a site profile is linked to a security policy, you cannot edit the profile from this page. See +[Scan execution policies](../policies/scan-execution-policies.md) for more information. When a validated site profile's file, header, or meta tag is edited, the site's [validation status](#site-profile-validation) is revoked. To edit a site profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row select **Manage**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. 1. In the profile's row select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the fields then select **Save profile**. ### Delete a site profile +NOTE: If a site profile is linked to a security policy, a user cannot delete the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. +See [Scan execution policies](../policies/scan-execution-policies.md) for more information. To delete a site profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row select **Manage**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Select the **Site Profiles** tab. 1. In the profile's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete** to confirm the deletion. @@ -766,8 +728,9 @@ have their validation status revoked. To revoke a site profile's validation status: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row select **Manage**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. 1. Beside the validated profile, select **Revoke validation**. The site profile's validation status is revoked. @@ -818,9 +781,6 @@ app.get('/dast-website-target', function(req, res) { ## Scanner profile -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222767) in GitLab 13.4. -> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/225804) in GitLab 13.5: scan mode, AJAX spider, debug messages. - A scanner profile defines the configuration details of a security scanner. A scanner profile can be referenced in `.gitlab-ci.yml` and on-demand scans. @@ -839,38 +799,41 @@ A scanner profile contains: To create a scanner profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row, select **Manage**. -1. Select **New > Scanner Profile**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select **New > Scanner profile**. 1. Complete the form. For details of each field, see [Scanner profile](#scanner-profile). 1. Select **Save profile**. ### Edit a scanner profile -If a scanner profile is linked to a security policy, a user cannot edit the profile from this page. -See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. +NOTE: +If a scanner profile is linked to a security policy, you cannot edit the profile from this page. +For more information, see [Scan execution policies](../policies/scan-execution-policies.md). To edit a scanner profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row, select **Manage**. -1. Select the **Scanner Profiles** tab. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select the **Scanner profiles** tab. 1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Edit**. 1. Edit the form. 1. Select **Save profile**. ### Delete a scanner profile +NOTE: If a scanner profile is linked to a security policy, a user cannot delete the profile from this -page. See [Scan execution policies](../policies/scan-execution-policies.md) -for more information. +page. For more information, see [Scan execution policies](../policies/scan-execution-policies.md). To delete a scanner profile: -1. From your project's home page, go to **Security & Compliance > Configuration**. -1. In the **DAST Profiles** row, select **Manage**. -1. Select the **Scanner Profiles** tab. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Secure > Security configuration**. +1. In the **Dynamic Application Security Testing (DAST)** section, select **Manage profiles**. +1. Select the **Scanner profiles** tab. 1. In the scanner's row, select the **More actions** (**{ellipsis_v}**) menu, then select **Delete**. 1. Select **Delete**. diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index d494259ecc4..b03f9102dae 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -1950,7 +1950,7 @@ Follow these steps to view details of a vulnerability: 1. You can view vulnerabilities in a project, or a merge request: - - In a project, go to the project's **Security and Compliance > Vulnerability Report** + - In a project, go to the project's **Secure > Vulnerability report** page. This page shows all vulnerabilities from the default branch only. - In a merge request, go the merge request's **Security** section and select the **Expand** button. DAST API vulnerabilities are available in a section labeled @@ -2360,7 +2360,7 @@ See the following documentation sections for assistance: See [Performance Tuning and Testing Speed](#performance-tuning-and-testing-speed) -### Error waiting for DAST API 'http://127.0.0.1:5000' to become available +### Error: `Error waiting for DAST API 'http://127.0.0.1:5000' to become available` A bug exists in versions of the DAST API analyzer prior to v1.6.196 that can cause a background process to fail under certain conditions. The solution is to update to a newer version of the DAST API analyzer. diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index afed5b3b0ca..d41c0eff860 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -7,16 +7,19 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Dependency list **(ULTIMATE)** -> - Application dependencies [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10075) in GitLab 12.0. > - System dependencies [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6698) in GitLab 14.6. +> - Group-level dependency list [introduced](https://gitlab.com/groups/gitlab-org/-/epics/8090) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `group_level_dependencies`. Disabled by default. -Use the dependency list to review your project's dependencies and key +FLAG: +On self-managed GitLab, by default the group-level dependency list is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `group_level_dependencies`. On GitLab.com, this feature is not available. + +Use the dependency list to review your project or group's dependencies and key details about those dependencies, including their known vulnerabilities. It is a collection of dependencies in your project, including existing and new findings. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Project Dependency](https://www.youtube.com/watch?v=ckqkn9Tnbw4). -To see the dependency list, go to your project and select **Security and Compliance > Dependency list**. +To see the dependency list, go to your project and select **Secure > Dependency list**. This information is sometimes referred to as a Software Bill of Materials, SBOM, or BOM. @@ -89,4 +92,4 @@ You can download your project's list of dependencies and their details in JSON f ### Using the API -You can download your project's list of dependencies [using the API](../../../api/dependencies.md#list-project-dependencies). Note this only provides the dependencies identified by the Gemnasium family of analyzers and [not any other of the GitLab dependency analyzers](../dependency_scanning/analyzers.md). +You can download your project's list of dependencies [using the API](../../../api/dependencies.md#list-project-dependencies). Note this only provides the dependencies identified by the [Gemnasium family of analyzers](../dependency_scanning/index.md#dependency-analyzers) and not any other of the GitLab dependency analyzers. diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 4feac0cb5e6..0d7d495ba49 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -1,124 +1,11 @@ --- -type: reference, howto -stage: Secure -group: Composition Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2023-09-22' --- -# Dependency Scanning Analyzers **(ULTIMATE)** +This document was moved to [another location](index.md). -Dependency Scanning supports the following official analyzers: - -- [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) -- [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) -- [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python) - -The analyzers are published as Docker images, which Dependency Scanning uses -to launch dedicated containers for each analysis. - -Dependency Scanning is pre-configured with a set of **default images** that are -maintained by GitLab, but users can also integrate their own **custom images**. - -## Official default analyzers - -Any custom change to the official analyzers can be achieved by using a -[CI/CD variable in your `.gitlab-ci.yml`](index.md#customizing-the-dependency-scanning-settings). - -### Using a custom Docker mirror - -You can switch to a custom Docker registry that provides the official analyzer -images under a different prefix. For instance, the following instructs Dependency -Scanning to pull `my-docker-registry/gl-images/gemnasium` -instead of `registry.gitlab.com/security-products/gemnasium`. -In `.gitlab-ci.yml` define: - -```yaml -include: - template: Security/Dependency-Scanning.gitlab-ci.yml - -variables: - SECURE_ANALYZERS_PREFIX: my-docker-registry/gl-images -``` - -This configuration requires that your custom registry provides images for all -the official analyzers. - -### Disable specific analyzers - -You can select the official analyzers you don't want to run. Here's how to disable -the `gemnasium` analyzer. -In `.gitlab-ci.yml` define: - -```yaml -include: - template: Security/Dependency-Scanning.gitlab-ci.yml - -variables: - DS_EXCLUDED_ANALYZERS: "gemnasium" -``` - -### Disabling default analyzers - -Setting `DS_EXCLUDED_ANALYZERS` to a list of the official analyzers disables them. -In `.gitlab-ci.yml` define: - -```yaml -include: - template: Security/Dependency-Scanning.gitlab-ci.yml - -variables: - DS_EXCLUDED_ANALYZERS: "gemnasium, gemnasium-maven, gemnasium-python" -``` - -This is used when one totally relies on [custom analyzers](#custom-analyzers). - -## Custom analyzers - -You can provide your own analyzers by -defining CI jobs in your CI configuration. For consistency, you should suffix your custom Dependency -Scanning jobs with `-dependency_scanning`. Here's how to add a scanning job that's based on the -Docker image `my-docker-registry/analyzers/nuget` and generates a Dependency Scanning report -`gl-dependency-scanning-report.json` when `/analyzer run` is executed. Define the following in -`.gitlab-ci.yml`: - -```yaml -nuget-dependency_scanning: - image: - name: "my-docker-registry/analyzers/nuget" - script: - - /analyzer run - artifacts: - reports: - dependency_scanning: gl-dependency-scanning-report.json -``` - -The [Security Scanner Integration](../../../development/integrations/secure.md) documentation explains how to integrate custom security scanners into GitLab. - -## Analyzers data - -The following table lists the data available for the Gemnasium analyzer. - -| Property \ Tool | Gemnasium | -|---------------------------------------|:------------------:| -| Severity | ✓ | -| Title | ✓ | -| File | ✓ | -| Start line | 𐄂 | -| End line | 𐄂 | -| External ID (for example, CVE) | ✓ | -| URLs | ✓ | -| Internal doc/explanation | ✓ | -| Solution | ✓ | -| Confidence | 𐄂 | -| Affected item (for example, class or package) | ✓ | -| Source code extract | 𐄂 | -| Internal ID | ✓ | -| Date | ✓ | -| Credits | ✓ | - -- ✓ => we have that data -- ⚠ => we have that data, but it's partially reliable, or we need to extract that data from unstructured content -- 𐄂 => we don't have that data, or it would need to develop specific or inefficient/unreliable logic to obtain it. - -The values provided by these tools are heterogeneous, so they are sometimes -normalized into common values (for example, `severity`, `confidence`, etc). +<!-- This redirect file can be deleted after <2023-09-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index df9e1d72dba..15fed4f2adc 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -1,5 +1,4 @@ --- -type: reference, howto stage: Secure group: Composition Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments @@ -17,6 +16,11 @@ aspects of inspecting the items your code uses. These items typically include ap dependencies that are almost always imported from external sources, rather than sourced from items you wrote yourself. +GitLab offers both Dependency Scanning and [Container Scanning](../container_scanning/index.md) to +ensure coverage for all of these dependency types. To cover as much of your risk area as possible, +we encourage you to use all of our security scanners. For a comparison of these features, see +[Dependency Scanning compared to Container Scanning](../comparison_dependency_and_container_scanning.md). + If you're using [GitLab CI/CD](../../../ci/index.md), you can use dependency scanning to analyze your dependencies for known vulnerabilities. GitLab scans all dependencies, including transitive dependencies (also known as nested dependencies). You can take advantage of dependency scanning by @@ -38,48 +42,6 @@ vulnerability. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Dependency Scanning](https://www.youtube.com/watch?v=TBnfbGk4c4o). -## Dependency Scanning compared to Container Scanning - -GitLab offers both Dependency Scanning and Container Scanning -to ensure coverage for all of these dependency types. To cover as much of your risk area as -possible, we encourage you to use all of our security scanning tools: - -- Dependency Scanning analyzes your project and tells you which software dependencies, - including upstream dependencies, have been included in your project, and what known - risks the dependencies contain. Dependency Scanning modifies its behavior based - on the language and package manager of the project. It typically looks for a lock file - then performs a build to fetch upstream dependency information. In the case of - containers, Dependency Scanning uses the compatible manifest and reports only these - declared software dependencies (and those installed as a sub-dependency). - Dependency Scanning cannot detect software dependencies that are pre-bundled - into the container's base image. To identify pre-bundled dependencies, enable - [Container Scanning](../container_scanning/index.md) language scanning using the - [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). -- [Container Scanning](../container_scanning/index.md) analyzes your containers and tells - you about known risks in the operating system's (OS) packages. You can configure it - to also report on software and language dependencies, if you enable it and use - the [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). - Turning this variable on can result in some duplicate findings, as we do not yet - de-duplicate results between Container Scanning and Dependency Scanning. For more details, - efforts to de-duplicate these findings can be tracked in - [this epic](https://gitlab.com/groups/gitlab-org/-/epics/8026). - -The following table summarizes which types of dependencies each scanning tool can detect: - -| Feature | Dependency Scanning | Container Scanning | -| ----------------------------------------------------------- | ------------------- | ------------------- | -| Identify the manifest, lock file, or static file that introduced the dependency | **{check-circle}** | **{dotted-circle}** | -| Development dependencies | **{check-circle}** | **{dotted-circle}** | -| Dependencies in a lock file committed to your repository | **{check-circle}** | **{check-circle}** <sup>1</sup> | -| Binaries built by Go | **{dotted-circle}** | **{check-circle}** <sup>2</sup> <sup>3</sup> | -| Dynamically-linked language-specific dependencies installed by the Operating System | **{dotted-circle}** | **{check-circle}** <sup>3</sup> | -| Operating system dependencies | **{dotted-circle}** | **{check-circle}** | -| Language-specific dependencies installed on the operating system (not built by your project) | **{dotted-circle}** | **{check-circle}** | - -1. Lock file must be present in the image to be detected. -1. Binary file must be present in the image to be detected. -1. Only when using Trivy - ## Requirements Dependency Scanning runs in the `test` stage, which is available by default. If you redefine the @@ -101,21 +63,6 @@ If you need it, explain why by filling out [the survey](https://docs.google.com/ ## Supported languages and package managers -Dependency Scanning automatically detects the languages used in the repository. All analyzers -matching the detected languages are run. There is usually no need to customize the selection of -analyzers. We recommend not specifying the analyzers so you automatically use the full selection -for best coverage, avoiding the need to make adjustments when there are deprecations or removals. -However, you can override the selection using the variable `DS_EXCLUDED_ANALYZERS`. - -The language detection relies on CI job [`rules`](../../../ci/yaml/index.md#rules) and searches a -maximum of two directory levels from the repository's root. For example, the -`gemnasium-dependency_scanning` job is enabled if a repository contains either `Gemfile`, -`api/Gemfile`, or `api/client/Gemfile`, but not if the only supported dependency file is `api/v1/client/Gemfile`. - -For Java and Python, when a supported dependency file is detected, Dependency Scanning attempts to build the project and execute some Java or Python commands to get the list of dependencies. For all other projects, the lock file is parsed to obtain the list of dependencies without needing to build the project first. - -When a supported dependency file is detected, all dependencies, including transitive dependencies are analyzed. There is no limit to the depth of nested or transitive dependencies that are analyzed. - The following languages and dependency managers are supported: <style> @@ -341,6 +288,40 @@ table.supported-languages ul { </ol> <!-- markdownlint-enable MD044 --> +## Dependency detection + +Dependency Scanning automatically detects the languages used in the repository. All analyzers +matching the detected languages are run. There is usually no need to customize the selection of +analyzers. We recommend not specifying the analyzers so you automatically use the full selection for +best coverage, avoiding the need to make adjustments when there are deprecations or removals. +However, you can override the selection using the variable `DS_EXCLUDED_ANALYZERS`. + +The language detection relies on CI job [`rules`](../../../ci/yaml/index.md#rules) and searches a +maximum of two directory levels from the repository's root. For example, the +`gemnasium-dependency_scanning` job is enabled if a repository contains either `Gemfile`, +`api/Gemfile`, or `api/client/Gemfile`, but not if the only supported dependency file is +`api/v1/client/Gemfile`. + +For Java and Python, when a supported dependency file is detected, Dependency Scanning attempts to +build the project and execute some Java or Python commands to get the list of dependencies. For all +other projects, the lock file is parsed to obtain the list of dependencies without needing to build +the project first. + +When a supported dependency file is detected, all dependencies, including transitive dependencies +are analyzed. There is no limit to the depth of nested or transitive dependencies that are analyzed. + +### Dependency analyzers + +Dependency Scanning supports the following official analyzers: + +- [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) +- [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) +- [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python) + +The analyzers are published as Docker images, which Dependency Scanning uses +to launch dedicated containers for each analysis. You can also integrate a custom +[security scanner](../../../development/integrations/secure.md). + ### How analyzers obtain dependency information GitLab analyzers obtain dependency information using one of the following two methods: @@ -358,7 +339,7 @@ The following package managers use lockfiles that GitLab analyzers are capable o | Composer | Not applicable | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/php-composer/default/composer.lock) | | Conan | 0.4 | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/c-conan/default/conan.lock#L38) | | Go | Not applicable | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/go-modules/gosum/default/go.sum) <sup><strong><a href="#notes-regarding-parsing-lockfiles-1">1</a></strong></sup> | -| NuGet | v1 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | +| NuGet | v1, v2 | [4.9](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/csharp-nuget-dotnetcore/default/src/web.api/packages.lock.json#L2) | | npm | v1, v2, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-2">2</a></b></sup> | [6.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/default/package-lock.json#L4), [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-npm/lockfileVersion2/package-lock.json#L4), [9.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/npm/fixtures/lockfile-v3/simple/package-lock.json#L4) | | pnpm | v5.3, v5.4, v6 | [7.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-pnpm/default/pnpm-lock.yaml#L1), [8.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/scanner/parser/pnpm/fixtures/v6/simple/pnpm-lock.yaml#L1) | | yarn | v1, v2<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup>, v3<sup><b><a href="#notes-regarding-parsing-lockfiles-3">3</a></b></sup> | [1.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/classic/default/yarn.lock#L2), [2.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v2/default/yarn.lock), [3.x](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/master/qa/fixtures/js-yarn/berry/v3/default/yarn.lock) | @@ -649,12 +630,11 @@ The following variables allow configuration of global dependency scanning settin | CI/CD variables | Description | | ----------------------------|------------ | | `ADDITIONAL_CA_CERT_BUNDLE` | Bundle of CA certs to trust. The bundle of certificates provided here is also used by other tools during the scanning process, such as `git`, `yarn`, or `npm`. See [Using a custom SSL CA certificate authority](#using-a-custom-ssl-ca-certificate-authority) for more details. | -| `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](analyzers.md). | +| `DS_EXCLUDED_ANALYZERS` | Specify the analyzers (by name) to exclude from Dependency Scanning. For more information, see [Dependency Scanning Analyzers](#dependency-analyzers). | | `DS_EXCLUDED_PATHS` | Exclude files and directories from the scan based on the paths. A comma-separated list of patterns. Patterns can be globs (see [`doublestar.Match`](https://pkg.go.dev/github.com/bmatcuk/doublestar/v4@v4.0.2#Match) for supported patterns), or file or folder paths (for example, `doc,spec`). Parent directories also match patterns. Default: `"spec, test, tests, tmp"`. | | `DS_IMAGE_SUFFIX` | Suffix added to the image name. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354796) in GitLab 14.10.) Automatically set to `"-fips"` when FIPS mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357922) in GitLab 15.0.) | | `DS_MAX_DEPTH` | Defines how many directory levels deep that the analyzer should search for supported files to scan. A value of `-1` scans all directories regardless of depth. Default: `2`. | -| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | -| `SECURE_LOG_LEVEL` | Set the minimum logging level. Messages of this logging level or higher are output. From highest to lowest severity, the logging levels are: `fatal`, `error`, `warn`, `info`, `debug`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. Default: `info`. | +| `SECURE_ANALYZERS_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). | #### Configuring specific analyzers used by dependency scanning @@ -1044,24 +1024,6 @@ See explanations of the variables above in the [configuration section](#configur See the following sections for configuring specific languages and package managers. -#### JavaScript (npm and yarn) projects - -Add the following to the variables section of `.gitlab-ci.yml`: - -```yaml -RETIREJS_JS_ADVISORY_DB: "example.com/jsrepository.json" -RETIREJS_NODE_ADVISORY_DB: "example.com/npmrepository.json" -``` - -#### Ruby (gem) projects - -Add the following to the variables section of `.gitlab-ci.yml`: - -```yaml -BUNDLER_AUDIT_ADVISORY_DB_REF_NAME: "master" -BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git" -``` - #### Python (pip) If you need to install Python packages before the analyzer runs, you should use `pip install --user` in the `before_script` of the scanning job. The `--user` flag causes project dependencies to be installed in the user directory. If you do not pass the `--user` option, packages are installed globally, and they are not scanned and don't show up when listing project dependencies. @@ -1173,12 +1135,10 @@ version number). ## Troubleshooting -### Increase log verbosity +### Debug-level logging -When a [job log](../../../ci/jobs/index.md#expand-and-collapse-job-log-sections) -doesn't contain enough information about a dependency-scanning failure, -[set `SECURE_LOG_LEVEL` to `debug`](#configuring-dependency-scanning) -and check the resulting, more verbose log. +Debug-level logging can help when troubleshooting. For details, see +[debug-level logging](../index.md#debug-level-logging). ### Working around missing support for certain languages or package managers diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 8e2f54fed44..7213fa2ba18 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -250,6 +250,7 @@ kics-iac-sast: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. > - Enabled by default in 15.10. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/375128) in GitLab 16.2. To help you focus on the vulnerabilities that are still relevant, GitLab IaC Scanning automatically [resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: @@ -270,15 +271,10 @@ pipelines tab on merge requests by [setting `artifacts: paths`](../../../ci/yaml ## Troubleshooting -### IaC debug logging +### Debug-level logging -To help troubleshoot IaC jobs, you can increase the [Secure scanner log verbosity](../sast/index.md#logging-level) -by using a global CI/CD variable set to `debug`: - -```yaml -variables: - SECURE_LOG_LEVEL: "debug" -``` +Debug-level logging can help when troubleshooting. For details, see +[debug-level logging](../index.md#debug-level-logging). ### IaC Scanning findings show as `No longer detected` unexpectedly diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 8bbe4db62a9..56a79191833 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -82,7 +82,7 @@ Dependency analysis can run: image is built - [Container Scanning](container_scanning/index.md). For more details, see -[Dependency Scanning compared to Container Scanning](dependency_scanning/index.md#dependency-scanning-compared-to-container-scanning). +[Dependency Scanning compared to Container Scanning](comparison_dependency_and_container_scanning.md). Additionally, dependencies in operational container images can be analyzed for vulnerabilities on a regular schedule or cadence. For more details, see [Operational Container Scanning](../../user/clusters/agent/vulnerabilities.md). @@ -239,6 +239,16 @@ reports are available to download. To download a report, select  +Security scans produce at least one of these [CI `artifacts:reports` types](../../ci/yaml/artifacts_reports.md): + +- `artifacts:reports:api_fuzzing` +- `artifacts:reports:container_scanning` +- `artifacts:reports:coverage_fuzzing` +- `artifacts:reports:dast` +- `artifacts:reports:dependency_scanning` +- `artifacts:reports:sast` +- `artifacts:reports:secret_detection` + ### Ultimate A merge request contains a security widget which displays a summary of the _new_ results. New results are determined by comparing the findings of the merge request against the findings of the most recent completed pipeline (`success`, `failed`, `canceled` or `skipped`) for the commit when the feature branch was created from the target branch. @@ -247,7 +257,10 @@ If security scans have not run for the completed pipeline in the target branch w The merge request security widget displays only a subset of the vulnerabilities in the generated JSON artifact because it contains both new and existing findings. -From the merge request security widget, select **Expand** to unfold the widget, displaying any new and no longer detected (removed) findings by scan type. Select **View full report** to go directly to the **Security** tab in the latest branch pipeline. +From the merge request security widget, select **Expand** to unfold the widget, displaying any new and no longer detected (removed) findings by scan type. + +For each security report type, the widget displays the first 25 added and 25 fixed findings, sorted by severity. To see all +findings, select **View full report** to go directly to the **Security** tab in the latest branch pipeline.  @@ -523,24 +536,48 @@ Feedback is welcome on our vision for [unifying the user experience for these tw ## Troubleshooting -<!-- NOTE: The below subsection(`### Secure job failing with exit code 1`) documentation URL is referred in the [/gitlab-org/security-products/analyzers/command](https://gitlab.com/gitlab-org/security-products/analyzers/command/-/blob/main/command.go#L19) repository. If this section/subsection changes, please ensure to update the corresponding URL in the mentioned repository. ---> +### Logging level -### Secure job failing with exit code 1 +The verbosity of logs output by GitLab analyzers is determined by the `SECURE_LOG_LEVEL` environment +variable. Messages of this logging level or higher are output. + +From highest to lowest severity, the logging levels are: + +- `fatal` +- `error` +- `warn` +- `info` (default) +- `debug` + +#### Debug-level logging WARNING: Debug logging can be a serious security risk. The output may contain the content of environment variables and other secrets available to the job. The output is uploaded -to the GitLab server and visible in job logs. +to the GitLab server and is visible in job logs. -If a Secure job is failing and it's unclear why, add `SECURE_LOG_LEVEL: "debug"` as a global CI/CD variable for -more verbose output that is helpful for troubleshooting. +To enable debug-level logging, add the following to your `.gitlab-ci.yml` file: ```yaml variables: SECURE_LOG_LEVEL: "debug" ``` +This indicates to all GitLab analyzers that they are to output **all** messages. For more details, +see [logging level](#logging-level). + +<!-- NOTE: The below subsection(`### Secure job failing with exit code 1`) documentation URL is referred in the [/gitlab-org/security-products/analyzers/command](https://gitlab.com/gitlab-org/security-products/analyzers/command/-/blob/main/command.go#L19) repository. If this section/subsection changes, please ensure to update the corresponding URL in the mentioned repository. +--> + +### Secure job failing with exit code 1 + +If a Secure job is failing and it's unclear why: + +1. Enable [debug-level logging](#debug-level-logging). +1. Run the job. +1. Examine the job's output. +1. Set the logging level to `info` (default). + ### Outdated security reports When a security report generated for a merge request becomes outdated, the merge request shows a diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 1ee4d9b2a19..63f3763cab9 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -7,6 +7,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Offline environments **(ULTIMATE SELF)** +NOTE: +To set up an offline environment, you must receive an [opt-out exemption of cloud licensing](https://about.gitlab.com/pricing/licensing-faq/cloud-licensing/#offline-cloud-licensing) prior to purchase. For more details, contact your GitLab sales representative. + It's possible to run most of the GitLab security scanners when not connected to the internet. This document describes how to operate Secure Categories (that is, scanner types) in an offline diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index bd3da0b5913..4e610d64ec9 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Policies in GitLab provide security teams a way to require scans of their choice to be run whenever a project pipeline runs according to the configuration specified. Security teams can therefore be confident that the scans they set up have not been changed, altered, or disabled. You -can access these by navigating to your project's **Security and Compliance > Policies** page. +can access these by navigating to your project's **Secure > Policies** page. GitLab supports the following security policies: @@ -141,7 +141,6 @@ The workaround is to amend your group or instance push rules to allow branches f ### Troubleshooting common issues configuring security policies -- Confirm that projects contain a `.gitlab-ci.yml` file. This file is required for scan execution policies. - Confirm that scanners are properly configured and producing results for the latest branch. Security Policies are designed to require approval when there are no results (no security report), as this ensures that no vulnerabilities are introduced. We cannot know if there are any vulnerabilities unless the scans enforced by the policy complete successfully and are evaluated. - When running scan execution policies based on a SAST action, ensure target repositories contain proper code files. SAST runs different analyzers [based on the types of files in the repo](../sast/index.md#supported-languages-and-frameworks), and if no supported files are found it will not run any jobs. See the [SAST CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml) for more details. - Check for any branch configuration conflicts. If your policy is configured to enforce rules on `main` but some projects within the scope are using `master` as their default branch, the policy is not applied for the latter. Support for specifying the `default` branch in your policies is proposed in [epic 9468](https://gitlab.com/groups/gitlab-org/-/epics/9468). diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index e18d4d1787e..b84d4d2e49e 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -6,16 +6,32 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Scan execution policies **(ULTIMATE)** -> - Group-level security policies were [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4425) in GitLab 15.2. -> - Group-level security policies were [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/356258) in GitLab 15.4. +> - Group-level security policies [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4425) in GitLab 15.2. +> - Group-level security policies [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/356258) in GitLab 15.4. > - Operational container scanning [introduced](https://gitlab.com/groups/gitlab-org/-/epics/3410) in GitLab 15.5 - -Group, subgroup, or project owners can use scan execution policies to require that security scans run on a specified -schedule or with the project pipeline. The security scan runs with multiple project pipelines if you define the policy -at a group or subgroup level. GitLab injects the required scans into the CI pipeline as new jobs. In the event -of a job name collision, GitLab adds a dash and a number to the job name. GitLab increments the number until the name -no longer conflicts with existing job names. If you create a policy at the group level, it applies to every child project -or subgroup. You cannot edit a group-level policy from a child project or subgroup. +> - Support for custom CI variables in the Scan Execution Policies editor [introduced](https://gitlab.com/groups/gitlab-org/-/epics/9566) in GitLab 16.2. +> - Enforcement of scan execution policies on projects with an existing GitLab CI/CD configuration [introduced](https://gitlab.com/groups/gitlab-org/-/epics/6880) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `scan_execution_policy_pipelines`. Enabled by default. + +FLAG: +On self-managed GitLab, this feature is enabled by default. To disable it, ask an +administrator to [disable the feature flag](../../../administration/feature_flags.md) named +`scan_execution_policy_pipelines`. On GitLab.com, this feature is enabled. + +Group, subgroup, or project owners can use scan execution policies to require that security scans +run on a specified schedule or with the project pipeline. The security scan runs with multiple +project pipelines if you define the policy at a group or subgroup level. GitLab injects the required +scans into the CI/CD pipeline as new jobs. + +Scan execution policies are enforced for all applicable projects, even those without a GitLab +CI/CD configuration file or where AutoDevOps is disabled. Security policies create the file +implicitly so that the policies can be enforced. This ensures policies enabling execution of +secret detection, static analysis, or other scanners that do not require a build in the +project, are still able to execute and be enforced. + +In the event of a job name collision, GitLab appends a hyphen and a number to the job name. GitLab +increments the number until the name no longer conflicts with existing job names. If you create a +policy at the group level, it applies to every child project or subgroup. You cannot edit a +group-level policy from a child project or subgroup. This feature has some overlap with [compliance framework pipelines](../../group/compliance_frameworks.md#compliance-pipelines), as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). @@ -46,6 +62,13 @@ before the policy changes take effect.  +NOTE: +Selection of site and scanner profiles using the rule mode editor for DAST execution policies differs based on +whether the policy is being created at the project or group level. For project-level policies the rule mode editor +presents a list of profiles to choose from that are already defined in the project. For group-level policies +you are required to type in the names of the profiles to use, and to prevent pipeline errors, profiles with +matching names must exist in all of the group's projects. + ## Scan execution policies schema The YAML file with scan execution policies consists of an array of objects matching scan execution @@ -57,41 +80,52 @@ When you save a new policy, GitLab validates its contents against [this JSON sch If you're not familiar with how to read [JSON schemas](https://json-schema.org/), the following sections and tables provide an alternative. -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `scan_execution_policy` | `array` of scan execution policy | | List of scan execution policies (maximum 5) | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `scan_execution_policy` | `array` of scan execution policy | true | | List of scan execution policies (maximum 5) | ## Scan execution policy schema -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `name` | `string` | | Name of the policy. Maximum of 255 characters.| -| `description` (optional) | `string` | | Description of the policy. | -| `enabled` | `boolean` | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | -| `rules` | `array` of rules | | List of rules that the policy applies. | -| `actions` | `array` of actions | | List of actions that the policy enforces. | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `name` | `string` | true | | Name of the policy. Maximum of 255 characters.| +| `description` (optional) | `string` | true | | Description of the policy. | +| `enabled` | `boolean` | true | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | +| `rules` | `array` of rules | true | | List of rules that the policy applies. | +| `actions` | `array` of actions | true | | List of actions that the policy enforces. | ## `pipeline` rule type +> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Disabled by default. +> - The `branch_type` field was [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/413062) in GitLab 16.2. + This rule enforces the defined actions whenever the pipeline runs for a selected branch. -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `type` | `string` | `pipeline` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). Cannot be used with the `branch_type` field. | -| `branch_type` | `string` | `default`, `protected` or `all` | The types of branches the given policy applies to. Cannot be used with the `branches` field. | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `type` | `string` | true | `pipeline` | The rule's type. | +| `branches` <sup>1</sup> | `array` of `string` | true if `branch_type` field does not exist | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `branch_type` <sup>1</sup> | `string` | true if `branches` field does not exist | `default`, `protected` or `all` | The types of branches the given policy applies to. | + +1. You must specify only one of `branches` or `branch_type`. ## `schedule` rule type +> - The `branch_type` field was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/404774) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `security_policies_branch_type`. Disabled by default. +> - The `branch_type` field was [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/413062) in GitLab 16.2. + This rule enforces the defined actions and schedules a scan on the provided date/time. -| Field | Type | Possible values | Description | -|------------|------|-----------------|-------------| -| `type` | `string` | `schedule` | The rule's type. | -| `branches` | `array` of `string` | `*` or the branch's name | The branch the given policy applies to (supports wildcard). This field is required if the `agents` field is not set. Cannot be used with the `branch_type` field. | -| `branch_type` | `string` | `default`, `protected` or `all` | The types of branches the given policy applies to. Cannot be used with the `branches` field. | -| `cadence` | `string` | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. Minimum of 15 minute intervals when used together with the `branches` field. | -| `agents` | `object` | | The name of the [GitLab agents](../../clusters/agent/index.md) where [Operational Container Scanning](../../clusters/agent/vulnerabilities.md) runs. The object key is the name of the Kubernetes agent configured for your project in GitLab. This field is required if the `branches` field is not set. | +| Field | Type | Required | Possible values | Description | +|------------|------|----------|-----------------|-------------| +| `type` | `string` | true | `schedule` | The rule's type. | +| `branches` <sup>1</sup> | `array` of `string` | true if either `branch_type` or `agents` fields does not exist | `*` or the branch's name | The branch the given policy applies to (supports wildcard). | +| `branch_type` <sup>1</sup> | `string` | true if either `branches` or `agents` fields does not exist | `default`, `protected` or `all` | The types of branches the given policy applies to. | +| `cadence` | `string` | true | CRON expression (for example, `0 0 * * *`) | A whitespace-separated string containing five fields that represents the scheduled time. Minimum of 15 minute intervals when used together with the `branches` field. | +| `timezone` | `string` | false | Time zone identifier (for example, `America/New_York`) | Time zone to apply to the cadence. Value must be an IANA Time Zone Database identifier. | +| `agents` <sup>1</sup> | `object` | true if either `branch_type` or `branches` fields do not exists | | The name of the [GitLab agents](../../clusters/agent/index.md) where [Operational Container Scanning](../../clusters/agent/vulnerabilities.md) runs. The object key is the name of the Kubernetes agent configured for your project in GitLab. | + +1. You must specify only one of `branches`, `branch_type`, or `agents`. GitLab supports the following types of CRON syntax for the `cadence` field: @@ -118,9 +152,9 @@ When using the `schedule` rule type in conjunction with the `branches` field, no Use this schema to define `agents` objects in the [`schedule` rule type](#schedule-rule-type). -| Field | Type | Possible values | Description | -|--------------|---------------------|--------------------------|-------------| -| `namespaces` | `array` of `string` | | The namespace that is scanned. If empty, all namespaces are scanned. | +| Field | Type | Required | Possible values | Description | +|--------------|---------------------|----------|--------------------------|-------------| +| `namespaces` | `array` of `string` | true | The namespace that is scanned. If empty, all namespaces are scanned. | #### Policy example diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index d8cd984ad40..0aac36988d4 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -61,32 +61,33 @@ When you save a new policy, GitLab validates its contents against [this JSON sch If you're not familiar with how to read [JSON schemas](https://json-schema.org/), the following sections and tables provide an alternative. -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `scan_result_policy` | `array` of Scan Result Policy | | List of scan result policies (maximum 5). | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `scan_result_policy` | `array` of Scan Result Policy | true | | List of scan result policies (maximum 5). | ## Scan result policy schema -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `name` | `string` | | Name of the policy. Maximum of 255 characters.| -| `description` (optional) | `string` | | Description of the policy. | -| `enabled` | `boolean` | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | -| `rules` | `array` of rules | | List of rules that the policy applies. | -| `actions` | `array` of actions | | List of actions that the policy enforces. | +| Field | Type | Required |Possible values | Description | +|-------|------|----------|----------------|-------------| +| `name` | `string` | true | | Name of the policy. Maximum of 255 characters.| +| `description` (optional) | `string` | true | | Description of the policy. | +| `enabled` | `boolean` | true | `true`, `false` | Flag to enable (`true`) or disable (`false`) the policy. | +| `rules` | `array` of rules | true | | List of rules that the policy applies. | +| `actions` | `array` of actions | true | | List of actions that the policy enforces. | ## `scan_finding` rule type This rule enforces the defined actions based on security scan findings. -| Field | Type | Possible values | Description | -|------------|------|--------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `type` | `string` | `scan_finding` | The rule's type. | -| `branches` | `array` of `string` | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. | -| `scanners` | `array` of `string` | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. `sast` includes results from both SAST and SAST IaC scanners. | -| `vulnerabilities_allowed` | `integer` | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | -| `severity_levels` | `array` of `string` | `info`, `unknown`, `low`, `medium`, `high`, `critical` | The severity levels for this rule to consider. | -| `vulnerability_states` | `array` of `string` | `newly_detected`, `detected`, `confirmed`, `resolved`, `dismissed`, `new_needs_triage`, `new_dismissed` | All vulnerabilities fall into two categories:<br><br>**Newly Detected Vulnerabilities** - the `newly_detected` policy option covers vulnerabilities identified in the merge request branch itself but that do not currently exist on the default branch. This policy option requires a pipeline to complete before the rule is evaluated so that it knows whether vulnerabilities are newly detected or not. Merge requests are blocked until the pipeline and necessary security scans are complete. The `newly_detected` option considers both of the following statuses:<br><br> • Detected<br> • Dismissed<br><br> The `new_needs_triage` option considers the status<br><br> • Detected<br><br> The `new_dismissed` option considers the status<br><br> • Dismissed<br><br>**Pre-Existing Vulnerabilities** - these policy options are evaluated immediately and do not require a pipeline complete as they consider only vulnerabilities previously detected in the default branch.<br><br> • `Detected` - the policy looks for vulnerabilities in the detected state.<br> • `Confirmed` - the policy looks for vulnerabilities in the confirmed state.<br> • `Dismissed` - the policy looks for vulnerabilities in the dismissed state.<br> • `Resolved` - the policy looks for vulnerabilities in the resolved state. | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `type` | `string` | true | `scan_finding` | The rule's type. | +| `branches` | `array` of `string` | true if `branch_type` field does not exist | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. Cannot be used with the `branch_type` field. | +| `branch_type` | `string` | true if `branches` field does not exist | `default` or `protected` | The types of branches the given policy applies to. Cannot be used with the `branches` field. | +| `scanners` | `array` of `string` | true | `sast`, `secret_detection`, `dependency_scanning`, `container_scanning`, `dast`, `coverage_fuzzing`, `api_fuzzing` | The security scanners for this rule to consider. `sast` includes results from both SAST and SAST IaC scanners. | +| `vulnerabilities_allowed` | `integer` | true | Greater than or equal to zero | Number of vulnerabilities allowed before this rule is considered. | +| `severity_levels` | `array` of `string` | true | `info`, `unknown`, `low`, `medium`, `high`, `critical` | The severity levels for this rule to consider. | +| `vulnerability_states` | `array` of `string` | true | `newly_detected`, `detected`, `confirmed`, `resolved`, `dismissed`, `new_needs_triage`, `new_dismissed` | All vulnerabilities fall into two categories:<br><br>**Newly Detected Vulnerabilities** - the `newly_detected` policy option covers vulnerabilities identified in the merge request branch itself but that do not currently exist on the default branch. This policy option requires a pipeline to complete before the rule is evaluated so that it knows whether vulnerabilities are newly detected or not. Merge requests are blocked until the pipeline and necessary security scans are complete. The `newly_detected` option considers both of the following statuses:<br><br> • Detected<br> • Dismissed<br><br> The `new_needs_triage` option considers the status<br><br> • Detected<br><br> The `new_dismissed` option considers the status<br><br> • Dismissed<br><br>**Pre-Existing Vulnerabilities** - these policy options are evaluated immediately and do not require a pipeline complete as they consider only vulnerabilities previously detected in the default branch.<br><br> • `Detected` - the policy looks for vulnerabilities in the detected state.<br> • `Confirmed` - the policy looks for vulnerabilities in the confirmed state.<br> • `Dismissed` - the policy looks for vulnerabilities in the dismissed state.<br> • `Resolved` - the policy looks for vulnerabilities in the resolved state. | ## `license_finding` rule type @@ -95,28 +96,29 @@ This rule enforces the defined actions based on security scan findings. This rule enforces the defined actions based on license findings. -| Field | Type | Possible values | Description | -|------------|------|-----------------|-------------| -| `type` | `string` | `license_finding` | The rule's type. | -| `branches` | `array` of `string` | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. | -| `match_on_inclusion` | `boolean` | `true`, `false` | Whether the rule matches inclusion or exclusion of licenses listed in `license_types`. | -| `license_types` | `array` of `string` | license types | License types to match on, for example `BSD` or `MIT`. | -| `license_states` | `array` of `string` | `newly_detected`, `detected` | Whether to match newly detected and/or previously detected licenses. The `newly_detected` state triggers approval when either a new package is introduced or when a new license for an existing package is detected. | +| Field | Type | Required | Possible values | Description | +|------------|------|----------|-----------------|-------------| +| `type` | `string` | true | `license_finding` | The rule's type. | +| `branches` | `array` of `string` | true if `branch_type` field does not exist | `[]` or the branch's name | Applicable only to protected target branches. An empty array, `[]`, applies the rule to all protected target branches. Cannot be used with the `branch_type` field. | +| `branch_type` | `string` | true if `branches` field does not exist | `default` or `protected` | The types of branches the given policy applies to. Cannot be used with the `branches` field. | +| `match_on_inclusion` | `boolean` | true | `true`, `false` | Whether the rule matches inclusion or exclusion of licenses listed in `license_types`. | +| `license_types` | `array` of `string` | true | license types | [SPDX license names](https://spdx.org/licenses) to match on, for example `Affero General Public License v1.0` or `MIT License`. | +| `license_states` | `array` of `string` | true | `newly_detected`, `detected` | Whether to match newly detected and/or previously detected licenses. The `newly_detected` state triggers approval when either a new package is introduced or when a new license for an existing package is detected. | ## `require_approval` action type This action sets an approval rule to be required when conditions are met for at least one rule in the defined policy. -| Field | Type | Possible values | Description | -|-------|------|-----------------|-------------| -| `type` | `string` | `require_approval` | The action's type. | -| `approvals_required` | `integer` | Greater than or equal to zero | The number of MR approvals required. | -| `user_approvers` | `array` of `string` | Username of one of more users | The users to consider as approvers. Users must have access to the project to be eligible to approve. | -| `user_approvers_ids` | `array` of `integer` | ID of one of more users | The IDs of users to consider as approvers. Users must have access to the project to be eligible to approve. | -| `group_approvers` | `array` of `string` | Path of one of more groups | The groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | -| `group_approvers_ids` | `array` of `integer` | ID of one of more groups | The IDs of groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | -| `role_approvers` | `array` of `string` | One or more [roles](../../../user/permissions.md#roles) (for example: `owner`, `maintainer`) | The roles to consider as approvers that are eligible to approve. | +| Field | Type | Required | Possible values | Description | +|-------|------|----------|-----------------|-------------| +| `type` | `string` | true | `require_approval` | The action's type. | +| `approvals_required` | `integer` | true | Greater than or equal to zero | The number of MR approvals required. | +| `user_approvers` | `array` of `string` | false | Username of one of more users | The users to consider as approvers. Users must have access to the project to be eligible to approve. | +| `user_approvers_ids` | `array` of `integer` | false | ID of one of more users | The IDs of users to consider as approvers. Users must have access to the project to be eligible to approve. | +| `group_approvers` | `array` of `string` | false | Path of one of more groups | The groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | +| `group_approvers_ids` | `array` of `integer` | false | ID of one of more groups | The IDs of groups to consider as approvers. Users with [direct membership in the group](../../project/merge_requests/approvals/rules.md#group-approvers) are eligible to approve. | +| `role_approvers` | `array` of `string` | false | One or more [roles](../../../user/permissions.md#roles) (for example: `owner`, `maintainer`) | The roles to consider as approvers that are eligible to approve. | Requirements and limitations: @@ -215,7 +217,16 @@ It corresponds to a single object from the previous example: There are several situations where the scan result policy requires an additional approval step. For example: -- The number of security jobs is reduced in the working branch and no longer matches the number of security jobs in the target branch. Users can't skip the Scanning Result Policies by removing scanning jobs from the CI configuration. +- The number of security jobs is reduced in the working branch and no longer matches the number of + security jobs in the target branch. Users can't skip the Scanning Result Policies by removing + scanning jobs from the CI/CD configuration. Only the security scans that are configured in the + scan result policy rules are checked for removal. + + For example, consider a situation where the default branch pipeline has four security scans: + `sast`, `secret_detection`, `container_scanning`, and `dependency_scanning`. A scan result + policy enforces two scanners: `container_scanning` and `dependency_scanning`. If an MR removes a + scan that is configured in scan result policy, `container_scanning` for example, an additional + approval is required. - Someone stops a pipeline security job, and users can't skip the security scan. - A job in a merge request fails and is configured with `allow_failure: false`. As a result, the pipeline is in a blocked state. - A pipeline has a manual job that must run successfully for the entire pipeline to pass. diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 2008375d2a2..a9bc331ae7b 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -172,6 +172,7 @@ GitLab SAST uses an advanced vulnerability tracking algorithm to more accurately Advanced vulnerability tracking is available in a subset of the [supported languages](#supported-languages-and-frameworks) and [analyzers](analyzers.md): - C, in the Semgrep-based analyzer only +- C#, in the Semgrep-based analyzer only - Go, in the Semgrep-based analyzer only - Java, in the Semgrep-based analyzer only - JavaScript, in the Semgrep-based analyzer only @@ -186,6 +187,7 @@ For more information, see the confidential project `https://gitlab.com/gitlab-or > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/368284) in GitLab 15.9 [with a project-level flag](../../../administration/feature_flags.md) named `sec_mark_dropped_findings_as_resolved`. > - Enabled by default in GitLab 15.10. On GitLab.com, [contact Support](https://about.gitlab.com/support/) if you need to disable the flag for your project. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/375128) in GitLab 16.2. To help you focus on the vulnerabilities that are still relevant, GitLab SAST automatically [resolves](../vulnerabilities/index.md#vulnerability-status-values) vulnerabilities when: @@ -290,9 +292,7 @@ The method you can use depends on your GitLab license tier. #### Configure SAST with customizations **(ULTIMATE)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3659) in GitLab 13.3. -> - [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/232862) in GitLab 13.4. -> - [Improved](https://gitlab.com/groups/gitlab-org/-/epics/3635) in GitLab 13.5. +> [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/410013) individual SAST analyzers configuration options from the UI in GitLab 16.2. NOTE: The configuration tool works best with no existing `.gitlab-ci.yml` file, or with a minimal @@ -309,9 +309,6 @@ To enable and configure SAST with customizations: Custom values are stored in the `.gitlab-ci.yml` file. For CI/CD variables not in the SAST Configuration page, their values are inherited from the GitLab SAST template. - -1. Optionally, expand the **SAST analyzers** section, select individual - [SAST analyzers](analyzers.md) and enter custom analyzer values. 1. Select **Create Merge Request**. 1. Review and merge the merge request. @@ -519,21 +516,6 @@ variables: SEARCH_MAX_DEPTH: 10 ``` -#### Logging level - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. - -To control the verbosity of logs, set the `SECURE_LOG_LEVEL` environment variable. Messages of this -logging level or higher are output. - -From highest to lowest severity, the logging levels are: - -- `fatal` -- `error` -- `warn` -- `info` (default) -- `debug` - #### Custom Certificate Authority To trust a custom Certificate Authority, set the `ADDITIONAL_CA_CERT_BUNDLE` variable to the bundle @@ -616,7 +598,7 @@ flags are added to the scanner's CLI options. | Analyzer | CLI option | Description | |------------------------------------------------------------------------------|--------------------|------------------------------------------------------------------------------| | [Semgrep](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep) | `--max-memory` | Sets the maximum system memory to use when running a rule on a single file. Measured in MB. | -| [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | `--neverignore` | Never ignore security issues, even if they have an “ignore” directive in a comment. Adding this option is likely to result in the analyzer detecting additional vulnerability findings which cannot be automatically resolved. | +| [Flawfinder](https://gitlab.com/gitlab-org/security-products/analyzers/flawfinder) | `--neverignore` | Never ignore security issues, even if they have an "ignore" directive in a comment. Adding this option is likely to result in the analyzer detecting additional vulnerability findings which cannot be automatically resolved. | | [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) | `-effort` | Sets the analysis effort level. Valid values are `min`, `less`, `more` and `max`, defined in increasing order of scan's precision and ability to detect more vulnerabilities. Default value is set to `max` which may require more memory and time to complete the scan, depending on the project's size. In case you face memory or performance issues, you may reduce the analysis effort level to a lower value. For example: `-effort less`. | #### Custom CI/CD variables @@ -772,14 +754,10 @@ By default SAST analyzers are supported in GitLab instances hosted on SELinux. A ## Troubleshooting -### SAST debug logging - -Increase the [Secure scanner log verbosity](#logging-level) to `debug` in a global CI variable to help troubleshoot SAST jobs. +### Debug-level logging -```yaml -variables: - SECURE_LOG_LEVEL: "debug" -``` +Debug-level logging can help when troubleshooting. For details, see +[debug-level logging](../index.md#debug-level-logging). ### Pipeline errors related to changes in the GitLab-managed CI/CD template diff --git a/doc/user/application_security/secret_detection/automatic_response.md b/doc/user/application_security/secret_detection/automatic_response.md index 8f78d36976b..66ed2f10a3c 100644 --- a/doc/user/application_security/secret_detection/automatic_response.md +++ b/doc/user/application_security/secret_detection/automatic_response.md @@ -23,6 +23,7 @@ GitLab supports automatic response for the following types of secrets: | GitLab [Personal access tokens](../../profile/personal_access_tokens.md) | Immediately revoke token, send email to owner | ✅ | ✅ [15.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/371658) | | Amazon Web Services (AWS) [IAM access keys](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_access-keys.html) | Notify AWS | ✅ | ⚙ | | Google Cloud [service account keys](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys), [API keys](https://cloud.google.com/docs/authentication/api-keys), and [OAuth client secrets](https://support.google.com/cloud/answer/6158849#rotate-client-secret) | Notify Google Cloud | ✅ | ⚙ | +| Postman [API keys](https://learning.postman.com/docs/developer/postman-api/authentication/) | Notify Postman; Postman emails the key owner | ✅ | ⚙ | **Component legend** diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 2e6de222ec3..ea2a66c7cc7 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -40,7 +40,7 @@ contains more than 100 patterns. Most Secret Detection patterns search for specific types of secrets. Many services add prefixes or other structural details to their secrets so they can be identified if they're leaked. -For example, GitLab [adds a `glpat-` prefix](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) to project, group, and personal access tokens by default. +For example, GitLab [adds a `glpat-` prefix](../../../administration/settings/account_and_limit_settings.md#personal-access-token-prefix) to project, group, and personal access tokens by default. To provide more reliable, high-confidence results, Secret Detection only looks for passwords or other unstructured secrets in specific contexts like URLs. @@ -83,7 +83,7 @@ Secret Detection can detect if a secret was added in one commit and removed in a - Commit range - If the `SECRET_DETECTION_LOG_OPTS` variable is set, the secrets analyzer fetches the entire + If the `SECRET_DETECTION_LOG_OPTIONS` variable is set, the secrets analyzer fetches the entire history of the branch or reference the pipeline is being run for. Secret Detection then runs, scanning the commit range specified. @@ -621,7 +621,7 @@ The check is always on; you don't have to set it up. Your text is checked for the following secret types: - GitLab [personal access tokens](../../../security/token_overview.md#personal-access-tokens) - - If a [personal access token prefix](../../../user/admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) has been configured, a token using this prefix is checked. + - If a [personal access token prefix](../../../administration/settings/account_and_limit_settings.md#personal-access-token-prefix) has been configured, a token using this prefix is checked. - GitLab [feed tokens](../../../security/token_overview.md#feed-token) This feature is separate from Secret Detection scanning, which checks your Git repository for leaked secrets. @@ -629,21 +629,10 @@ This feature is separate from Secret Detection scanning, which checks your Git r ## Troubleshooting -### Set the logging level +### Debug-level logging -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10880) in GitLab 13.1. - -Set the logging level to `debug` when you need diagnostic information in a Secret Detection job log. - -WARNING: -Debug logging can be a serious security risk. The output may contain the content of environment -variables and other secrets available to the job. The output is uploaded to the GitLab server and -visible in job logs. - -1. In the `.gitlab-ci.yml` file, set the `SECURE_LOG_LEVEL` CI/CD variable to `debug`. -1. Run the Secret Detection job. -1. Analyze the content of the Secret Detection job. -1. In the `.gitlab-ci.yml` file, set the `SECURE_LOG_LEVEL` CI/CD variable to `info` (default). +Debug-level logging can help when troubleshooting. For details, see +[debug-level logging](../index.md#debug-level-logging). ### Warning: `gl-secret-detection-report.json: no matching files` @@ -661,8 +650,8 @@ For example, you could have a pipeline triggered from a merge request containing clone is not deep enough to contain all of the relevant commits. To verify the current value, see [pipeline configuration](../../../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone). -To confirm this as the cause of the error, set the [logging level](#set-the-logging-level) to -`debug`, then rerun the pipeline. The logs should look similar to the following example. The text +To confirm this as the cause of the error, enable [debug-level logging](../index.md#debug-level-logging), +then rerun the pipeline. The logs should look similar to the following example. The text "object not found" is a symptom of this error. ```plaintext diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 39d8e28e564..1950c620627 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -199,8 +199,9 @@ To manually apply the patch that GitLab generated for a vulnerability: > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. NOTE: -Security training is not available in an offline environment because it uses content from -third-party vendors. +Security training is not accessible in an environment that is offline, meaning computers that are isolated from the public internet as a security measure. Some third-party training vendors may require you to sign up for a _free_ account. Sign up for an account by going to +any of [Secure Code Warrior](https://www.securecodewarrior.com/), [Kontra](https://application.security/), or [SecureFlag](https://www.secureflag.com/). +GitLab does not send any user information to these third-party vendors; we do send the CWE or OWASP identifier and the language name of the file extension. Security training helps your developers learn how to fix vulnerabilities. Developers can view security training from selected educational providers, relevant to the detected vulnerability. @@ -211,6 +212,8 @@ To enable security training for vulnerabilities in your project: 1. On the tab bar, select **Vulnerability Management**. 1. To enable a security training provider, turn on the toggle. +Each integration submits the Vulnerability identifier, for example CWE or OWASP, and the language to the security training vendor. The resulting link to the vendor training is what appears in a GitLab Vulnerability. + ## View security training for a vulnerability > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. @@ -218,8 +221,8 @@ To enable security training for vulnerabilities in your project: The vulnerability page may include a training link relevant to the detected vulnerability if security training is enabled. The availability of training depends on whether the enabled training vendor has content matching the particular vulnerability. Training content is requested based on the [vulnerability identifiers](../../../development/integrations/secure.md#identifiers). -The identifier given to a vulnerability varies from one vulnerability to the next. The available training -content varies between vendors. This means some vulnerabilities do not display training content. +The identifier given to a vulnerability varies from one vulnerability to the next and the available training +content varies between vendors. Some vulnerabilities do not display training content. Vulnerabilities with a CWE are most likely to return a training result. To view the security training for a vulnerability: diff --git a/doc/user/application_security/vulnerabilities/severities.md b/doc/user/application_security/vulnerabilities/severities.md index ab90ac18b8e..9553d15bbe9 100644 --- a/doc/user/application_security/vulnerabilities/severities.md +++ b/doc/user/application_security/vulnerabilities/severities.md @@ -18,7 +18,33 @@ most to least severe: - `Info` - `Unknown` -Most GitLab vulnerability analyzers are wrappers around popular open source scanning tools. Each +GitLab analyzers make an effort to fit the severity descriptions below, but they may not always be correct. Analyzers and scanners provided by third-party vendors may not follow the same classification. + +## Critical severity + +Vulnerabilities identified at the Critical Severity level should be investigated immediately. Vulnerabilities at this level assume exploitation of the flaw could lead to full system or data compromise. Examples of critical severity flaws are Command/Code Injection and SQL Injection. Typically these flaws are rated with CVSS 3.1 between 9.0-10.0. + +## High severity + +High severity vulnerabilities can be characterized as flaws that may lead to an attacker accessing application resources or unintended exposure of data. Examples of high severity flaws are External XML Entity Injection (XXE), Server Side Request Forgery (SSRF), Local File Include/Path Traversal and certain forms of Cross-Site Scripting (XSS). Typically these flaws are rated with CVSS 3.1 between 7.0-8.9. + +## Medium severity + +Medium severity vulnerabilities usually arise from misconfiguration of systems or lack of security controls. Exploitation of these vulnerabilities may lead to accessing a restricted amount of data or could be used in conjunction with other flaws to gain unintended access to systems or resources. Examples of medium severity flaws are reflected XSS, incorrect HTTP session handling, and missing security controls. Typically these flaws are rated with CVSS 3.1 between 4.0-6.9. + +## Low severity + +Low severity vulnerabilities contain flaws that may not be directly exploitable but introduce unnecessary weakness to an application or system. These flaws are usually due to missing security controls, or unnecessary disclose information about the application environment. Examples of low severity vulnerabilities are missing cookie security directives, verbose error or exception messages. Typically these flaws are rated with CVSS 3.1 between 1.0-3.9. + +## Info severity + +Info level severity vulnerabilities contain information that may have value, but are not necessarily associated to a particular flaw or weakness. Typically these issues do not have a CVSS rating. + +## Unknown severity + +Issues identified at this level do not have enough context to clearly demonstrate severity. + +GitLab vulnerability analyzers include popular open source scanning tools. Each open source scanning tool provides their own native vulnerability severity level value. These values can be one of the following: diff --git a/doc/user/asciidoc.md b/doc/user/asciidoc.md index f04fece7b76..d2041083d36 100644 --- a/doc/user/asciidoc.md +++ b/doc/user/asciidoc.md @@ -411,7 +411,7 @@ Color written inside backticks is followed by a color "chip": ### Equations and Formulas (STEM) -If you need to include Science, Technology, Engineering and Math (STEM) +If you need to include Science, Technology, Engineering, and Math (STEM) expressions, set the `stem` attribute in the document's header to `latexmath`. Equations and formulas are rendered using [KaTeX](https://katex.org/): diff --git a/doc/user/award_emojis.md b/doc/user/award_emojis.md index c3aad86461b..aaba0225653 100644 --- a/doc/user/award_emojis.md +++ b/doc/user/award_emojis.md @@ -6,11 +6,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Emoji reactions **(FREE)** -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emojis" to "emoji reactions" in GitLab 16.0. -> - Reacting with emojis on work items (such as tasks, objectives, and key results) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393599) in GitLab 16.0. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/409884) from "award emoji" to "emoji reactions" in GitLab 16.0. +> - Reacting with emoji on work items (such as tasks, objectives, and key results) [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/393599) in GitLab 16.0. +> - Reacting with emoji on design discussion comments [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29756) in GitLab 16.2. When you're collaborating online, you get fewer opportunities for high-fives -and thumbs-ups. React with emojis on: +and thumbs-ups. React with emoji on: - [Issues](project/issues/index.md). - [Tasks](tasks.md). @@ -25,19 +26,10 @@ and thumbs-ups. React with emojis on: Emoji reactions make it much easier to give and receive feedback without a long comment thread. -For information on the relevant API, see [Emoji reactions API](../api/award_emoji.md). - -## Sort issues and merge requests on vote count - -You can quickly sort issues and merge requests by the number of votes ("thumbs up" and "thumbs down" emoji) they -have received. The sort options can be found in the dropdown list as "Most -popular" and "Least popular". +"Thumbs up" and "thumbs down" emoji are used to calculate an issue or merge request's position when +[sorting by popularity](project/issues/sorting_issue_lists.md#sorting-by-popularity). - - -The total number of votes is not summed up. An issue with 18 upvotes and 5 -downvotes is considered more popular than an issue with 17 upvotes and no -downvotes. +For information on the relevant API, see [Emoji reactions API](../api/award_emoji.md). ## Emoji reactions for comments @@ -51,14 +43,26 @@ To add an emoji reaction: To remove an emoji reaction, select the emoji again. -## Custom emojis +## Custom emoji + +> - [Introduced for GraphQL API](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37911) in GitLab 13.6 [with a flag](../administration/feature_flags.md) named `custom_emoji`. Disabled by default. +> - Enabled on GitLab.com in GitLab 14.0. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/333095) UI to add emoji in GitLab 16.2. + +Custom emoji show in the emoji picker everywhere you can react with emoji. +To add an emoji reaction to a comment or description: + +1. Select **Add reaction** (**{slight-smile}**). +1. Select the GitLab logo (**{tanuki}**) or scroll down to the **Custom** section. +1. Select an emoji from the emoji picker. + + -You can upload custom emojis to a GitLab instance with the GraphQL API. -For more information, see [Use custom emojis with GraphQL](../api/graphql/custom_emoji.md). +To use them in a text box, type the filename between two colons. +For example, `:thank-you:`. -Custom emojis don't show in the emoji picker. -To use them in a text box, type the filename without the extension and surrounded by colons. -For example, for a file named `thank-you.png`, type `:thank-you:`. +You can upload custom emoji to a GitLab instance with the GraphQL API. +For more information, see [Use custom emoji with GraphQL](../api/graphql/custom_emoji.md). -For a list of custom emojis available for GitLab.com, see +For a list of custom emoji available for GitLab.com, see [the `custom_emoji` project](https://gitlab.com/custom_emoji/custom_emoji/-/tree/main/img). diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md index 106f751555a..aff78ed477b 100644 --- a/doc/user/clusters/agent/gitops.md +++ b/doc/user/clusters/agent/gitops.md @@ -11,10 +11,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/346567) from GitLab Premium to GitLab Free in 15.3. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/346585) to make the `id` attribute optional in GitLab 15.7. > - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/395364) in GitLab 16.1 to prioritize Flux for GitOps. -NOTE: -From GitLab 15.10, you should use [Flux](gitops/flux.md) for GitOps. For more information, see -[this announcement blog post](https://about.gitlab.com/blog/2023/02/08/why-did-we-choose-to-integrate-fluxcd-with-gitlab/). +GitLab integrates [Flux](https://fluxcd.io/flux/) for GitOps. +To get started with Flux, see the [Flux for GitOps tutorial](gitops/flux_tutorial.md). With GitOps, you can manage containerized clusters and applications from a Git repository that: @@ -25,163 +25,155 @@ By combining GitLab, Kubernetes, and GitOps, you can have: - GitLab as the GitOps operator. - Kubernetes as the automation and convergence system. -- GitLab CI/CD for Continuous Integration and the agent for Continuous Deployment. +- GitLab CI/CD for Continuous Integration. +- The agent for Continuous Deployment and cluster observability. - Built-in automatic drift remediation. - Resource management with [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/) for transparent multi-actor field management. +## Deployment sequence + This diagram shows the repositories and main actors in a GitOps deployment: ```mermaid sequenceDiagram participant D as Developer participant A as Application code repository - participant M as Manifest repository - participant K as GitLab agent + participant M as Deployment repository + participant R as OCI registry participant C as Agent configuration repository + participant K as GitLab agent + participant F as Flux loop Regularly K-->>C: Grab the configuration end D->>+A: Pushing code changes A->>M: Updating manifest - loop Regularly - K-->>M: Watching changes - M-->>K: Pulling and applying changes - end + M->>R: Build an OCI artifact + M->>K: Notify + K->>F: Notify and watch sync + R-->>F: Pulling and applying changes + K->>M: Notify after sync ``` -For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). +You should use both Flux and `agentk` for GitOps deployments. Flux keeps the cluster state synchronized with the source, while `agentk` simplifies the Flux setup, provides cluster-to-GitLab access management, and visualizes the cluster state in the GitLab UI. -## GitOps workflow steps +### OCI for source control -To update a Kubernetes cluster by using GitOps, complete the following steps. +You should use OCI images as a source controller for Flux, instead of a Git repository. The [GitLab container registry](../../packages/container_registry/index.md) supports OCI images. -1. Ensure you have a working Kubernetes cluster, and that the manifests or [Helm charts](gitops/helm.md) are in a GitLab project. -1. In the same project, [register and install the GitLab agent](install/index.md). -1. Configure the agent configuration file so that the agent monitors the project for changes to the Kubernetes manifests. - Use the [GitOps configuration reference](#gitops-configuration-reference) for guidance. +| OCI registry | Git repository | +| --- | --- | +| Designed to serve container images at scale. | Designed to version and store source code. | +| Immutable, supports security scans. | Mutable. | +| The default Git branch can store cluster state without triggering a sync. | The default Git branch triggers a sync when used to store cluster state. | -Any time you commit updates to your Kubernetes manifests, the agent updates the cluster. +## Repository structure -## GitOps configuration reference +To simplify configuration, use one delivery repository per team. +You can package the delivery repository into multiple OCI images per application. -The following snippet shows an example of the possible keys and values for the GitOps section of an [agent configuration file](install/index.md#create-an-agent-configuration-file) (`config.yaml`). +For additional repository structure recommendations, see the [Flux documentation](https://fluxcd.io/flux/guides/repository-structure/). -```yaml -gitops: - manifest_projects: - - id: gitlab-org/cluster-integration/gitlab-agent - ref: # either `branch`, `tag` or `commit` can be specified - branch: production - # commit: <mysha> - # tag: v1.0 - default_namespace: my-ns - paths: - # Read all YAML files from this directory. - - glob: '/team1/app1/*.yaml' - # Read all .yaml files from team2/apps and all subdirectories. - - glob: '/team2/apps/**/*.yaml' - # If 'paths' is not specified or is an empty list, the configuration below is used. - - glob: '/**/*.{yaml,yml,json}' - reconcile_timeout: 3600s - dry_run_strategy: none - prune: true - prune_timeout: 3600s - prune_propagation_policy: foreground - inventory_policy: must_match -``` +## Immediate Git repository reconciliation -| Keyword | Description | -|--|--| -| `manifest_projects` | Projects where your Kubernetes manifests are stored. The agent monitors the files in the repositories in these projects. When manifest files change, the agent deploys the changes to the cluster. | -| `id` | Path to a Git repository that has Kubernetes manifests in YAML or JSON format. No authentication mechanisms are supported. Default is the agent configuration repository. | -| `ref` | Optional. Git reference in the configured Git repository to fetch the Kubernetes manifest files from. If not specified or empty, the default branch is used. If specified, it must contain either `branch`, `tag`, or `commit`. | -| `ref.branch` | Branch name in the configured Git repository to fetch the Kubernetes manifest files from. | -| `ref.tag` | Tag name in the configured Git repository to fetch the Kubernetes manifest files from. | -| `ref.commit` | Commit SHA in the configured Git repository to fetch the Kubernetes manifest files from. | -| `default_namespace` | Namespace to use if not set explicitly in object manifest. Also used for inventory `ConfigMap` objects. | -| `paths` | Repository paths to scan for manifest files. Directories with names that start with a dot `(.)` are ignored. | -| `paths[].glob` | Required. See [doublestar](https://github.com/bmatcuk/doublestar#about) and [the match function](https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match) for globbing rules. | -| `reconcile_timeout` | Determines whether the applier should wait until all applied resources have been reconciled, and if so, how long to wait. Default is 3600 seconds (1 hour). | -| `dry_run_strategy` | Determines whether changes [should be performed](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89). Can be: `none`, `client`, or `server`. Default is `none`.| -| `prune` | Determines whether pruning of previously applied objects should happen after apply. Default is `true`. | -| `prune_timeout` | Determines whether to wait for all resources to be fully deleted after pruning, and if so, how long to wait. Default is 3600 seconds (1 hour). | -| `prune_propagation_policy` | The deletion propagation policy that [should be used for pruning](https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470). Can be: `orphan`, `background`, or `foreground`. Default is `foreground`. | -| `inventory_policy` | Determines whether an inventory object can take over objects that belong to another inventory object or don't belong to any inventory object. This is done by determining if the apply/prune operation can go through for a resource based on comparison of the `inventory-id` value in the package and the `owning-inventory` annotation (`config.k8s.io/owning-inventory`) [in the live object](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66). Can be: `must_match`, `adopt_if_no_inventory`, or `adopt_all`. Default is `must_match`. | +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/392852) in GitLab 16.1 with a [flag](../../../administration/feature_flags.md) named `notify_kas_on_git_push`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/126527) in GitLab 16.2. -## GitOps annotations +Usually, the Flux source controller reconciles Git repositories at configured intervals. +This can cause delays between a `git push` and the reconciliation of the cluster state, and results in +unnecessary pulls from GitLab. -The GitLab agent for Kubernetes has annotations you can use to: +The agent for Kubernetes automatically detects Flux `GitRepository` objects that +reference GitLab projects in the instance the agent is connected to, +and configures a [`Receiver`](https://fluxcd.io/flux/components/notification/receiver/) for the instance. +When the agent for Kubernetes detects a `git push`, the `Receiver` is triggered +and Flux reconciles the cluster with any changes to the repository. -- **Sort resources**: Apply or delete resources in a specific order. -- **Use apply-time mutation**: Dynamically substitute fields from one resource configuration to another. +To use immediate Git repository reconciliation, you must have a Kubernetes cluster that runs: -The agent has [default sorting](https://github.com/kubernetes-sigs/cli-utils/blob/d7d63f4b62897f584ca9e02b6faf4d2f327a9b09/pkg/ordering/sort.go#L74), -but with annotations, you can fine-tune the order and apply time-value injection. +- The agent for Kubernetes. +- Flux `source-controller` and `notification-controller`. -To provide the GitOps functionality, the GitLab agent for Kubernetes uses the [`cli-utils` library](https://github.com/kubernetes-sigs/cli-utils/), -a Kubernetes SIG project. For more information, see the available annotations in the [`cli-utils` documentation](https://github.com/kubernetes-sigs/cli-utils/blob/master/README.md). +Immediate Git repository reconciliation can reduce the time between a push and reconciliation, +but it doesn't guarantee that every `git push` event is received. You should still set +[`GitRepository.spec.interval`](https://fluxcd.io/flux/components/source/gitrepositories/#interval) +to an acceptable duration. -## Automatic drift remediation +### Custom webhook endpoints -Drift happens when the current configuration of an infrastructure resource differs from its desired configuration. -Typically, this is caused by manually editing resources directly rather than via the used infrastructure-as-code -mechanism. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. +When the agent for Kubernetes calls the `Receiver` webhook, +the agent defaults to `http://webhook-receiver.flux-system.svc.cluster.local`, +which is also the default URL set by a Flux bootstrap installation. To configure a custom +endpoint, set `flux.webhook_receiver_url` to a URL that the agent can resolve. For example: -In GitLab, the agent for Kubernetes regularly compares the desired state from the `git` repository with -the actual state from the Kubernetes cluster. Deviations from the `git` state are fixed at every check. These checks -happen automatically every 5 minutes. They are not configurable. +```yaml +flux: + webhook_receiver_url: http://webhook-receiver.another-flux-namespace.svc.cluster.local +``` -The agent uses [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/). -As a result, every field in a resource can have different managers. Only fields managed by `git` -are checked for drift. This facilitates the use of in-cluster controllers to modify resources like -[Horizontal Pod Autoscalers](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/). +There is special handing for +[service proxy URLs](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster-services/) configured +in this format: `/api/v1/namespaces/[^/]+/services/[^/]+/proxy`. For example: -## Related topics +```yaml +flux: + webhook_receiver_url: /api/v1/namespaces/flux-system/services/http:webhook-receiver:80/proxy +``` -- [Deploying Helm charts with the GitOps workflow](gitops/helm.md) -- [GitOps working examples for training and demos](https://gitlab.com/groups/guided-explorations/gl-k8s-agent/gitops/-/wikis/home) -- [Self-paced classroom workshop](https://gitlab-for-eks.awsworkshop.io) (Uses AWS EKS, but you can use for other Kubernetes clusters) -- [Managing Kubernetes secrets in a GitOps workflow](gitops/secrets_management.md) -- [Application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) +In these cases, the agent for Kubernetes uses the available Kubernetes configuration +and context to connect to the API endpoint. +You can use this if you run an agent outside a cluster +and you haven't [configured an `Ingress`](https://fluxcd.io/flux/guides/webhook-receivers/#expose-the-webhook-receiver) +for the Flux notification controller. + +WARNING: +You should configure only trusted service proxy URLs. +When you provide a service proxy URL, +the agent for Kubernetes sends typical Kubernetes API requests which include +the credentials necessary to authenticate with the API service. -## Troubleshooting +## Token management -### Avoiding conflicts when you have multiple projects +To use certain Flux features, you might need multiple access tokens. Additionally, you can use multiple token types to achieve the same result. -The agent watches each glob pattern set under a project's `paths` section independently, and makes updates to the cluster concurrently. -If changes are found at multiple paths, when the agent attempts to update the cluster, -a conflict can occur. +This section provides guidelines for the tokens you might need, and provides token type recommendations where possible. -To prevent this from happening, consider storing a logical group of manifests in a single place and reference them only once to avoid overlapping globs. +### GitLab access by Flux -For example, both of these globs match `*.yaml` files in the root directory -and could cause conflicts: +To access the GitLab the container registry or Git repositories, Flux can use: -```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' - - glob: '/*.yaml' -``` +- A project or group deploy token. +- A project or group deploy key. +- A project or group access token. +- A personal access token. -Instead, specify a single glob that matches all `*.yaml` files recursively: +The token does not need write access. -```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' -``` +You should use project deploy tokens if `http` access is possible. +If you require `git+ssh` access, you should use deploy keys. +To compare deploy keys and deploy tokens, see [Deploy keys](../../project/deploy_keys/index.md). -### Use multiple agents or projects +Support for automating deploy token creation, rotation, and reporting is proposed in [issue 389393](https://gitlab.com/gitlab-org/gitlab/-/issues/389393). -If you store your Kubernetes manifests in separate GitLab projects, -update your agent configuration file with the location of these projects. +### Flux to GitLab notification -WARNING: -The project with the agent's -configuration file can be private or public. Other projects with Kubernetes manifests must be public. Support for private manifest projects is tracked -in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7704). +If you configure Flux to synchronize from a Git source, [Flux can register an external job status](https://fluxcd.io/flux/components/notification/provider/#git-commit-status-updates) in GitLab pipelines. + +To get external job statuses from Flux, you can use: + +- A project or group deploy token. +- A project or group access token. +- A personal access token. + +The token requires `api` scope. To minimize the attack surface of a leaked token, you should use +a project access token. + +Integrating Flux into GitLab pipelines as a job is proposed in [issue 405007](https://gitlab.com/gitlab-org/gitlab/-/issues/405007). + +## Related topics + +- [GitOps working examples for training and demos](https://gitlab.com/groups/guided-explorations/gl-k8s-agent/gitops/-/wikis/home) +- [Self-paced classroom workshop](https://gitlab-for-eks.awsworkshop.io) (Uses AWS EKS, but you can use for other Kubernetes clusters) +- Managing Kubernetes secrets in a GitOps workflow + - [with SOPS built-in to Flux](https://fluxcd.io/flux/guides/mozilla-sops/) + - [with Sealed Secrets](https://fluxcd.io/flux/guides/sealed-secrets/) diff --git a/doc/user/clusters/agent/gitops/agent.md b/doc/user/clusters/agent/gitops/agent.md new file mode 100644 index 00000000000..07ed2b3a691 --- /dev/null +++ b/doc/user/clusters/agent/gitops/agent.md @@ -0,0 +1,255 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Using GitOps with the agent for Kubernetes (deprecated) **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/346567) from GitLab Premium to GitLab Free in 15.3. +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/346585) to make the `id` attribute optional in GitLab 15.7. +> - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. + +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/406545) in GitLab 16.2. You should use the [Flux integration](../gitops.md) for GitOps. + +This diagram shows the repositories and main actors in a GitOps deployment: + +```mermaid +sequenceDiagram + participant D as Developer + participant A as Application code repository + participant M as Manifest repository + participant K as GitLab agent + participant C as Agent configuration repository + loop Regularly + K-->>C: Grab the configuration + end + D->>+A: Pushing code changes + A->>M: Updating manifest + loop Regularly + K-->>M: Watching changes + M-->>K: Pulling and applying changes + end +``` + +For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). + +## Migrate to Flux + +You should migrate your legacy GitOps deployments so they can be +deployed with Flux. To migrate, configure Flux to deploy manifests +with Kustomize. If you don't have a `kustomization.yaml` file in +the given path, it is generated automatically. + +Prerequisites: + +- A configuration like: + + ```yaml + manifest_projects: + - id: my-group/my-project + default_namespace: production + paths: + - glob: 'environments/production/**/*.yaml' + ``` + +- A [Flux installation](https://fluxcd.io/flux/installation/) with manifests in `environments/flux-system`. +- You use a deploy token to access GitLab. +- Your cluster can access GitLab over HTTPS. + +To migrate: + +1. Create a file called `environments/flux-system/production.yaml` with the following contents: + + ```yaml + # This manifest was generated by flux. DO NOT EDIT. + --- + apiVersion: source.toolkit.fluxcd.io/v1 + kind: GitRepository + metadata: + name: production + namespace: flux-system + spec: + interval: 1m0s + ref: + branch: main + secretRef: + name: gitlab-deploy-token + url: https://gitlab.example.com/my-group/my-project.git + --- + apiVersion: kustomize.toolkit.fluxcd.io/v1 + kind: Kustomization + metadata: + name: production + namespace: flux-system + spec: + interval: 10m0s + path: ./environments/production + prune: true + sourceRef: + kind: GitRepository + name: production + ``` + +1. Optional. Because `agentk` uses the `default_namespace` by default, you might need to add a `kustomization.yaml` file to `/environments/production/` and list the relevant resources. + For example: + + ```yaml + apiVersion: kustomize.config.k8s.io/v1beta1 + kind: Kustomization + namespace: default + resources: + - relative/path/to-your/resource-1.yaml + - relative/path/to-your/resource-2.yaml + ``` + + When you commit the `kustomization.yaml` file to the repository, a reconciliation with Flux and `agentk` is triggered. + Because `agentk` can't handle the Kustomization file, it logs errors when you commit the file. + +1. To ensure Flux has taken over the management of the resource, check for the resource in the `status.inventory` value of the `production` Flux Kustomization object: + + ```shell + kubectl get kustomization production -n flux-system -o json | jq '.status.inventory.entries' + ``` + +1. Remove the entry from the `manifest_projects` list. + +After you migrate, your GitOps deployments deploy with Flux. +To get the most out of your Flux integration, see the [Flux Kustomization CRD](https://fluxcd.io/flux/components/kustomize/kustomization/) and [GitLab Flux documentation](../gitops.md). + +## GitOps workflow steps + +To update a Kubernetes cluster by using GitOps, complete the following steps. + +1. Ensure you have a working Kubernetes cluster, and that the manifests are in a GitLab project. +1. In the same project, [register and install the GitLab agent](../install/index.md). +1. Configure the agent configuration file so that the agent monitors the project for changes to the Kubernetes manifests. + Use the [GitOps configuration reference](#gitops-configuration-reference) for guidance. + +Any time you commit updates to your Kubernetes manifests, the agent updates the cluster. + +## GitOps configuration reference + +The following snippet shows an example of the possible keys and values for the GitOps section of an [agent configuration file](../install/index.md#create-an-agent-configuration-file) (`config.yaml`). + +```yaml +gitops: + manifest_projects: + - id: gitlab-org/cluster-integration/gitlab-agent + ref: # either `branch`, `tag` or `commit` can be specified + branch: production + # commit: <mysha> + # tag: v1.0 + default_namespace: my-ns + paths: + # Read all YAML files from this directory. + - glob: '/team1/app1/*.yaml' + # Read all .yaml files from team2/apps and all subdirectories. + - glob: '/team2/apps/**/*.yaml' + # If 'paths' is not specified or is an empty list, the configuration below is used. + - glob: '/**/*.{yaml,yml,json}' + reconcile_timeout: 3600s + dry_run_strategy: none + prune: true + prune_timeout: 3600s + prune_propagation_policy: foreground + inventory_policy: must_match +``` + +| Keyword | Description | +|--|--| +| `manifest_projects` | Projects where your Kubernetes manifests are stored. The agent monitors the files in the repositories in these projects. When manifest files change, the agent deploys the changes to the cluster. | +| `id` | Path to a Git repository that has Kubernetes manifests in YAML or JSON format. No authentication mechanisms are supported. Default is the agent configuration repository. | +| `ref` | Optional. Git reference in the configured Git repository to fetch the Kubernetes manifest files from. If not specified or empty, the default branch is used. If specified, it must contain either `branch`, `tag`, or `commit`. | +| `ref.branch` | Branch name in the configured Git repository to fetch the Kubernetes manifest files from. | +| `ref.tag` | Tag name in the configured Git repository to fetch the Kubernetes manifest files from. | +| `ref.commit` | Commit SHA in the configured Git repository to fetch the Kubernetes manifest files from. | +| `default_namespace` | Namespace to use if not set explicitly in object manifest. Also used for inventory `ConfigMap` objects. | +| `paths` | Repository paths to scan for manifest files. Directories with names that start with a dot `(.)` are ignored. | +| `paths[].glob` | Required. See [doublestar](https://github.com/bmatcuk/doublestar#about) and [the match function](https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match) for globbing rules. | +| `reconcile_timeout` | Determines whether the applier should wait until all applied resources have been reconciled, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `dry_run_strategy` | Determines whether changes [should be performed](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89). Can be: `none`, `client`, or `server`. Default is `none`.| +| `prune` | Determines whether pruning of previously applied objects should happen after apply. Default is `true`. | +| `prune_timeout` | Determines whether to wait for all resources to be fully deleted after pruning, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `prune_propagation_policy` | The deletion propagation policy that [should be used for pruning](https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470). Can be: `orphan`, `background`, or `foreground`. Default is `foreground`. | +| `inventory_policy` | Determines whether an inventory object can take over objects that belong to another inventory object or don't belong to any inventory object. This is done by determining if the apply/prune operation can go through for a resource based on comparison of the `inventory-id` value in the package and the `owning-inventory` annotation (`config.k8s.io/owning-inventory`) [in the live object](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66). Can be: `must_match`, `adopt_if_no_inventory`, or `adopt_all`. Default is `must_match`. | + +## GitOps annotations + +The GitLab agent for Kubernetes has annotations you can use to: + +- **Sort resources**: Apply or delete resources in a specific order. +- **Use apply-time mutation**: Dynamically substitute fields from one resource configuration to another. + +The agent has [default sorting](https://github.com/kubernetes-sigs/cli-utils/blob/d7d63f4b62897f584ca9e02b6faf4d2f327a9b09/pkg/ordering/sort.go#L74), +but with annotations, you can fine-tune the order and apply time-value injection. + +To provide the GitOps functionality, the GitLab agent for Kubernetes uses the [`cli-utils` library](https://github.com/kubernetes-sigs/cli-utils/), +a Kubernetes SIG project. For more information, see the available annotations in the [`cli-utils` documentation](https://github.com/kubernetes-sigs/cli-utils/blob/master/README.md). + +## Automatic drift remediation + +Drift happens when the current configuration of an infrastructure resource differs from its desired configuration. +Typically, this is caused by manually editing resources directly rather than via the used infrastructure-as-code +mechanism. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. + +In GitLab, the agent for Kubernetes regularly compares the desired state from the `git` repository with +the actual state from the Kubernetes cluster. Deviations from the `git` state are fixed at every check. These checks +happen automatically every 5 minutes. They are not configurable. + +The agent uses [server-side applies](https://kubernetes.io/docs/reference/using-api/server-side-apply/). +As a result, every field in a resource can have different managers. Only fields managed by `git` +are checked for drift. This facilitates the use of in-cluster controllers to modify resources like +[Horizontal Pod Autoscalers](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/). + +## Related topics + +- [GitOps working examples for training and demos](https://gitlab.com/groups/guided-explorations/gl-k8s-agent/gitops/-/wikis/home) +- [Self-paced classroom workshop](https://gitlab-for-eks.awsworkshop.io) (Uses AWS EKS, but you can use for other Kubernetes clusters) +- [Managing Kubernetes secrets in a GitOps workflow](secrets_management.md) +- [Application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) + +## Troubleshooting + +### Avoiding conflicts when you have multiple projects + +The agent watches each glob pattern set under a project's `paths` section independently, and makes updates to the cluster concurrently. +If changes are found at multiple paths, when the agent attempts to update the cluster, +a conflict can occur. + +To prevent this from happening, consider storing a logical group of manifests in a single place and reference them only once to avoid overlapping globs. + +For example, both of these globs match `*.yaml` files in the root directory +and could cause conflicts: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' + - glob: '/*.yaml' +``` + +Instead, specify a single glob that matches all `*.yaml` files recursively: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' +``` + +### Use multiple agents or projects + +If you store your Kubernetes manifests in separate GitLab projects, +update your agent configuration file with the location of these projects. + +WARNING: +The project with the agent's +configuration file can be private or public. Other projects with Kubernetes manifests must be public. Support for private manifest projects is tracked +in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7704). diff --git a/doc/user/clusters/agent/gitops/example_repository_structure.md b/doc/user/clusters/agent/gitops/example_repository_structure.md index 93c05e40bfd..a5bc3b153fe 100644 --- a/doc/user/clusters/agent/gitops/example_repository_structure.md +++ b/doc/user/clusters/agent/gitops/example_repository_structure.md @@ -60,7 +60,7 @@ the deployment branch back to the repository using the provided token. To preser commit history between both branches, the CI/CD job uses a fast-forward merge. Each cluster has an agent for Kubernetes, and each agent is configured to -[sync manifests from the branch corresponding to its cluster](../gitops.md#gitops-configuration-reference). +sync manifests from the branch corresponding to its cluster. In your own project, you can different GitOps tool like Flux, or use the same configuration to deploy to virtual machines with GitLab CI/CD. diff --git a/doc/user/clusters/agent/gitops/flux.md b/doc/user/clusters/agent/gitops/flux.md index f0a681c1a5c..13fcfeb28e9 100644 --- a/doc/user/clusters/agent/gitops/flux.md +++ b/doc/user/clusters/agent/gitops/flux.md @@ -1,97 +1,11 @@ --- -stage: Deploy -group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../gitops.md' +remove_date: '2023-10-12' --- -# Flux (Beta) **(FREE)** +This document was moved to [another location](../gitops.md). -Flux is a GitOps tool that helps you manage your Kubernetes clusters. -You can use Flux to: - -- Keep your clusters in sync with your Git repositories. -- Reconcile code changes with your deployments. -- Manage your Flux installation itself with a bootstrap. - -You can use the agent for Kubernetes with Flux to: - -- Trigger immediate Git repository reconciliation. - -To get started, see the [Flux installation documentation](https://fluxcd.io/flux/installation). - -Support for Flux is in [Beta](../../../../policy/experiment-beta-support.md#beta). - -## Bootstrap installation - -Use the Flux command [`bootstrap gitlab`](https://fluxcd.io/flux/installation/#gitlab-and-gitlab-enterprise) -to configure a Kubernetes cluster to manage itself from a Git repository. - -You must authenticate your installation with either: - -- Recommended. [A project access token](../../../project/settings/project_access_tokens.md). -- A [group access token](../../../group/settings/group_access_tokens.md). -- A [personal access token](../../../profile/personal_access_tokens.md). - -Some Flux features like [automated image updates](https://fluxcd.io/flux/guides/image-update/) require -write access to the source repositories. - -## GitOps repository structure - -You should organize your repositories to meet the needs of your team. For detailed recommendations, see the Flux [repository structure documentation](https://fluxcd.io/flux/guides/repository-structure/). - -## Immediate Git repository reconciliation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/392852) in GitLab 16.1. - -Usually, the Flux source controller reconciles Git repositories at configured intervals. -This can cause delays between a `git push` and the reconciliation of the cluster state, and results in -unnecessary pulls from GitLab. - -The agent for Kubernetes automatically detects Flux `GitRepository` objects that -reference GitLab projects in the instance the agent is connected to, -and configures a [`Receiver`](https://fluxcd.io/flux/components/notification/receiver/) for the instance. -When the agent for Kubernetes detects a `git push`, the `Receiver` is triggered -and Flux reconciles the cluster with any changes to the repository. - -To use immediate Git repository reconciliation, you must have a Kubernetes cluster that runs: - -- The agent for Kubernetes. -- Flux `source-controller` and `notification-controller`. - -Immediate Git repository reconciliation can reduce the time between a push and reconciliation, -but it doesn't guarantee that every `git push` event is received. You should still set -[`GitRepository.spec.interval`](https://fluxcd.io/flux/components/source/gitrepositories/#interval) -to an acceptable duration. - -### Custom webhook endpoints - -When the agent for Kubernetes calls the `Receiver` webhook, -the agent defaults to `http://webhook-receiver.flux-system.svc.cluster.local`, -which is also the default URL set by a Flux bootstrap installation. To configure a custom -endpoint, set `flux.webhook_receiver_url` to a URL that the agent can resolve. For example: - -```yaml -flux: - webhook_receiver_url: http://webhook-receiver.another-flux-namespace.svc.cluster.local -``` - -There is special handing for -[service proxy URLs](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster-services/) configured -in this format: `/api/v1/namespaces/[^/]+/services/[^/]+/proxy`. For example: - -```yaml -flux: - webhook_receiver_url: /api/v1/namespaces/flux-system/services/http:webhook-receiver:80/proxy -``` - -In these cases, the agent for Kubernetes uses the available Kubernetes configuration -and context to connect to the API endpoint. -You can use this if you run an agent outside a cluster -and you haven't [configured an `Ingress`](https://fluxcd.io/flux/guides/webhook-receivers/#expose-the-webhook-receiver) -for the Flux notification controller. - -WARNING: -You should configure only trusted service proxy URLs. -When you provide a service proxy URL, -the agent for Kubernetes sends typical Kubernetes API requests which include -the credentials necessary to authenticate with the API service. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/clusters/agent/gitops/flux_tutorial.md b/doc/user/clusters/agent/gitops/flux_tutorial.md index c6c9ed9e373..8aee0c01d65 100644 --- a/doc/user/clusters/agent/gitops/flux_tutorial.md +++ b/doc/user/clusters/agent/gitops/flux_tutorial.md @@ -9,6 +9,9 @@ info: A tutorial using Flux This tutorial teaches you how to set up Flux for GitOps. You'll complete a bootstrap installation, install `agentk` in your cluster, and deploy a simple `nginx` application. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview of an example Flux +configuration, see [Flux bootstrap and manifest synchronization with GitLab](https://www.youtube.com/watch?v=EjPVRM-N_PQ). + To set up Flux for GitOps: 1. [Create a personal access token](#create-a-personal-access-token) @@ -97,7 +100,7 @@ To install `agentk`: apiVersion: v1 kind: Namespace metadata: - name: gitlab + name: gitlab ``` 1. Apply the agent registration token as a secret in the cluster: @@ -140,7 +143,7 @@ To install `agentk`: sourceRef: kind: HelmRepository name: gitlab-agent - namespace: gitlab-agent + namespace: gitlab interval: 1h0m0s values: config: @@ -177,8 +180,6 @@ To demonstrate, deploy an `nginx` application and point Flux at it: interval: 1m0s ref: branch: main - secretRef: - name: example-nginx-app url: https://gitlab.com/gitlab-examples/ops/gitops-demo/example-mini-flux-deployment.git --- apiVersion: kustomize.toolkit.fluxcd.io/v1 diff --git a/doc/user/clusters/agent/gitops/helm.md b/doc/user/clusters/agent/gitops/helm.md index 182fd87e6f9..22587cd0e5d 100644 --- a/doc/user/clusters/agent/gitops/helm.md +++ b/doc/user/clusters/agent/gitops/helm.md @@ -1,152 +1,11 @@ --- -stage: Deploy -group: Environments -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +redirect_to: '../gitops.md' +remove_date: '2023-10-22' --- -# Using Helm charts to update a Kubernetes cluster (Experiment) **(FREE)** +This document was moved to [another location](../gitops.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371019) in GitLab 15.4. -> - Specifying a branch, tag, or commit reference to fetch the Kubernetes manifest files [introduced](https://gitlab.com/groups/gitlab-org/-/epics/4516) in GitLab 15.7. - -You can deploy Helm charts to your Kubernetes cluster and keep the resources in your cluster in sync -with your charts and values. To do this, you use the pull-based GitOps features of the agent for -Kubernetes. - -This feature is an Experiment and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/7938) -to track future work. Tell us about your use cases by leaving comments in the epic. - -NOTE: -This feature is an Experiment. In future releases, to accommodate new features, the configuration format might change without notice. - -## GitOps workflow steps - -To update a Kubernetes cluster by using GitOps with charts, complete the following steps. - -1. Ensure you have a working Kubernetes cluster, and that the chart is in a GitLab project. -1. In the same project, [register and install the GitLab agent](../install/index.md). -1. Configure the agent configuration file so that the agent monitors the project for changes to the chart. - Use the [GitOps configuration reference](#helm-configuration-reference) for guidance. - -## Helm chart with GitOps workflow - -To update a Kubernetes cluster by using Helm charts: - -1. Ensure you have a working Kubernetes cluster. -1. In a GitLab project: - - Store your Helm charts. - - [Register and install the GitLab agent](../install/index.md). -1. Update the agent configuration file so that the agent monitors the project for changes to the chart. - Use the [configuration reference](#helm-configuration-reference) for guidance. - -Any time you commit updates to your chart repository, the agent applies the chart in the cluster. - -## Helm configuration reference - -The following snippet shows an example of the possible keys and values for the GitOps section of an [agent configuration file](../install/index.md#create-an-agent-configuration-file) (`config.yaml`). - -```yaml -gitops: - charts: - - release_name: my-application-release - source: - project: - id: my-group/my-project-with-chart - ref: - branch: production - path: dir-in-project/with/charts - namespace: my-ns - max_history: 1 - values: - - inline: - someKey: example value -``` - -| Keyword | Description | -|--|--| -| `charts` | List of charts you want to be applied in your cluster. Charts are applied concurrently. | -| `release_name` | Required. Name of the release to use when applying the chart. | -| `values` | Optional. [Custom values](#custom-values) for the release. An array of objects. Only supports `inline` values. | -| `namespace` | Optional. Namespace to use when applying the chart. Defaults to `default`. | -| `max_history` | Optional. Maximum number of release [revisions to store in the cluster](https://helm.sh/docs/helm/helm_history/). | -| `source` | Required. From where the chart should get installed. Only supports project sources. | -| `source.project.id` | Required. ID of the project where Helm chart is committed. Authentication is not supported. | -| `source.project.ref` | Optional. Git reference in the configured Git repository to fetch the Chart from. If not specified or empty, the default branch is used. If specified, it must contain either `branch`, `tag`, or `commit`. | -| `source.project.ref.branch` | Branch name in the configured Git repository to fetch the Chart from. | -| `source.project.ref.tag` | Tag name in the configured Git repository to fetch the Chart from. | -| `source.project.ref.commit` | Commit SHA in the configured Git repository to fetch the Chart from. | -| `source.project.path` | Optional. Path of the chart in the project repository. Root of the repository is used by default. Should be the directory with the `Chart.yaml` file. | - -## Custom values - -> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/766) in GitLab 15.6. Requires both GitLab and the installed agent to be version 15.6 or later. - -To customize the values for a release, set the `values` key. It must be -an array of objects. Each object must have exactly one top-level key that describes -where the values come from. The supported top-level keys are: - -- `inline`: Specify the values inline in YAML format, similar to a Helm values - file. - -When installing a chart with custom values: - -- Custom values get merged on top of the chart's default `values.yaml` file. -- Values from subsequent entries in the `values` array overwrite values from - previous entries. - -Example: - -```yaml -gitops: - charts: - - release_name: some-release - values: - - inline: - someKey: example value - # ... -``` - -## Automatic drift remediation - -Drift happens when the current configuration of an infrastructure resource differs from its desired configuration. -Typically, drift is caused by manually editing resources directly, rather than by editing the code that describes the desired state. Minimizing the risk of drift helps to ensure configuration consistency and successful operations. - -In GitLab, the agent for Kubernetes regularly compares the desired state from the chart source with -the actual state from the Kubernetes cluster. Deviations from the desired state are fixed at every check. These checks -happen automatically every 5 minutes. They are not configurable. - -## Example repository layout - -```plaintext -/my-chart - ├── templates - | └── ... - ├── charts - | └── ... - ├── Chart.yaml - ├── Chart.lock - ├── values.yaml - ├── values.schema.json - └── some-file-used-in-chart.txt -``` - -## Known issues - -The following are known issues: - -- Your chart must be in a GitLab project. The project must be an agent configuration project or a public - project. This known issue also exists for manifest-based GitOps and is tracked in - [this epic](https://gitlab.com/groups/gitlab-org/-/epics/7704). -- Values for the chart must be in a `values.yaml` file. This file must be with the chart, - in the same project and path. -- Because of drift detection and remediation, the release history stored in the cluster is not useful. - A new release is created every five minutes and the oldest release is discarded. - Eventually history consists only of the same information. - View [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/372023) for details. - -## Troubleshooting - -### Agent cannot find values for the chart - -Make sure values are in `values.yaml` and in the same directory as the `Chart.yaml` file. -The filename must be lowercase, with `.yaml` extension (not `.yml`). +<!-- This redirect file can be deleted after 2023-10-22. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md index 6e1b7da9c6c..a9590f34183 100644 --- a/doc/user/clusters/agent/gitops/secrets_management.md +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -4,7 +4,11 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Managing Kubernetes secrets in a GitOps workflow +# Managing Kubernetes secrets in a GitOps workflow (deprecated) + +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/406545) in GitLab 16.2. +To manage cluster resources with GitOps, you should use the [Flux integration](../../../clusters/agent/gitops.md). You should never store Kubernetes secrets in unencrypted form in a `git` repository. If you use a GitOps workflow, you can follow these steps to securely manage your secrets. diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 8b1a55bc7bd..a0d6e0d415d 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -34,7 +34,7 @@ You can choose from two primary workflows. The GitOps workflow is recommended. ### GitOps workflow -You should use Flux for GitOps. To get started, see the GitLab [Flux documentation](../../../user/clusters/agent/gitops/flux.md). +You should use Flux for GitOps. To get started, see [Tutorial: Set up Flux for GitOps](gitops/flux_tutorial.md) ### GitLab CI/CD workflow @@ -55,8 +55,11 @@ This workflow has a weaker security model and is not recommended for production ## Supported Kubernetes versions for GitLab features GitLab supports the following Kubernetes versions. If you want to run -GitLab in a Kubernetes cluster, you might need a different version of Kubernetes. -GitLab in a Kubernetes cluster, you might need [a different version of Kubernetes](https://docs.gitlab.com/charts/installation/cloud/index.html). +GitLab in a Kubernetes cluster, you might need a different version of Kubernetes: + +- For the [Helm Chart](https://docs.gitlab.com/charts/installation/cloud/index.html). +- For [GitLab Operator](https://docs.gitlab.com/operator/installation.html#kubernetes). + You can upgrade your Kubernetes version to a supported version at any time: diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 76fcc73504c..c847eaff13f 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -42,7 +42,7 @@ To install the agent in your cluster: For configuration settings, the agent uses a YAML file in the GitLab project. You must create this file if: -- You use [a GitOps workflow](../gitops.md#gitops-workflow-steps). +- You use [a GitOps workflow](../gitops/agent.md#gitops-workflow-steps). - You use [a GitLab CI/CD workflow](../ci_cd_workflow.md#gitlab-cicd-workflow-steps) and want to authorize a different project to use the agent. - You [allow specific project or group members to access Kubernetes](../user_access.md). @@ -181,7 +181,7 @@ GitLab also provides a [KPT package for the agent](https://gitlab.com/gitlab-org To configure your agent, add content to the `config.yaml` file: -- For a GitOps workflow, [view the configuration reference](../gitops.md#gitops-configuration-reference). +- For a GitOps workflow, [view the configuration reference](../gitops/agent.md#gitops-configuration-reference). - For a GitLab CI/CD workflow, [authorize the agent to access your projects](../ci_cd_workflow.md#authorize-the-agent). Then [add `kubectl` commands to your `.gitlab-ci.yml` file](../ci_cd_workflow.md#update-your-gitlab-ciyml-file-to-run-kubectl-commands). @@ -238,6 +238,10 @@ helm upgrade gitlab-agent gitlab/gitlab-agent \ --set image.tag=v14.9.1 ``` +The Helm chart is updated separately from the agent for Kubernetes, and might occasionally lag behind the latest version of the agent. If you run `helm repo update` and don't specify an image tag, your agent runs the version specified in the chart. + +To use the latest release of the agent for Kubernetes, set the image tag to match the most recent agent image. + ## Uninstall the agent If you [installed the agent with Helm](#install-the-agent-with-helm), then you can also uninstall with Helm. For example, if the release and namespace are both called `gitlab-agent`, then you can uninstall the agent using the following command: diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index 9c0733d66b7..eddc6ef3e16 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -11,7 +11,7 @@ When you are using the GitLab agent for Kubernetes, you might experience issues You can start by viewing the service logs: ```shell -kubectl logs -f -l=app=gitlab-agent -n gitlab-agent +kubectl logs -f -l=app.kubernetes.io/name=gitlab-agent -n gitlab-agent ``` If you are a GitLab administrator, you can also view the [GitLab agent server logs](../../../administration/clusters/kas.md#troubleshooting). @@ -108,8 +108,7 @@ certificate authority that is unknown to the agent. To fix this issue, you can present the CA certificate file to the agent by [customizing the Helm installation](install/index.md#customize-the-helm-installation). -Add `--set config.caCert="$(cat ~/path/to/ca.crt)"` to the `helm install` command. Make sure to replace `~/path/to/ca.crt` -with the path to your internal CA's certificate file. The file should be a valid PEM or DER-encoded certificate. +Add `--set-file config.caCert=my-custom-ca.pem` to the `helm install` command. The file should be a valid PEM or DER-encoded certificate. When you deploy `agentk` with a set `config.caCert` value, the certificate is added to `configmap` and the certificate file is mounted in `/etc/ssl/certs`. diff --git a/doc/user/clusters/agent/user_access.md b/doc/user/clusters/agent/user_access.md index be3f88d072e..7e04091c940 100644 --- a/doc/user/clusters/agent/user_access.md +++ b/doc/user/clusters/agent/user_access.md @@ -6,7 +6,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Grant users Kubernetes access (Beta) **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../../policy/experiment-beta-support.md#beta). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390769) in GitLab 16.1, with [flags](../../../administration/feature_flags.md) named `environment_settings_to_graphql`, `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents`. This feature is in [Beta](../../../policy/experiment-beta-support.md#beta). +> - Feature flag `environment_settings_to_graphql` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124177) in GitLab 16.2. +> - Feature flags `kas_user_access`, `kas_user_access_project`, and `expose_authorized_cluster_agents` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125835) in GitLab 16.2. As an administrator of Kubernetes clusters in an organization, you can grant Kubernetes access to members of a specific project or group. diff --git a/doc/user/clusters/agent/work_with_agent.md b/doc/user/clusters/agent/work_with_agent.md index 8bf9ac7cf06..e8fb399d1b0 100644 --- a/doc/user/clusters/agent/work_with_agent.md +++ b/doc/user/clusters/agent/work_with_agent.md @@ -107,7 +107,8 @@ For more information about debugging, see [troubleshooting documentation](troubl > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327152) in GitLab 14.9. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/336641) in GitLab 14.10, the agent token can be revoked from the UI. -> - Two-token limit [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361030) in GitLab 16.1. +> - Two-token limit [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/361030/) in GitLab 16.1 with a [flag](../../../administration/feature_flags.md) named `cluster_agents_limit_tokens_created`. +> - Two-token limit [generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/412399) in GitLab 16.2. Feature flag `cluster_agents_limit_tokens_created` removed. An agent can have only two active tokens at one time. diff --git a/doc/user/clusters/create/index.md b/doc/user/clusters/create/index.md index 57526394301..8dfeafeb48e 100644 --- a/doc/user/clusters/create/index.md +++ b/doc/user/clusters/create/index.md @@ -11,4 +11,5 @@ You connect the clusters to GitLab by using the agent for Kubernetes. - [Create a cluster on Google GKE](../../infrastructure/clusters/connect/new_gke_cluster.md) - [Create a cluster on Amazon EKS](../../infrastructure/clusters/connect/new_eks_cluster.md) +- [Create a cluster on Azure AKS](../../infrastructure/clusters/connect/new_aks_cluster.md) - [Create a cluster on Civo](../../infrastructure/clusters/connect/new_civo_cluster.md) diff --git a/doc/user/clusters/environments.md b/doc/user/clusters/environments.md index c6a61f58974..fa56dc320be 100644 --- a/doc/user/clusters/environments.md +++ b/doc/user/clusters/environments.md @@ -15,7 +15,7 @@ WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. Cluster environments provide a consolidated view of which CI [environments](../../ci/environments/index.md) are deployed to the Kubernetes cluster and it: @@ -48,7 +48,7 @@ To: After you have successful deployments to your group-level or instance-level cluster: -1. Navigate to your group's **Kubernetes** page. +1. Go to your group's **Kubernetes** page. 1. Select the **Environments** tab. Only successful deployments to the cluster are included in this page. diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index ff6cf6bb1a8..9f769c6535c 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -16,7 +16,7 @@ To manage cluster applications, use the [GitLab agent](agent/index.md) with the [Cluster Management Project Template](management_project_template.md). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. A project can be designated as the management project for a cluster. A management project can be used to run deployment jobs with @@ -58,7 +58,7 @@ To associate a cluster management project with your cluster: 1. Navigate to the appropriate configuration page. For a: - [Project-level cluster](../project/clusters/index.md), go to your project's - **Infrastructure > Kubernetes clusters** page. + **Operate > Kubernetes clusters** page. - [Group-level cluster](../group/clusters/index.md), go to your group's **Kubernetes** page. - [Instance-level cluster](../instance/clusters/index.md): diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index b7a68317fba..238cf10cba9 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -695,7 +695,7 @@ Additional configuration may be needed for connecting to private registries for: Prior to GitLab 13.3, offline environments required an exact name match for [project policies](../license_approval_policies.md). In GitLab 13.3 and later, GitLab matches the name of [project policies](../license_approval_policies.md) -with identifiers from the [SPDX license list](https://spdx.org/licenses/). +with license names from the [SPDX license list](https://spdx.org/licenses/). A local copy of the SPDX license list is distributed with the GitLab instance. If needed, the GitLab instance's administrator can manually update it with a [Rake task](../../../raketasks/spdx.md). @@ -705,9 +705,9 @@ We recommend that you use the most recent version of all containers, and the mos ## Troubleshooting -### ASDF_PYTHON_VERSION does not automatically install the version +### `ASDF_PYTHON_VERSION` does not automatically install the version -Defining a non-latest Python version in ASDF_PYTHON_VERSION [doesn't have it automatically installed](https://gitlab.com/gitlab-org/gitlab/-/issues/325604). If your project requires a non-latest version of Python: +Defining a non-latest Python version in `ASDF_PYTHON_VERSION` [doesn't have it automatically installed](https://gitlab.com/gitlab-org/gitlab/-/issues/325604). If your project requires a non-latest version of Python: 1. Define the required version by setting the `ASDF_PYTHON_VERSION` CI/CD variable. 1. Pass a custom script to the `SETUP_CMD` CI/CD variable to install the required version and dependencies. diff --git a/doc/user/compliance/license_list.md b/doc/user/compliance/license_list.md index 7f55ba50c5b..deec4e28911 100644 --- a/doc/user/compliance/license_list.md +++ b/doc/user/compliance/license_list.md @@ -24,7 +24,7 @@ Alternatively, licenses will also appear under the license list when using our d 1. Your project must use at least one of the [supported languages and package managers](license_compliance/index.md#supported-languages-and-package-managers). -When everything is configured, on the left sidebar, select **Security & Compliance > License Compliance**. +When everything is configured, on the left sidebar, select **Secure > License compliance**. The licenses are displayed, where: diff --git a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md index 22defd593cd..1fbe67a62b2 100644 --- a/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md +++ b/doc/user/compliance/license_scanning_of_cyclonedx_files/index.md @@ -23,7 +23,7 @@ Licenses not in the SPDX list are reported as "Unknown". License information can Prerequisites: -- On GitLab self-managed only, enable [Synchronization with the GitLab License Database](../../admin_area/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in Admin Area for the GitLab instance. On GitLab SaaS this step has already been completed. +- On GitLab self-managed only, enable [Synchronization with the GitLab License Database](../../../administration/settings/security_and_compliance.md#choose-package-registry-metadata-to-sync) in Admin Area for the GitLab instance. On GitLab SaaS this step has already been completed. - Enable [Dependency Scanning](../../application_security/dependency_scanning/index.md#configuration) and ensure that its prerequisites are met. diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 03b48fd42b6..2e26b67170c 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -56,7 +56,7 @@ in a different color. > [Flag](../../administration/feature_flags.md) named `disable_all_mention` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110586) in GitLab 16.1. Disabled by default. [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/18442). FLAG: -On self-managed GitLab, by default this flag is not enabled. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) +On self-managed GitLab, by default this flag is not enabled. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `disable_all_mention`. On GitLab.com, this flag is enabled. @@ -103,7 +103,7 @@ To add a commit diff comment: The comment is displayed on the merge request's **Overview** tab. -The comment is not displayed on your project's **Repository > Commits** page. +The comment is not displayed on your project's **Code > Commits** page. NOTE: When your comment contains a reference to a commit included in the merge request, diff --git a/doc/user/enterprise_user/index.md b/doc/user/enterprise_user/index.md index 36f698daf81..27fff4bf8e3 100644 --- a/doc/user/enterprise_user/index.md +++ b/doc/user/enterprise_user/index.md @@ -10,7 +10,7 @@ type: reference Enterprise users have user accounts that are administered by an organization that has purchased a [GitLab subscription](../../subscriptions/index.md). -Enterprise users are identified by the [**Enterprise** badge](../project/badges.md) +Enterprise users are identified by the **Enterprise** badge next to their names on the [Members list](../group/index.md#filter-and-sort-members-in-a-group). ## Provision an enterprise user @@ -47,17 +47,33 @@ Prerequisites: - A project in the group. - You must have the Owner role in the top-level group. -Setting up a verified domain is similar to [setting up a custom domain on GitLab Pages](../project/pages/custom_domains_ssl_tls_certification/index.md). However, you must: +Domain verification applies at the top-level group and to all subgroups and projects +nested under that top-level parent group. -- Link the domain to a single project. -- Configure the `TXT` only in the DNS record to verify the domain's ownership. +You cannot verify a domain for more than one group. For example, if a group named +'group1' has a verified domain named 'domain1', you cannot also verify 'domain1' +for a different group named 'group2'. -Domain verification is tied to the project you choose. A project is required because domain verification reuses the GitLab Pages verification feature, which requires a project. Domain verification applies at the top-level group and to all subgroups and projects nested under that top-level parent group. -A member in the chosen project with [at least the Maintainer role](../permissions.md#project-members-permissions) can modify or remove the domain verification. -If needed, you can create a new project to set up domain verification directly under your top-level group. This limits the ability to modify the domain verification to members with at least the Maintainer role. -For more information on group-level domain verification, see [epic 5299](https://gitlab.com/groups/gitlab-org/-/epics/5299). +Setting up a verified domain is similar to [setting up a custom domain on GitLab Pages](../project/pages/custom_domains_ssl_tls_certification/index.md). However, you: + +- Do not need to have a GitLab Pages website. +- Must link the domain to a single project, despite domain verification applying + at the top-level group and to all nested subgroups and projects, because domain + verification: + - Is tied to the project you choose. + - Reuses the GitLab Pages custom domain verification feature, which requires a project. +- Must configure the `TXT` only in the DNS record to verify the domain's ownership. + +In addition to appearing in the top-level group Domain Verification list, the +domain will also appear in the chosen project. A member in this project with +[at least the Maintainer role](../permissions.md#project-members-permissions) +can modify or remove the domain verification. -Steps: +If needed, you can create a new project to set up domain verification directly +under your top-level group. This limits the ability to modify the domain verification +to members with at least the Maintainer role. + +For more information on group-level domain verification, see [epic 5299](https://gitlab.com/groups/gitlab-org/-/epics/5299). #### 1. Add a custom domain for the matching email domain @@ -96,21 +112,13 @@ After you have added all the DNS records:  WARNING: -For GitLab instances with domain verification enabled, -if the domain cannot be verified for 7 days, that domain is removed -from the GitLab project. +For GitLab instances with domain verification enabled, if the domain cannot be verified for 7 days, that domain is removed from the GitLab project. > **Notes:** > -> - Domain verification is **required for GitLab.com users**; - for GitLab self-managed instances, your GitLab administrator has the option - to [disabled custom domain verification](../../administration/pages/index.md#custom-domain-verification). -> - [DNS propagation may take some time (up to 24 hours)](https://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/complete-guide-to-dns-records/), - although it's usually a matter of minutes to complete. Until it does, verification - fails, and attempts to visit your domain result in a 404. -> - Once your domain has been verified, leave the verification record - in place. Your domain is periodically reverified, and may be - disabled if the record is removed. +> - Domain verification is **required for GitLab.com users** to be marked as enterprise users. +> - [DNS propagation can take up to 24 hours](https://www.inmotionhosting.com/support/domain-names/dns-nameserver-changes/complete-guide-to-dns-records/), although it's usually a couple of minutes to complete. Until it completes, the domain shows as unverified. +> - Once your domain has been verified, leave the verification record in place. Your domain is periodically reverified, and may be disabled if the record is removed. > - A valid certificate is not required for domain verification. ### View domains in group diff --git a/doc/user/free_user_limit.md b/doc/user/free_user_limit.md index 410bdc4b5f5..0af5c76ade8 100644 --- a/doc/user/free_user_limit.md +++ b/doc/user/free_user_limit.md @@ -7,9 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Free user limit **(FREE SAAS)** A five-user limit applies to newly created top-level namespaces with -private visibility on GitLab SaaS. For existing namespaces, this limit -is being rolled out gradually. Impacted users are notified in -GitLab.com at least 60 days before the limit is applied. +private visibility on GitLab SaaS. For existing namespaces created before December 28, 2022, the limit was applied on June 13, 2023. When the five-user limit is applied, top-level private namespaces exceeding the user limit are placed in a read-only state. These diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 82d0bfb035a..04858d8ac34 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -26,15 +26,23 @@ GitLab.com uses the default [SSH key restrictions](../../security/ssh_keys_restr ## SSH host keys fingerprints -Below are the fingerprints for SSH host keys on GitLab.com. The first time you -connect to a GitLab.com repository, one of these keys is displayed in the output. +Go to the current instance configuration to see the SSH host key fingerprints on +GitLab.com. + +1. Sign in to GitLab. +1. On the left sidebar, select **Help** (**{question-o}**) > **Help**. +1. On the Help page, select **Check the current instance configuration**. + +In the instance configuration, you see the **SSH host key fingerprints**: | Algorithm | MD5 (deprecated) | SHA256 | |------------------|------------------|---------| +| ECDSA | `f1:d0:fb:46:73:7a:70:92:5a:ab:5d:ef:43:e2:1c:35` | `HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw` | | ED25519 | `2e:65:6a:c8:cf:bf:b2:8b:9a:bd:6d:9f:11:5c:12:16` | `eUXGGm1YGsMAS7vkcx6JOJdOGHPem5gQp4taiCfCLB8` | | RSA | `b6:03:0e:39:97:9e:d0:e7:24:ce:a3:77:3e:01:42:09` | `ROQFvPThGrW4RuWLoL9tq9I9zJ42fK4XywyRtbOz/EQ` | -| DSA (deprecated) | `7a:47:81:3a:ee:89:89:64:33:ca:44:52:3d:30:d4:87` | `p8vZBUOR0XQz6sYiaWSMLmh0t9i8srqYKool/Xfdfqw` | -| ECDSA | `f1:d0:fb:46:73:7a:70:92:5a:ab:5d:ef:43:e2:1c:35` | `HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw` | + +The first time you connect to a GitLab.com repository, one of these keys is +displayed in the output. ## SSH `known_hosts` entries @@ -61,11 +69,11 @@ and has its own dedicated IP addresses: The IP addresses for `mg.gitlab.com` are subject to change at any time. -### Service Desk custom mailbox +### Service Desk alias email address On GitLab.com, there's a mailbox configured for Service Desk with the email address: `contact-project+%{key}@incoming.gitlab.com`. To use this mailbox, configure the -[custom suffix](../project/service_desk.md#configure-a-custom-email-address-suffix) in project +[custom suffix](../project/service_desk.md#configure-a-suffix-for-service-desk-alias-email) in project settings. ## Backups @@ -156,8 +164,8 @@ the related documentation. | Setting | GitLab.com | Default (self-managed) | |----------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------|------------------------| -| Artifacts maximum size (compressed) | 1 GB | See [Maximum artifacts size](../../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size). | -| Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | See [Default artifacts expiration](../admin_area/settings/continuous_integration.md#default-artifacts-expiration). | +| Artifacts maximum size (compressed) | 1 GB | See [Maximum artifacts size](../../administration/settings/continuous_integration.md#maximum-artifacts-size). | +| Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | See [Default artifacts expiration](../../administration/settings/continuous_integration.md#default-artifacts-expiration). | | Scheduled Pipeline Cron | `*/5 * * * *` | See [Pipeline schedules advanced configuration](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency). | | Maximum jobs in active pipelines | `500` for Free tier, `1000` for all trial tiers, `20000` for Premium, and `100000` for Ultimate. | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines). | | Maximum CI/CD subscriptions to a project | `2` | See [Number of CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project). | @@ -191,11 +199,11 @@ varies by format: ## Account and limit settings GitLab.com has the following account limits enabled. If a setting is not listed, -the default value [is the same as for self-managed instances](../admin_area/settings/account_and_limit_settings.md): +the default value [is the same as for self-managed instances](../../administration/settings/account_and_limit_settings.md): | Setting | GitLab.com default | |-------------------------------|--------------------| -| [Repository size including LFS](../admin_area/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | +| [Repository size including LFS](../../administration/settings/account_and_limit_settings.md#repository-size-limit) | 10 GB | | [Maximum import size](../project/settings/import_export.md#import-a-project-and-its-data) | 5 GB | | Maximum attachment size | 100 MB | @@ -216,7 +224,7 @@ The import sources that are available by default depend on which GitLab you use: - GitLab.com: all available import sources are enabled by default. - GitLab self-managed: no import sources are enabled by default and must be - [enabled](../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). + [enabled](../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources). | Import source | GitLab.com default | GitLab self-managed default | |:----------------------------------------------------------------------------------------------------|:-----------------------|:----------------------------| @@ -343,7 +351,7 @@ after the limits change in January, 2021: More details are available on the rate limits for [protected paths](#protected-paths-throttle) and -[raw endpoints](../../user/admin_area/settings/rate_limits_on_raw_endpoints.md). +[raw endpoints](../../administration/settings/rate_limits_on_raw_endpoints.md). GitLab can rate-limit requests at several layers. The rate limits listed here are configured in the application. These limits are the most @@ -355,8 +363,8 @@ for GitLab.com, see For information on rate limiting responses, see: -- [List of headers on responses to blocked requests](../admin_area/settings/user_and_ip_rate_limits.md#response-headers). -- [Customizable response text](../admin_area/settings/user_and_ip_rate_limits.md#use-a-custom-rate-limit-response). +- [List of headers on responses to blocked requests](../../administration/settings/user_and_ip_rate_limits.md#response-headers). +- [Customizable response text](../../administration/settings/user_and_ip_rate_limits.md#use-a-custom-rate-limit-response). ### Protected paths throttle @@ -366,7 +374,7 @@ paths that exceed 10 requests per **minute** per IP address. See the source below for which paths are protected. This includes user creation, user confirmation, user sign in, and password reset. -[User and IP rate limits](../admin_area/settings/user_and_ip_rate_limits.md#response-headers) +[User and IP rate limits](../../administration/settings/user_and_ip_rate_limits.md#response-headers) includes a list of the headers responded to blocked requests. See [Protected Paths](../admin_area/settings/protected_paths.md) for more details. diff --git a/doc/user/group/access_and_permissions.md b/doc/user/group/access_and_permissions.md index b7acded6720..8bebaae003c 100644 --- a/doc/user/group/access_and_permissions.md +++ b/doc/user/group/access_and_permissions.md @@ -41,7 +41,7 @@ The group's new subgroups have push rules set for them based on either: > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/365357) in GitLab 16.0. You can set the permitted protocols used to access a group's repositories to either SSH, HTTPS, or both. This setting -is disabled when the [instance setting](../admin_area/settings/visibility_and_access_controls.md#configure-enabled-git-access-protocols) is +is disabled when the [instance setting](../../administration/settings/visibility_and_access_controls.md#configure-enabled-git-access-protocols) is configured by an administrator. To change the permitted Git access protocols for a group: @@ -63,11 +63,11 @@ address. This top-level group setting applies to: - The GitLab UI, including subgroups, projects, and issues. It does not apply to GitLab Pages. - [In GitLab 12.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/12874), the API. - In self-managed installations of GitLab 15.1 and later, you can also configure -[globally-allowed IP address ranges](../admin_area/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges) +[globally-allowed IP address ranges](../../administration/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges) at the group level. Administrators can combine restricted access by IP address with -[globally-allowed IP addresses](../admin_area/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges). +[globally-allowed IP addresses](../../administration/settings/visibility_and_access_controls.md#configure-globally-allowed-ip-address-ranges). To restrict group access by IP address: @@ -253,6 +253,8 @@ For more information on the administration of LDAP and group sync, refer to the NOTE: When you add LDAP synchronization, if an LDAP user is a group member and they are not part of the LDAP group, they are removed from the group. +You can use a workaround to [manage project access through LDAP groups](../project/settings/index.md#manage-project-access-through-ldap-groups). + ### Create group links via CN **(PREMIUM SELF)** To create group links via CN: diff --git a/doc/user/group/compliance_frameworks.md b/doc/user/group/compliance_frameworks.md index 47764b0c915..267cdbbebd3 100644 --- a/doc/user/group/compliance_frameworks.md +++ b/doc/user/group/compliance_frameworks.md @@ -11,8 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can create a compliance framework that is a label to identify that your project has certain compliance requirements or needs additional oversight. The label can optionally enforce -[compliance pipeline configuration](#compliance-pipelines) to the projects on which it is -[applied](../project/settings/index.md#add-a-compliance-framework-to-a-project). +[compliance pipeline configuration](#compliance-pipelines) to the projects on which it is applied. Compliance frameworks are created on top-level groups. Group owners can create, edit, and delete compliance frameworks: @@ -24,6 +23,33 @@ Compliance frameworks are created on top-level groups. Group owners can create, Subgroups and projects have access to all compliance frameworks created on their top-level group. However, compliance frameworks cannot be created, edited, or deleted at the subgroup or project level. Project owners can choose a framework to apply to their projects. +## Add a compliance framework to a project + +Prerequisite: + +- The group to which the project belongs must have a compliance framework. + +To assign a compliance framework to a project: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings** > **General**. +1. Expand **Compliance frameworks**. +1. Select a compliance framework. +1. Select **Save changes**. + +NOTE: +Frameworks cannot be added to projects in personal namespaces. + +### GraphQL API + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/333249) in GitLab 14.2. + +You can use the [GraphQL API](../../api/graphql/reference/index.md#mutationprojectsetcomplianceframework) to add a +compliance framework to a project. + +If you create compliance frameworks on subgroups with GraphQL, the framework is created on the root ancestor if the user +has the correct permissions. The GitLab UI presents a read-only view to discourage this behavior. + ## Default compliance frameworks > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/375036) in GitLab 15.6. @@ -104,6 +130,10 @@ However, the compliance pipeline configuration can reference the `.gitlab-ci.yml - Jobs and variables defined in the compliance pipeline can't be changed by variables in the labeled project's `.gitlab-ci.yml` file. +NOTE: +Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/414004), project pipelines must be included first at the top of compliance pipeline configuration +to prevent projects overriding settings downstream. + For more information, see: - [Example configuration](#example-configuration) for help configuring a compliance pipeline that runs jobs from @@ -151,6 +181,13 @@ The following example `.compliance-gitlab-ci.yml` includes the `include` keyword configuration is also executed. ```yaml +include: # Execute individual project's configuration (if project contains .gitlab-ci.yml) + - project: '$CI_PROJECT_PATH' + file: '$CI_CONFIG_PATH' + ref: '$CI_COMMIT_SHA' # Must be defined or MR pipelines always use the use default branch + rules: + - if: $CI_PROJECT_PATH != "my-group/project-1" # Must be the hardcoded path to the project that hosts this configuration. + # Allows compliance team to control the ordering and interweaving of stages/jobs. # Stages without jobs defined will remain hidden. stages: @@ -210,13 +247,6 @@ audit trail: - echo "running $FOO" after_script: - "# No after scripts." - -include: # Execute individual project's configuration (if project contains .gitlab-ci.yml) - - project: '$CI_PROJECT_PATH' - file: '$CI_CONFIG_PATH' - ref: '$CI_COMMIT_SHA' # Must be defined or MR pipelines always use the use default branch - rules: - - if: $CI_PROJECT_PATH != "my-group/project-1" # Must be the hardcoded path to the project that hosts this configuration. ``` The `rules` configuration in the `include` definition avoids circular inclusion in case the compliance pipeline must be able to run in the host project itself. @@ -334,20 +364,6 @@ This alternative ensures the compliance pipeline does not re-start the parent pi ## Troubleshooting -### Cannot remove compliance framework from a project - -Because of a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/390626), if you move a project, its compliance -framework becomes orphaned and can't be removed. To manually remove a compliance framework from a project, run the -following GraphQL mutation with your project's ID: - -```graphql -mutation { - projectSetComplianceFramework(input: {projectId: "gid://gitlab/Project/1234567", complianceFrameworkId: null}) { - errors - } -} -``` - ### Compliance jobs are overwritten by target repository If you use the `extends` statement in a compliance pipeline configuration, compliance jobs are overwritten by the target repository job. For example, diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md index cbab83dd61e..6bd57079c67 100644 --- a/doc/user/group/custom_project_templates.md +++ b/doc/user/group/custom_project_templates.md @@ -17,7 +17,7 @@ You can [customize the list](../project/index.md#create-a-project) of available that all projects in your group have the same list. To do this, you populate a subgroup with the projects you want to use as templates. -You can also configure [custom templates for the instance](../admin_area/custom_project_templates.md). +You can also configure [custom templates for the instance](../../administration/custom_project_templates.md). ## Set up group-level project templates diff --git a/doc/user/group/epics/epic_boards.md b/doc/user/group/epics/epic_boards.md index 4a913c385a0..47bcd92f134 100644 --- a/doc/user/group/epics/epic_boards.md +++ b/doc/user/group/epics/epic_boards.md @@ -107,8 +107,8 @@ To remove a list from an epic board: 1. On the top of the list you want to remove, select the **List settings** icon (**{settings}**). The list settings sidebar opens on the right. -1. Select **Remove list**. A confirmation dialog appears. -1. Select **OK**. +1. Select **Remove list**. +1. On the confirmation dialog, select **OK**. ### Create an epic from an epic board diff --git a/doc/user/group/epics/manage_epics.md b/doc/user/group/epics/manage_epics.md index 9436ff317a7..8265540cc34 100644 --- a/doc/user/group/epics/manage_epics.md +++ b/doc/user/group/epics/manage_epics.md @@ -66,7 +66,7 @@ The parent epic's start date then reflects this change and propagates upwards to > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79940) in GitLab 14.9 [with a flag](../../../administration/feature_flags.md) named `epic_color_highlight`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `epic_color_highlight`. +On self-managed GitLab, by default this feature is not available. To make it available per group, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `epic_color_highlight`. On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. The feature is not ready for production use. @@ -161,9 +161,9 @@ Prerequisites: To close an epic, at the top of an epic, select **Close epic**. -<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> +<!-- Delete when the `move_close_into_dropdown` feature flag is removed --> If you don't see this action at the top of an epic, your project or instance might have -enabled a feature flag for [moved actions](../../project/merge_requests/index.md#move-sidebar-actions) +enabled a feature flag to [moved it in the actions menu](../../project/issues/managing_issues.md#move-the-close-button-into-the-actions-menu). You can also use the `/close` [quick action](../../project/quick_actions.md). @@ -183,6 +183,10 @@ To do so, either: - Use the `/reopen` [quick action](../../project/quick_actions.md). +<!-- Delete when the `move_close_into_dropdown` feature flag is removed --> +If you don't see this action at the top of an epic, your project or instance might have +enabled a feature flag to [moved it in the actions menu](../../project/issues/managing_issues.md#move-the-close-button-into-the-actions-menu). + ## Go to an epic from an issue If an issue belongs to an epic, you can go to the parent epic with the @@ -262,7 +266,7 @@ To filter: FLAG: On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `or_issuable_queries`. +To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `or_issuable_queries`. On GitLab.com, this feature is not available. The feature is not ready for production use. @@ -351,8 +355,8 @@ you might not have permission to. ### Add a new issue to an epic -You can add an existing issue to an epic, or create a new issue that's -automatically added to the epic. +Add an existing issue to an epic, or create a new issue that's automatically +added to the epic. #### Add an existing issue to an epic @@ -389,6 +393,12 @@ To add an existing issue to an epic: Creating an issue from an epic enables you to maintain focus on the broader context of the epic while dividing work into smaller parts. +You can create a new issue from an epic only in projects that are in the epic's group or one of its +descendant subgroups. +To create a new issue in a [project that was shared with the epic's group](../../project/members/share_project_with_groups.md), +first [create the issue directly in the project](../../project/issues/create_issues.md#from-a-project), and +then [add an existing issue to an epic](#add-an-existing-issue-to-an-epic). + Prerequisites: - You must have at least the Guest role for the issue's project and the epic's group. diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index 0b9a1552281..154eb467ece 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -30,7 +30,7 @@ If you migrate from GitLab.com to self-managed GitLab, an administrator can crea > - `bulk_import_projects` feature flag [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/339941) in GitLab 15.10. On self-managed GitLab, by default [migrating group items](#migrated-group-items) is not available. To show the -feature, ask an administrator to [enable it in application settings](../../admin_area/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer). +feature, an administrator can [enable it in application settings](../../../administration/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer). Migrating groups by direct transfer copies the groups from one place to another. You can: @@ -66,6 +66,8 @@ transfer. ### Limits +Hardcoded limits apply on migration by direct transfer. + | Limit | Description | |:------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | 6 | Maximum number of migrations permitted by a destination GitLab instance per minute per user. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/386452) in GitLab 15.9. | @@ -105,7 +107,7 @@ To migrate groups by direct transfer: - The network connection between instances or GitLab.com must support HTTPS. - Any firewalls must not block the connection between the source and destination GitLab instances. - Both GitLab instances must have group migration by direct transfer - [enabled in application settings](../../admin_area/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer) + [enabled in application settings](../../../administration/settings/visibility_and_access_controls.md#enable-migration-of-groups-and-projects-by-direct-transfer) by an instance administrator. - The source GitLab instance must be running GitLab 14.0 or later. - You must have a [personal access token](../../../user/profile/personal_access_tokens.md) for the source GitLab @@ -136,10 +138,10 @@ To ensure GitLab maps users and their contributions correctly: Create the group you want to import to and connect the source GitLab instance: 1. Create either: - - A new group. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. Then select **Import group**. + - A new group. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. Then select **Import group**. - A new subgroup. On existing group's page, either: - Select **New subgroup**. - - On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New subgroup**. Then select the **import an existing group** link. + - On the left sidebar, at the top, select **Create new** (**{plus}**) and **New subgroup**. Then select the **import an existing group** link. 1. Enter the URL of a GitLab instance running GitLab 14.0 or later. 1. Enter the [personal access token](../../../user/profile/personal_access_tokens.md) for your source GitLab instance. 1. Select **Connect instance**. @@ -176,7 +178,7 @@ You can view all groups migrated by you by direct transfer listed on the group i To view group import history: 1. Sign in to GitLab. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. 1. Select **Import group**. 1. In the upper-right corner, select **History**. 1. If there are any errors for a particular import, you can see them by selecting **Details**. @@ -208,10 +210,10 @@ Group items that are migrated to the destination GitLab instance include: | Boards | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18938) | | Board lists | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/24863) | | Epics <sup>1</sup> | [GitLab 13.7](https://gitlab.com/gitlab-org/gitlab/-/issues/250281) | -| Group labels | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/292429) | +| Group labels <sup>2</sup> | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/292429) | | Iterations | [GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/292428) | | Iteration cadences | [GitLab 15.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/96570) | -| Members <sup>2</sup> | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/299415) | +| Members <sup>3</sup> | [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/issues/299415) | | Group milestones | [GitLab 13.10](https://gitlab.com/gitlab-org/gitlab/-/issues/292427) | | Namespace settings | [GitLab 14.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85128) | | Release milestones | [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/339422) | @@ -222,6 +224,8 @@ Group items that are migrated to the destination GitLab instance include: associations [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/62074) in GitLab 13.12, state and state ID [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28203) in GitLab 13.7, and system note metadata [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63551) in GitLab 14.0. +1. Group Labels cannot retain any associated Label Priorities during import. These labels will need to be re-prioritized manually + once the relevant Project is migrated to the destination instance. 1. Group members are associated with the imported group if the user: - Already exists in the destination GitLab instance. - Has a public email in the source GitLab instance that matches a confirmed email in the destination GitLab instance. @@ -490,7 +494,7 @@ for your version of GitLab to check which items can be imported to the destinati Group items that are exported include: - Milestones -- Labels +- Group Labels (_without_ associated label priorities) - Boards and Board Lists - Badges - Subgroups (including all the aforementioned data) @@ -550,7 +554,7 @@ You can also export a group [using the API](../../../api/group_import_export.md) ### Import the group -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New subgroup**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New subgroup**. 1. Select the **import an existing group** link. 1. Enter your group name. 1. Accept or modify the associated group URL. diff --git a/doc/user/group/index.md b/doc/user/group/index.md index 322cefc71d9..7ff9648e41e 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -38,7 +38,7 @@ Like projects, a group can be configured to limit the visibility of it to: - All authenticated users. - Only explicit group members. -The restriction for [visibility levels](../admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels) +The restriction for [visibility levels](../../administration/settings/visibility_and_access_controls.md#restrict-visibility-levels) on the application setting level also applies to groups. If set to internal, the explore page is empty for anonymous users. The group page has a visibility level icon. @@ -67,7 +67,7 @@ This page shows groups that you are a member of: To create a group: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New group**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New group**. 1. Select **Create group**. 1. Enter a name for the group in **Group name**. For a list of words that cannot be used as group names, see [reserved names](../reserved_names.md). @@ -107,7 +107,7 @@ A group can also be removed from the groups dashboard: This action removes the group. It also adds a background job to delete all projects in the group. -In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on GitLab [Premium](https://about.gitlab.com/pricing/premium/) and [Ultimate](https://about.gitlab.com/pricing/ultimate/), this action adds a background job to mark a group for deletion. By default, the job schedules the deletion seven days in the future. You can modify this retention period through the [instance settings](../admin_area/settings/visibility_and_access_controls.md#deletion-protection). +In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/33257), on GitLab [Premium](https://about.gitlab.com/pricing/premium/) and [Ultimate](https://about.gitlab.com/pricing/ultimate/), this action adds a background job to mark a group for deletion. By default, the job schedules the deletion seven days in the future. You can modify this retention period through the [instance settings](../../administration/settings/visibility_and_access_controls.md#deletion-protection). In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/39504), if the user who sets up the deletion is removed from the group before the deletion happens, the job is cancelled, and the group is no longer scheduled for deletion. @@ -160,6 +160,30 @@ Any group owner can approve or decline the request. If you change your mind before your request is approved, select **Withdraw Access Request**. +## View group members + +To view a group's members: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Manage > Members**. + +A table displays the member's: + +- **Account** name and username +- **Source** of their [membership](../project/members/index.md#membership-types). + For transparency, GitLab displays all membership sources of group members. + Members who have multiple membership sources are displayed and counted as separate members. + For example, if a member has been added to the group both directly and through inheritance, + the member is displayed twice in the **Members** table, with different sources, + and is counted as two individual members of the group. +- [**Max role**](../project/members/index.md#which-roles-you-can-assign) in the group +- **Expiration** date of their group membership +- **Activity** related to their account + +NOTE: +The display of group members' **Source** might be inconsistent. +For more information, see [issue 414557](https://gitlab.com/gitlab-org/gitlab/-/issues/414557). + ## Filter and sort members in a group > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21727) in GitLab 12.6. @@ -187,7 +211,7 @@ In lists of group members, entries can display the following badges: ### Search a group -You can search for members by name, username, or email. +You can search for members by name, username, or [public email](../profile/index.md#set-your-public-email). 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. 1. Select **Manage > Members**. @@ -283,4 +307,4 @@ To change the role that can create projects under a group: 1. Select the desired option in the **Roles allowed to create projects** dropdown list. 1. Select **Save changes**. -To change this setting globally, see [Default project creation protection](../admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). +To change this setting globally, see [Default project creation protection](../../administration/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). diff --git a/doc/user/group/manage.md b/doc/user/group/manage.md index 284fb8b3d7c..b7bdf236ff7 100644 --- a/doc/user/group/manage.md +++ b/doc/user/group/manage.md @@ -131,7 +131,7 @@ After sharing the `Frontend` group with the `Engineering` group: - The **Groups** tab lists the `Engineering` group. - The **Groups** tab lists a group regardless of whether it is a public or private group. - All direct members of the `Engineering` group have access to the `Frontend` group. Direct members of `Engineering` that gain access to the `Frontend` group keep their same access level as in `Engineering`, but up to the maximum access level selected when sharing the group. Inherited members of the `Engineering` group do not gain access to the `Frontend` group. -- All direct members of the `Engineering` group count towards the billable members of the `Frontend` group. +- Direct members of the `Engineering` group who have the **Group Invite** badge next to their profile on the group's usage quota page count towards the billable members of the `Frontend` group. ## Remove a shared group @@ -338,7 +338,7 @@ To enable group file templates: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/372040) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) name `support_group_level_merge_checks_setting`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `support_group_level_merge_checks_setting`. On GitLab.com, this feature is not available. @@ -434,11 +434,8 @@ for the ability to set merge request approval rules for groups is tracked in WARNING: This feature is in [Beta](../../policy/experiment-beta-support.md#beta). -Due to high demand, this feature may have unscheduled downtime and Code Suggestions in IDEs may be delayed. -Code Suggestions may produce -[low-quality or incomplete suggestions](../project/repository/code_suggestions.md#model-accuracy-and-quality). Beta users should read about the [known limitations](../project/repository/code_suggestions.md#known-limitations). -We look forward to hearing your feedback. +We look forward to hearing your [feedback](../project/repository/code_suggestions.md#feedback). You can give all users in a group and its subgroups access to [Code Suggestions](../project/repository/code_suggestions.md). diff --git a/doc/user/group/moderate_users.md b/doc/user/group/moderate_users.md index 1dde36c5c70..85fdeeb25c7 100644 --- a/doc/user/group/moderate_users.md +++ b/doc/user/group/moderate_users.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/modelops/anti-abuse/team-tasks/-/issues/155) in GitLab 15.8. -This is the group-level documentation. For self-managed instances, see the [administration documentation](../admin_area/moderate_users.md). +This is the group-level documentation. For self-managed instances, see the [administration documentation](../../administration/moderate_users.md). A group Owner can moderate user access by banning and unbanning users. diff --git a/doc/user/group/reporting/git_abuse_rate_limit.md b/doc/user/group/reporting/git_abuse_rate_limit.md index ee8ac50f021..cde19531ed3 100644 --- a/doc/user/group/reporting/git_abuse_rate_limit.md +++ b/doc/user/group/reporting/git_abuse_rate_limit.md @@ -9,11 +9,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8066) in GitLab 15.2 [with a flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. On GitLab.com, this feature is available. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `limit_unique_project_downloads_per_namespace_user`. On GitLab.com, this feature is available. This is the group-level documentation. For self-managed instances, see the [administration documentation](../../admin_area/reporting/git_abuse_rate_limit.md). -Git abuse rate limiting is a feature to automatically ban users who download, clone, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. +Git abuse rate limiting is a feature to automatically ban users who download, clone, pull, fetch, or fork more than a specified number of repositories of a group in a given time frame. Banned users cannot access the top-level group or any of its non-public subgroups via HTTP or SSH. The rate limit also applies to users who authenticate with a [personal](../../../user/profile/personal_access_tokens.md) or [group access token](../../../user/group/settings/group_access_tokens.md). Access to unrelated groups is unaffected. Git abuse rate limiting does not apply to top-level group owners, [deploy tokens](../../../user/project/deploy_tokens/index.md), or [deploy keys](../../../user/project/deploy_keys/index.md). diff --git a/doc/user/group/repositories_analytics/index.md b/doc/user/group/repositories_analytics/index.md index e5ca41accfc..6f5f4905102 100644 --- a/doc/user/group/repositories_analytics/index.md +++ b/doc/user/group/repositories_analytics/index.md @@ -18,7 +18,7 @@ It is similar to [repository analytics for projects](../../analytics/repository_ > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263478) in GitLab 13.7. -The **Analytics > Repositories** group page displays the overall test coverage of all your projects in your group. +The **Analyze > Repository analytics** group page displays the overall test coverage of all your projects in your group. In the **Overall activity** section, you can see: - The number of projects with coverage reports. @@ -29,7 +29,7 @@ In the **Overall activity** section, you can see: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215140) in GitLab 13.9. -The **Analytics > Repositories** group page displays the average test coverage of all your projects in your group in a graph for the last 30 days. +The **Analyze > Repository analytics** group page displays the average test coverage of all your projects in your group in a graph for the last 30 days. ## Latest project test coverage list diff --git a/doc/user/group/saml_sso/example_saml_config.md b/doc/user/group/saml_sso/example_saml_config.md index f2db36e80b1..f6eb4ad539c 100644 --- a/doc/user/group/saml_sso/example_saml_config.md +++ b/doc/user/group/saml_sso/example_saml_config.md @@ -55,8 +55,13 @@ Attribute mapping:  -NOTE: -Using the **Group ID** source attribute requires users to enter the group ID or object ID when configuring SAML group links. If available, use the **sAMAccountName** source attribute for the friendly group name instead. +Using the **Group ID** source attribute requires users to enter the group ID or object ID when configuring SAML group links. + +If available, you can add user-friendly group names instead. When setting up Azure group claims: + +1. Select the **sAMAccountName** source attribute. +1. Enter a group name. You can specify a name up to 256 characters long. +1. To ensure the attribute is part of the assertion, select **Emit group names for cloud-only groups**. [Azure AD limits the number of groups that can be sent in a SAML response to 150](https://support.esri.com/en-us/knowledge-base/000022190). If a user is a member of more than 150 groups, Azure does not include that user's group claim in the SAML response. diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 5838b9821de..7795aed5fd0 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -396,11 +396,10 @@ convert the information to XML. An example SAML response is shown here. By default, users provisioned with SAML or SCIM are sent a verification email to verify their identity. Instead, you can [configure GitLab with a custom domain](../../enterprise_user/index.md#set-up-a-verified-domain) and GitLab automatically confirms user accounts. Users still receive an -[enterprise user](../../enterprise_user/index.md) welcome email. Confirmation is -bypassed for users: +[enterprise user](../../enterprise_user/index.md) welcome email. Confirmation is bypassed if both of the following are true: -- That are provisioned with SAML or SCIM. -- That have an email address that belongs to the verified domain. +- The user is provisioned with SAML or SCIM. +- The user has an email address that belongs to the verified domain. ### Block user access @@ -490,6 +489,18 @@ When the **Enforce SSO-only authentication for web activity for this group** opt - For non-members or users who are not signed in: - SSO is not enforced when they access public group resources. - SSO is enforced when they access private group resources. +- For items in the organization's group hierarchy, dashboard visibility is as + follows: + - SSO is enforced when viewing your [To-Do List](../../todos.md). Your + to-do items are hidden if your SSO session has expired, and an + [alert is shown](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115254). + - SSO is enforced when viewing your list of assigned issues. Your issues are + hidden if your SSO session has expired. + [Issue 414475](https://gitlab.com/gitlab-org/gitlab/-/issues/414475) proposes to change this + behavior so that issues are visible. + - SSO is not enforced when viewing lists of merge requests where you are the + assignee or your review is requested. You can see merge requests even if + your SSO session has expired. SSO enforcement for web activity has the following effects when enabled: diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index e5d6c86a5ac..9096824cc2c 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -38,7 +38,6 @@ You can configure one of the following as an identity provider: - [Azure Active Directory](#configure-azure-active-directory). - [Okta](#configure-okta). -- [OneLogin](#configure-onelogin). NOTE: Other providers can work with GitLab but they have not been tested and are not supported. @@ -159,15 +158,6 @@ To configure Okta for SCIM: 1. Select **Save**. 1. Assign users in the **Assignments** tab. Assigned users are created and managed in your GitLab group. -### Configure OneLogin - -Prerequisites: - -- [GitLab is configured](#configure-gitlab). - -OneLogin provides a **GitLab (SaaS)** app in their catalog, which includes a SCIM integration. Contact OneLogin if you -encounter issues. - ## User access > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325712) in GitLab 14.0, GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning are displayed with an [**Enterprise**](../../enterprise_user/index.md) badge in the **Members** view. @@ -183,7 +173,11 @@ The following diagram describes what happens when you add users to your SCIM app graph TD A[Add User to SCIM app] -->|IdP sends user info to GitLab| B(GitLab: Does the email exist?) B -->|No| C[GitLab creates user with SCIM identity] - B -->|Yes| D[GitLab sends message back 'Email exists'] + B -->|Yes| D(GitLab: Is the user part of the group?) + D -->|No| E(GitLab: Is SSO enforcement enabled?) + E -->|No| G + E -->|Yes| F[GitLab sends message back:\nThe member's email address is not linked to a SAML account] + D -->|Yes| G[Associate SCIM identity to user] ``` During provisioning: diff --git a/doc/user/group/saml_sso/troubleshooting_scim.md b/doc/user/group/saml_sso/troubleshooting_scim.md index 33343c9b339..e4531882fc1 100644 --- a/doc/user/group/saml_sso/troubleshooting_scim.md +++ b/doc/user/group/saml_sso/troubleshooting_scim.md @@ -38,7 +38,7 @@ The following are possible solutions for problems where users cannot sign in: To check if a user's SAML `NameId` matches their SCIM `externalId`: -- Administrators can use the Admin Area to [list SCIM identities for a user](../../admin_area/index.md#user-identities). +- Administrators can use the Admin Area to [list SCIM identities for a user](../../../administration/admin_area.md#user-identities). - Group owners can see the list of users and the identifier stored for each user in the group SAML SSO Settings page. - You can use the [SCIM API](../../../api/scim.md) to manually retrieve the `external_uid` GitLab has stored for users and compare the value for each user from the [SAML API](../../../api/saml.md) . - Have the user use a [SAML Tracer](troubleshooting.md#saml-debugging-tools) and compare the `external_uid` to @@ -182,7 +182,7 @@ In your Okta SCIM application, check that the SCIM **Base URL** is correct and p SCIM API endpoint URL. Check the following documentation to find information on this URL for: - [GitLab.com groups](scim_setup.md#configure-gitlab). -- [Self-managed GitLab instances](../../admin_area/settings/scim_setup.md#configure-gitlab). +- [Self-managed GitLab instances](../../../administration/settings/scim_setup.md#configure-gitlab). For self-managed GitLab instances, ensure that GitLab is publicly available so Okta can connect to it. If needed, you can [allow access to Okta IP addresses](https://help.okta.com/en-us/Content/Topics/Security/ip-address-allow-listing.htm) diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index e264778062b..76aa77d026b 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -25,7 +25,7 @@ Group access tokens are similar to [project access tokens](../../project/setting and [personal access tokens](../../profile/personal_access_tokens.md), except they are associated with a group rather than a project or user. -In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. +In self-managed instances, group access tokens are subject to the same [maximum lifetime limits](../../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. WARNING: The ability to create group access tokens without an expiry date was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. In GitLab 16.0 and later, existing group access tokens without an expiry date are automatically given an expiry date 365 days later than the current date. The automatic adding of an expiry date occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry date occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. @@ -35,13 +35,13 @@ You can use group access tokens: - On GitLab SaaS: If you have the Premium or Ultimate license tier. Group access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). - On self-managed instances: With any license tier. If you have the Free tier: - Review your security and compliance policies around - [user self-enrollment](../../admin_area/settings/sign_up_restrictions.md#disable-new-sign-ups). + [user self-enrollment](../../../administration/settings/sign_up_restrictions.md#disable-new-sign-ups). - Consider [disabling group access tokens](#enable-or-disable-group-access-token-creation) to lower potential abuse. You cannot use group access tokens to create other group, project, or personal access tokens. -Group access tokens inherit the [default prefix setting](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) +Group access tokens inherit the [default prefix setting](../../../administration/settings/account_and_limit_settings.md#personal-access-token-prefix) configured for personal access tokens. ## Create a group access token using UI @@ -64,7 +64,7 @@ To create a group access token: - The token expires on that date at midnight UTC. - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. - By default, this date can be a maximum of 365 days later than the current date. - - An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. + - An instance-wide [maximum lifetime](../../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-group-access-token). 1. Select **Create group access token**. @@ -148,6 +148,7 @@ The scope determines the actions you can perform when you authenticate with a gr | `write_registry` | Grants write access (push) to the [Container Registry](../../packages/container_registry/index.md). | | `read_repository` | Grants read access (pull) to all repositories within a group. | | `write_repository` | Grants read and write access (pull and push) to all repositories within a group. | +| `create_runner` | Grants permission to create runners in a group. | ## Enable or disable group access token creation diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 81e972bf73b..84902a5cbe9 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -77,7 +77,7 @@ Prerequisites: - At least the Maintainer role for a group to create subgroups for it. - The [role determined by a setting](#change-who-can-create-subgroups). These users can create subgroups even if group creation is - [disabled by an Administrator](../../admin_area/index.md#prevent-a-user-from-creating-groups) in the user's settings. + [disabled by an Administrator](../../../administration/admin_area.md#prevent-a-user-from-creating-groups) in the user's settings. NOTE: You cannot host a GitLab Pages subgroup website with a top-level domain name. For example, `subgroupname.example.io`. diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index 952552fc648..dba7a507fef 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -45,8 +45,8 @@ Value stream analytics offers different features at the project and group level |Total time chart|Yes|Yes|No| |Task by type chart|Yes|No|No| |DORA Metrics|Yes|Yes|No| -|Cycle time and lead time summary (Key metrics)|Yes|Yes|No| -|New issues, commits, and deploys (Key metrics)|Yes, excluding commits|Yes|Yes| +|Cycle time and lead time summary (Lifecycle metrics)|Yes|Yes|No| +|New issues, commits, and deploys (Lifecycle metrics)|Yes, excluding commits|Yes|Yes| |Uses aggregated backend|Yes|Yes|No| |Date filter behavior|Filters items [finished within the date range](https://gitlab.com/groups/gitlab-org/-/epics/6046)|Filters items by creation date.|Filters items by creation date.| |Authorization|At least reporter|At least reporter|Can be public| @@ -253,9 +253,9 @@ For the "Tasks by type" chart, only the Date range and Project selector filters The **Overview** page in value stream analytics displays key metrics of the DevSecOps lifecycle performance for projects and groups. -### Key metrics +### Lifecycle metrics -Value stream analytics includes the following key metrics: +Value stream analytics includes the following lifecycle metrics: - **Lead time**: Median time from when the issue was created to when it was closed. - **Cycle time**: Median time from first commit to issue closed. GitLab measures cycle time from the earliest commit of a @@ -290,21 +290,21 @@ NOTE: In GitLab 13.9 and later, deployment frequency metrics are calculated based on when the deployment was finished. In GitLab 13.8 and earlier, deployment frequency metrics are calculated based on when the deployment was created. -## View key and DORA metrics +## View lifecycle and DORA metrics Prerequisite: - To view deployment metrics, you must have a [production environment configured](#how-value-stream-analytics-identifies-the-production-environment). -To view key lifecycle metrics: +To view lifecycle metrics: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. 1. Select **Analyze > Value stream analytics**. - Key metrics display below the **Filter results** text box. + Lifecycle metrics display below the **Filter results** text box. 1. Optional. Filter the results: 1. Select the **Filter results** text box. - Based on the filter you select, the dashboard automatically aggregates key metrics and displays the status of the value stream. + Based on the filter you select, the dashboard automatically aggregates lifecycle metrics and displays the status of the value stream. 1. Select a parameter. 1. Select a value or enter text to refine the results. 1. To adjust the date range: @@ -315,7 +315,7 @@ To view the [Value Streams Dashboard](../../analytics/value_streams_dashboard.md 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project or group. 1. Select **Analyze > Value stream analytics**. -1. Below the **Filter results** text box, in the **Key metrics** row, select **Value Streams Dashboard / DORA**. +1. Below the **Filter results** text box, in the **Lifecycle metrics** row, select **Value Streams Dashboard / DORA**. 1. Optional. To open the new page, append this path `/analytics/dashboards/value_streams_dashboard` to the group URL (for example, `https://gitlab.com/groups/gitlab-org/-/analytics/dashboards/value_streams_dashboard`). diff --git a/doc/user/img/award_emoji_votes_sort_options.png b/doc/user/img/award_emoji_votes_sort_options.png Binary files differdeleted file mode 100644 index dc02d5169e0..00000000000 --- a/doc/user/img/award_emoji_votes_sort_options.png +++ /dev/null diff --git a/doc/user/img/custom_emoji_reactions_v16_2.png b/doc/user/img/custom_emoji_reactions_v16_2.png Binary files differnew file mode 100644 index 00000000000..a4d18edf8a5 --- /dev/null +++ b/doc/user/img/custom_emoji_reactions_v16_2.png diff --git a/doc/user/img/objective_two_column_view_v16_2.png b/doc/user/img/objective_two_column_view_v16_2.png Binary files differnew file mode 100644 index 00000000000..d3f4f733e7e --- /dev/null +++ b/doc/user/img/objective_two_column_view_v16_2.png diff --git a/doc/user/img/rich_text_editor_01_v16_2.png b/doc/user/img/rich_text_editor_01_v16_2.png Binary files differnew file mode 100644 index 00000000000..7242f7169d5 --- /dev/null +++ b/doc/user/img/rich_text_editor_01_v16_2.png diff --git a/doc/user/img/rich_text_editor_02_v16_2.png b/doc/user/img/rich_text_editor_02_v16_2.png Binary files differnew file mode 100644 index 00000000000..b99f540cb60 --- /dev/null +++ b/doc/user/img/rich_text_editor_02_v16_2.png diff --git a/doc/user/img/rich_text_editor_03_v16_2.png b/doc/user/img/rich_text_editor_03_v16_2.png Binary files differnew file mode 100644 index 00000000000..5ee560fe79e --- /dev/null +++ b/doc/user/img/rich_text_editor_03_v16_2.png diff --git a/doc/user/img/rich_text_editor_04_v16_2.png b/doc/user/img/rich_text_editor_04_v16_2.png Binary files differnew file mode 100644 index 00000000000..70f89754f92 --- /dev/null +++ b/doc/user/img/rich_text_editor_04_v16_2.png diff --git a/doc/user/img/task_two_column_view_v16_2.png b/doc/user/img/task_two_column_view_v16_2.png Binary files differnew file mode 100644 index 00000000000..8ca87f55f8a --- /dev/null +++ b/doc/user/img/task_two_column_view_v16_2.png diff --git a/doc/user/img/todos_add_okrs_v16_0.png b/doc/user/img/todos_add_okrs_v16_0.png Binary files differdeleted file mode 100644 index 45c04e647e2..00000000000 --- a/doc/user/img/todos_add_okrs_v16_0.png +++ /dev/null diff --git a/doc/user/img/todos_add_todo_sidebar_v14_1.png b/doc/user/img/todos_add_todo_sidebar_v14_1.png Binary files differdeleted file mode 100644 index 65120beca29..00000000000 --- a/doc/user/img/todos_add_todo_sidebar_v14_1.png +++ /dev/null diff --git a/doc/user/img/todos_mark_done_okrs_v16_0.png b/doc/user/img/todos_mark_done_okrs_v16_0.png Binary files differdeleted file mode 100644 index 545b3569ed5..00000000000 --- a/doc/user/img/todos_mark_done_okrs_v16_0.png +++ /dev/null diff --git a/doc/user/img/todos_mark_done_sidebar_v14_1.png b/doc/user/img/todos_mark_done_sidebar_v14_1.png Binary files differdeleted file mode 100644 index 628fe65a7c1..00000000000 --- a/doc/user/img/todos_mark_done_sidebar_v14_1.png +++ /dev/null diff --git a/doc/user/infrastructure/clusters/connect/new_aks_cluster.md b/doc/user/infrastructure/clusters/connect/new_aks_cluster.md new file mode 100644 index 00000000000..24933875d48 --- /dev/null +++ b/doc/user/infrastructure/clusters/connect/new_aks_cluster.md @@ -0,0 +1,132 @@ +--- +stage: Deploy +group: Environments +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Create an Azure AKS cluster + +You can create a cluster on Azure Kubernetes Service (AKS) through +[Infrastructure as Code (IaC)](../../index.md). This process uses the Azure and +Kubernetes Terraform providers to create AKS clusters. You connect the clusters to GitLab +by using the GitLab agent for Kubernetes. + +**Prerequisites:** + +- A Microsoft Azure account, with a set of configured + [security credentials](https://learn.microsoft.com/en-us/cli/azure/authenticate-azure-cli). +- [A runner](https://docs.gitlab.com/runner/install/) you can use to run the GitLab CI/CD pipeline. + +**Steps:** + +1. [Import the example project](#import-the-example-project). +1. [Register the agent for Kubernetes](#register-the-agent). +1. [Configure your project](#configure-your-project). +1. [Provision your cluster](#provision-your-cluster). + +## Import the example project + +To create a cluster from GitLab using Infrastructure as Code, you must +create a project to manage the cluster from. In this tutorial, you start with +a sample project and modify it according to your needs. + +Start by [importing the example project by URL](../../../project/import/repo_by_url.md). + +To import the project: + +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. +1. Select **Import project**. +1. Select **Repository by URL**. +1. For the **Git repository URL**, enter `https://gitlab.com/gitlab-org/ci-cd/deploy-stage/environments-group/examples/gitlab-terraform-aks.git`. +1. Complete the fields and select **Create project**. + +This project provides you with: + +- An [Azure Kubernetes Service (AKS)](https://gitlab.com/gitlab-org/ci-cd/deploy-stage/environments-group/examples/gitlab-terraform-aks/-/blob/main/aks.tf) cluster. +- The [GitLab agent for Kubernetes](https://gitlab.com/gitlab-org/ci-cd/deploy-stage/environments-group/examples/gitlab-terraform-aks/-/blob/main/agent.tf) installed in the cluster. + +## Register the agent + +To create a GitLab agent for Kubernetes: + +1. On the left sidebar, select **Operate > Kubernetes clusters**. +1. Select **Connect a cluster (agent)**. +1. From the **Select an agent** dropdown list, select `aks-agent` and select **Register an agent**. +1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. +1. GitLab provides an address for the agent server (KAS), which you will also need later. + +## Configure your project + +Use CI/CD environment variables to configure your project. + +**Required configuration:** + +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Variables**. +1. Set the variable `AZURE_CLIENT_ID` to your Azure client ID. +1. Set the variable `AZURE_CLIENT_SECRET` to your Azure client secret. +1. Set the variable `AZURE_TENANT_ID` to your service principal. +1. Set the variable `TF_VAR_agent_token` to the agent token displayed in the previous task. +1. Set the variable `TF_VAR_kas_address` to the agent server address displayed in the previous task. + +**Optional configuration:** + +The file [`variables.tf`](https://gitlab.com/gitlab-org/ci-cd/deploy-stage/environments-group/examples/gitlab-terraform-aks/-/blob/main/variables.tf) +contains other variables that you can override according to your needs: + +- `TF_VAR_location`: Set your cluster's region. +- `TF_VAR_cluster_name`: Set your cluster's name. +- `TF_VAR_kubernetes_version`: Set the version of Kubernetes. +- `TF_VAR_create_resource_group`: Allow to enable or disable the creation of a new resource group. (Default set to true). +- `TF_VAR_resource_group_name`: Set the name of resource group. +- `TF_VAR_agent_namespace`: Set the Kubernetes namespace for the GitLab agent. + +See the [Azure Terraform provider](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. + +## Provision your cluster + +After configuring your project, manually trigger the provisioning of your cluster. In GitLab: + +1. On the left sidebar, select **Build > Pipelines**. +1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**). +1. Select **Deploy** to manually trigger the deployment job. + +When the pipeline finishes successfully, you can view the new cluster: + +- In Azure: From the [Azure portal](https://portal.azure.com/#home), select **Kubernetes services > View**. +- In GitLab: On the left sidebar, select **Operate > Kubernetes clusters**. + +## Use your cluster + +After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: + +1. On the left sidebar, select **Operate > Kubernetes clusters**. +1. In the list, view the **Connection status** column. + +For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). + +## Remove the cluster + +A cleanup job is not included in your pipeline by default. To remove all created resources, you +must modify your GitLab CI/CD template before running the cleanup job. + +To remove all resources: + +1. Add the following to your `.gitlab-ci.yml` file: + + ```yaml + stages: + - init + - validate + - test + - build + - deploy + - cleanup + + destroy: + extends: .terraform:destroy + needs: [] + ``` + +1. On the left sidebar, select **Build > Pipelines** and select the most recent pipeline. +1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md index 7c8065b6143..91429e203f3 100644 --- a/doc/user/infrastructure/clusters/connect/new_civo_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_civo_cluster.md @@ -52,7 +52,7 @@ This project provides you with: To create a GitLab agent for Kubernetes: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. Select **Connect a cluster (agent)**. 1. From the **Select an agent** dropdown list, select `civo-agent` and select **Register an agent**. 1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. @@ -91,20 +91,20 @@ Refer to the [Civo Terraform provider](https://registry.terraform.io/providers/c After configuring your project, manually trigger the provisioning of your cluster. In GitLab: -1. On the left sidebar, go to **CI/CD > Pipelines**. +1. On the left sidebar, go to **Build > Pipelines**. 1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**). 1. Select **Deploy** to manually trigger the deployment job. When the pipeline finishes successfully, you can see your new cluster: - In Civo dashboard: on your Kubernetes tab. -- In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. +- In GitLab: from your project's sidebar, select **Operate > Kubernetes clusters**. ## Use your cluster After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. In the list, view the **Connection status** column. For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). @@ -131,7 +131,7 @@ To remove all resources: needs: [] ``` -1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. +1. On the left sidebar, select **Build > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). ## Civo support diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md index 19bcce581e9..ef51db097d9 100644 --- a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -55,7 +55,7 @@ In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `ce To create a GitLab agent for Kubernetes: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. Select **Connect a cluster (agent)**. 1. From the **Select an agent** dropdown list, select `eks-agent` and select **Register an agent**. 1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. @@ -139,20 +139,20 @@ View the [AWS Terraform provider](https://registry.terraform.io/providers/hashic After configuring your project, manually trigger the provisioning of your cluster. In GitLab: -1. On the left sidebar, go to **CI/CD > Pipelines**. +1. On the left sidebar, go to **Build > Pipelines**. 1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**). 1. Select **Deploy** to manually trigger the deployment job. When the pipeline finishes successfully, you can view the new cluster: - In AWS: From the [EKS console](https://console.aws.amazon.com/eks/home), select **Amazon EKS > Clusters**. -- In GitLab: On the left sidebar, select **Infrastructure > Kubernetes clusters**. +- In GitLab: On the left sidebar, select **Operate > Kubernetes clusters**. ## Use your cluster After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. In the list, view the **Connection status** column. For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). @@ -180,5 +180,5 @@ To remove all resources: needs: [] ``` -1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. +1. On the left sidebar, select **Build > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index 25a0a7149e0..3d717e0f473 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -62,7 +62,7 @@ In GitLab 14.10, a [flag](../../../../administration/feature_flags.md) named `ce To create a GitLab agent for Kubernetes: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. Select **Connect a cluster (agent)**. 1. From the **Select an agent** dropdown list, select `gke-agent` and select **Register an agent**. 1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. @@ -117,20 +117,20 @@ Refer to the [Google Terraform provider](https://registry.terraform.io/providers After configuring your project, manually trigger the provisioning of your cluster. In GitLab: -1. On the left sidebar, go to **CI/CD > Pipelines**. +1. On the left sidebar, go to **Build > Pipelines**. 1. Next to **Play** (**{play}**), select the dropdown list icon (**{chevron-lg-down}**). 1. Select **Deploy** to manually trigger the deployment job. When the pipeline finishes successfully, you can see your new cluster: - In GCP: on your [GCP console's Kubernetes list](https://console.cloud.google.com/kubernetes/list). -- In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. +- In GitLab: from your project's sidebar, select **Operate > Kubernetes clusters**. ## Use your cluster After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. In the list, view the **Connection status** column. For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). @@ -158,5 +158,5 @@ To remove all resources: needs: [] ``` -1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. +1. On the left sidebar, select **Build > Pipelines** and select the most recent pipeline. 1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md index 50185966267..4c55a87a52c 100644 --- a/doc/user/infrastructure/clusters/deploy/inventory_object.md +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -4,11 +4,15 @@ group: Environments info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Tracking cluster resources managed by GitLab **(FREE)** +# Tracking cluster resources managed by GitLab (deprecated) **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/346567) from GitLab Premium to GitLab Free in 15.3. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/406545) in GitLab 16.2. +To manage cluster resources with GitOps, you should use the [Flux integration](../../../clusters/agent/gitops.md). + GitLab uses an inventory object to track the resources you deploy to your cluster. The inventory object is a `ConfigMap` that contains a list of controlled objects. The managed resources use the `cli-utils.sigs.k8s.io/inventory-id` annotation. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md index 3084cc28c9b..e23f5e8745c 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md @@ -25,15 +25,22 @@ For GitLab Runner to function, you _must_ specify the following in your - `gitlabUrl`: The GitLab server full URL (for example, `https://gitlab.example.com`) to register the Runner against. -- `runnerRegistrationToken`: The registration token for adding new runners to GitLab. - This must be [retrieved from your GitLab instance](../../../../../ci/runners/index.md). +- Runner token: This must be [retrieved](../../../../../ci/runners/index.md) from your GitLab instance. You can use + either of the following tokens: + + - `runnerToken`: The runner authentication token for the runner configuration [created in the GitLab UI](../../../../../ci/runners/runners_scope.md). + - `runnerRegistrationToken` ([deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102681) in GitLab 15.6 and scheduled to be removed in GitLab 17.0): The registration token for used to add new runners to GitLab. These values can be specified using [CI/CD variables](../../../../../ci/variables/index.md): - `CI_SERVER_URL` is used for `gitlabUrl`. If you are using GitLab.com, you don't need to set this variable. -- `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` +- `GITLAB_RUNNER_TOKEN` is used for `runnerToken`. +- `GITLAB_RUNNER_REGISTRATION_TOKEN` is used for `runnerRegistrationToken` (deprecated). + +The methods of specifying these values are mutually exclusive. You can either: -The methods of specifying these values are mutually exclusive. Either specify variables `GITLAB_RUNNER_REGISTRATION_TOKEN` and `CI_SERVER_URL` as CI variables (recommended) or provide values for `runnerRegistrationToken:` and `gitlabUrl:` in `applications/gitlab-runner/values.yaml.gotmpl`. +- Specify the variables `GITLAB_RUNNER_TOKEN` and `CI_SERVER_URL` as CI variables (recommended). +- Provide values for `runnerToken:` and `gitlabUrl:` in `applications/gitlab-runner/values.yaml.gotmpl`. The runner registration token allows connection to a project by a runner and therefore should be treated as a secret to prevent malicious use and code exfiltration through a runner. For this reason, we recommend that you specify the runner registration token as a [protected variable](../../../../../ci/variables/index.md#protect-a-cicd-variable) and [masked variable](../../../../../ci/variables/index.md#mask-a-cicd-variable) and do not commit them to the Git repository in the `values.yaml.gotmpl` file. diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index a21c7271e7a..614e0f5a7e5 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -70,7 +70,7 @@ In your Auto DevOps project, you can use the GitLab agent to connect with your K - Add a key called `KUBE_NAMESPACE` with a value of the Kubernetes namespace for your deployments to target. Set the same environment scope. 1. Select **Add variable**. -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Operate > Kubernetes clusters**. 1. From the certificate-based clusters section, open the cluster that serves the same environment scope. 1. Select the **Details** tab and disable the cluster. 1. Edit your `.gitlab-ci.yml` file and ensure it's using the Auto DevOps template. For example: @@ -85,7 +85,7 @@ In your Auto DevOps project, you can use the GitLab agent to connect with your K KUBE_NAMESPACE: "demo-agent" ``` -1. To test your pipeline, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. +1. To test your pipeline, on the left sidebar, select **Build > Pipelines** and then **Run pipeline**. For an example, [view this project](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service). diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 12ad207e4f8..f27f1306c31 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -54,7 +54,7 @@ If you use earlier versions of GitLab, you might face incompatibility errors between the GitLab version and the template version. In this case, you can opt to use one of these templates: -- [The stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) with an skeleton that you can built on top of. +- [The stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml) with a skeleton that you can built on top of. - [The advanced template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Base.gitlab-ci.yml) to fully customize your setup. NOTE: diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index b2e02cd7375..26bb1d16510 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -95,19 +95,17 @@ To manually configure a GitLab Terraform Report artifact: ```yaml default: image: registry.gitlab.com/gitlab-org/terraform-images/stable:latest - cache: key: example-production paths: - ${TF_ROOT}/.terraform + before_script: + - cd ${TF_ROOT} variables: TF_ROOT: ${CI_PROJECT_DIR}/environments/example/production TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/example-production -before_script: - - cd ${TF_ROOT} - stages: - prepare - validate @@ -161,7 +159,6 @@ default: entrypoint: - '/usr/bin/env' - 'PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin' - cache: paths: - .terraform diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 455f2ce19e8..31c4ad757c8 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -321,7 +321,7 @@ curl --header "Private-Token: <your_access_token>" --request DELETE "https://git If you have at least the Maintainer role, you can remove a state file. -1. On the left sidebar, select **Infrastructure > Terraform states**. +1. On the left sidebar, select **Operate > Terraform states**. 1. In the **Actions** column, select **Actions** (**{ellipsis_v}**) and then **Remove state file and versions**. ### Remove a state file by using the API diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 401fe0bcb09..c3e4f77411c 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -58,7 +58,7 @@ Features not found in standard Markdown: - [Color chips written in `HEX`, `RGB` or `HSL`](#colors) - [Diagrams and flowcharts](#diagrams-and-flowcharts) -- [Emoji](#emojis) +- [Emoji](#emoji) - [Front matter](#front-matter) - [Inline diffs](#inline-diff) - [Math equations and symbols written in LaTeX](#math) @@ -207,9 +207,9 @@ installation of GitLab, a GitLab administrator [must enable it](../administratio To make Kroki available in GitLab, a GitLab administrator must enable it. For more information, see the [Kroki integration](../administration/integration/kroki.md) page. -### Emojis +### Emoji -[View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#emojis). +[View this topic in GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md#emoji). ::Tabs @@ -250,13 +250,13 @@ for a list of all supported emoji codes. :thumbsup: ::EndTabs -#### Emojis and your operating system +#### Emoji and your operating system -The previous emoji example uses hard-coded images. Rendered emojis +The previous emoji example uses hard-coded images. Rendered emoji in GitLab may be different depending on the OS and browser used. -Most emojis are natively supported on macOS, Windows, iOS, Android, and fall back on image-based -emojis where there is no support. +Most emoji are natively supported on macOS, Windows, iOS, Android, and fall back on image-based +emoji where there is no support. <!-- vale gitlab.Spelling = NO --> @@ -266,7 +266,7 @@ this font installed by default. <!-- vale gitlab.Spelling = YES --> -To learn more about adding custom emojis, see [Custom emojis](award_emojis.md#custom-emojis). +To learn more about adding custom emoji, see [Custom emoji](award_emojis.md#custom-emoji). ### Front matter @@ -569,13 +569,13 @@ This example links to `<wiki_root>/miscellaneous.md`: In wikis, you can use the [diagrams.net](https://www.diagrams.net/) editor to create diagrams. You can also edit diagrams created with the diagrams.net editor. The diagram editor is available in both -the Markdown editor and the content editor. +the plain text editor and the rich text editor. For more information, see [Diagrams.net](../administration/integration/diagrams_net.md). -##### Markdown editor +##### Plain text editor -To create a diagram in the Markdown editor: +To create a diagram in the plain text editor: 1. In the editor's toolbar, select **Insert or edit diagram** (**{diagram}**). 1. Use the diagrams.net editor to create the diagram. @@ -583,20 +583,20 @@ To create a diagram in the Markdown editor: A Markdown image reference to the diagram is inserted in the wiki content. -To edit a diagram in the Markdown editor: +To edit a diagram in the plain text editor: -1. Place the Markdown editor's text field cursor in a Markdown image reference +1. Place the plain text editor's text field cursor in a Markdown image reference that contains the diagram. -1. Select **Insert or edit diagram** (**{diagram}**) in the Markdown editor. +1. Select **Insert or edit diagram** (**{diagram}**) in the plain text editor. 1. Use the diagrams.net editor to edit the diagram. 1. Select **Save & exit**. A Markdown image reference to the diagram is inserted in the wiki content, replacing the previous diagram. -##### Content editor +##### Rich text editor -To create a diagram in the content editor: +To create a diagram in the rich text editor: 1. In the editor's toolbar, select **More options** (**{plus}**). 1. In the dropdown list, select **Create or edit diagram**. @@ -605,7 +605,7 @@ To create a diagram in the content editor: The diagram as visualized in the diagrams.net editor is inserted in the wiki content. -To edit a diagram in the content editor: +To edit a diagram in the rich text editor: 1. Select the diagram that you want to edit. 1. In the floating toolbar, select **Edit diagram** (**{diagram}**). @@ -1082,6 +1082,8 @@ The following codeblock uses HTML to skip the Vale ReferenceLinks test. Do not change it back to a markdown codeblock. --> +<!-- markdownlint-disable proper-names --> + <pre class="highlight"><code>Inline-style (hover to see title text):  @@ -1093,6 +1095,8 @@ Reference-style (hover to see title text): [logo]: img/markdown_logo.png "Title Text" </code></pre> +<!-- markdownlint-enable proper-names --> + <!-- DO NOT change the name of markdown_logo.png. This file is used for a test in spec/controllers/help_controller_spec.rb. diff --git a/doc/user/okrs.md b/doc/user/okrs.md index 2abaf9f8008..6579ecbadbc 100644 --- a/doc/user/okrs.md +++ b/doc/user/okrs.md @@ -12,7 +12,7 @@ OKRs are an [Experiment](../policy/experiment-beta-support.md#experiment). For the OKR feature roadmap, see [epic 7864](https://gitlab.com/groups/gitlab-org/-/epics/7864). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project, ask an administrator to [enable the featured flag](../administration/feature_flags.md) named `okrs_mvc`. +On self-managed GitLab, by default this feature is not available. To make it available per project, an administrator can [enable the featured flag](../administration/feature_flags.md) named `okrs_mvc`. On GitLab.com, this feature is not available. The feature is not ready for production use. @@ -353,3 +353,18 @@ Prerequisites: By default, child OKRs are ordered by creation date. To reorder them, drag them around. + +## Two-column layout + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/415077) in GitLab 16.2 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `work_items_mvc_2`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +When enabled, OKRs use a two-column layout, similar to issues. +The description and threads are on the left, and attributes, such as labels +or assignees, on the right. + + diff --git a/doc/user/organization/index.md b/doc/user/organization/index.md index f5af26250f6..2a33543fea5 100644 --- a/doc/user/organization/index.md +++ b/doc/user/organization/index.md @@ -24,7 +24,7 @@ everything you do as a GitLab administrator, including: - Aggregating data from all your groups, subgroups, and projects. Our goal is to reach feature parity between SaaS and self-managed installations, with all -[Admin Area settings](/ee/user/admin_area/settings/index.md) moving to either: +[Admin Area settings](../../administration/settings/index.md) moving to either: - Groups. Available in the Organization, and subgroups. - Hardware Controls. For functionality that does not apply to groups, Hardware Controls are only diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index b990cf1f09b..3f8b0761188 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -76,7 +76,7 @@ To publish the package with a deploy token: - `<tag>` is the Git tag name of the version you want to publish. To publish a branch, use `branch=<branch>` instead of `tag=<tag>`. -You can view the published package by going to **Packages and registries > Package Registry** and +You can view the published package by going to **Deploy > Package Registry** and selecting the **Composer** tab. ## Publish a Composer package by using CI/CD @@ -99,13 +99,13 @@ You can publish a Composer package to the Package Registry as part of your CI/CD 1. Run the pipeline. -To view the published package, go to **Packages and registries > Package Registry** and select the **Composer** tab. +To view the published package, go to **Deploy > Package Registry** and select the **Composer** tab. ### Use a CI/CD template A more detailed Composer CI/CD file is also available as a `.gitlab-ci.yml` template: -1. On the left sidebar, select **Project information**. +1. On the left sidebar, select **Project overview**. 1. Above the file list, select **Set up CI/CD**. If this button is not available, select **CI/CD Configuration** and then **Edit**. 1. From the **Apply a template** list, select **Composer**. @@ -142,7 +142,7 @@ To install a package: - Connect to the Package Registry for your group: ```shell - composer config repositories.<group_id> composer https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/ + composer config repositories.<group_id> composer https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/packages.json ``` - Set the required package version: @@ -159,7 +159,7 @@ To install a package: "repositories": { "<group_id>": { "type": "composer", - "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/" + "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/packages.json" }, ... }, @@ -243,7 +243,7 @@ To install a package: { ... "repositories": [ - { "type": "composer", "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/" } + { "type": "composer", "url": "https://gitlab.example.com/api/v4/group/<group_id>/-/packages/composer/packages.json" } ], "config": { ... diff --git a/doc/user/packages/conan_repository/index.md b/doc/user/packages/conan_repository/index.md index f06a7d83f92..1ebd59d18fa 100644 --- a/doc/user/packages/conan_repository/index.md +++ b/doc/user/packages/conan_repository/index.md @@ -295,7 +295,7 @@ There are two ways to remove a Conan package from the GitLab Package Registry. - From the GitLab user interface: - Go to your project's **Packages and registries > Package Registry**. Remove the + Go to your project's **Deploy > Package Registry**. Remove the package by selecting **Remove repository** (**{remove}**). ## Search for Conan packages in the Package Registry diff --git a/doc/user/packages/container_registry/authenticate_with_container_registry.md b/doc/user/packages/container_registry/authenticate_with_container_registry.md index 31337d19d59..6a7c92fd924 100644 --- a/doc/user/packages/container_registry/authenticate_with_container_registry.md +++ b/doc/user/packages/container_registry/authenticate_with_container_registry.md @@ -21,7 +21,7 @@ To authenticate with the Container Registry, you can use a: All of these authentication methods require the minimum scope: - For read (pull) access, to be `read_registry`. -- For write (push) access, to be`write_registry` and `read_registry`. +- For write (push) access, to be `write_registry` and `read_registry`. To authenticate, run the `docker login` command. For example: diff --git a/doc/user/packages/container_registry/build_and_push_images.md b/doc/user/packages/container_registry/build_and_push_images.md index aa86b87660b..4d74e029cdd 100644 --- a/doc/user/packages/container_registry/build_and_push_images.md +++ b/doc/user/packages/container_registry/build_and_push_images.md @@ -152,9 +152,12 @@ registry and used by subsequent stages, downloading the container image when nee `latest` and deployed using an application-specific deploy script: ```yaml -image: docker:20.10.16 -services: - - docker:20.10.16-dind +default: + image: docker:20.10.16 + services: + - docker:20.10.16-dind + before_script: + - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY stages: - build @@ -169,9 +172,6 @@ variables: CONTAINER_TEST_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG CONTAINER_RELEASE_IMAGE: $CI_REGISTRY_IMAGE:latest -before_script: - - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY - build: stage: build script: diff --git a/doc/user/packages/container_registry/delete_container_registry_images.md b/doc/user/packages/container_registry/delete_container_registry_images.md index b645dc3a3e6..148fa65d8c7 100644 --- a/doc/user/packages/container_registry/delete_container_registry_images.md +++ b/doc/user/packages/container_registry/delete_container_registry_images.md @@ -45,7 +45,7 @@ To delete container images using the GitLab UI: by selecting the red **{remove}** **Trash** icon next to the tag you want to delete. -1. In the dialog box, select **Remove tag**. +1. On the dialog, select **Remove tag**. ## Use the GitLab API diff --git a/doc/user/packages/container_registry/troubleshoot_container_registry.md b/doc/user/packages/container_registry/troubleshoot_container_registry.md index 729f4919188..34c86a90d49 100644 --- a/doc/user/packages/container_registry/troubleshoot_container_registry.md +++ b/doc/user/packages/container_registry/troubleshoot_container_registry.md @@ -125,5 +125,5 @@ This error happens when your authentication token expires before the image push the Container Registry on self-managed GitLab instances expire every five minutes. On GitLab.com, the token expiration time is set to 15 minutes. -If you are using self-managed GitLab, you can ask an administrator to +If you are using self-managed GitLab, an administrator can [increase the token duration](../../../administration/packages/container_registry.md#increase-token-duration). diff --git a/doc/user/packages/harbor_container_registry/index.md b/doc/user/packages/harbor_container_registry/index.md index 2bff6f79a27..21468b9d3bb 100644 --- a/doc/user/packages/harbor_container_registry/index.md +++ b/doc/user/packages/harbor_container_registry/index.md @@ -71,4 +71,4 @@ To remove the Harbor Registry for a project: 1. Clear the **Active** checkbox under **Enable integration**. 1. Select **Save changes**. -The **Packages and registries > Harbor Registry** entry is removed from the sidebar. +The **Operate > Harbor Registry** entry is removed from the sidebar. diff --git a/doc/user/packages/maven_repository/index.md b/doc/user/packages/maven_repository/index.md index bcc55af65bc..f06db7ef1ac 100644 --- a/doc/user/packages/maven_repository/index.md +++ b/doc/user/packages/maven_repository/index.md @@ -490,7 +490,7 @@ Your changes are automatically saved. ### Request forwarding to Maven Central FLAG: -By default this feature is not available for self-managed. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `maven_central_request_forwarding`. +By default this feature is not available for self-managed. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `maven_central_request_forwarding`. This feature is not available for SaaS users. When a Maven package is not found in the Package Registry, the request is forwarded diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index 91186e6c159..d06b1ababb8 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -26,7 +26,7 @@ Learn how to use the GitLab Package Registry to build your own custom package wo You can view packages for your project or group. 1. Go to the project or group. -1. Go to **Packages and registries > Package Registry**. +1. Go to **Deploy > Package Registry**. You can search, sort, and filter packages on this page. You can share your search results by copying and pasting the URL from your browser. @@ -131,7 +131,7 @@ You can also remove the Package Registry for your project specifically: **Packages** feature. 1. Select **Save changes**. -The **Packages and registries > Package Registry** entry is removed from the sidebar. +The **Deploy > Package Registry** entry is removed from the sidebar. ## Package Registry visibility permissions @@ -161,7 +161,7 @@ Registry disables all Package Registry operations. FLAG: On self-managed GitLab, by default this feature is available. To disable it, -ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `package_registry_access_level`. +an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `package_registry_access_level`. If you want to allow anyone (everyone on the internet) to pull from the Package Registry, no matter what the project visibility is, you can use the additional toggle `Allow anyone to pull from Package Registry` that appears when the project visibility is Private or Internal. diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md index 020cad5ac14..4882753d6cf 100644 --- a/doc/user/packages/package_registry/reduce_package_registry_storage.md +++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md @@ -29,7 +29,7 @@ You can delete packages by using [the API](../../../api/packages.md#delete-a-pro To delete a package in the UI, from your group or project: -1. Go to **Packages and registries > Package Registry**. +1. Go to **Deploy > Package Registry**. 1. Find the name of the package you want to delete. 1. Select **Delete**. @@ -46,7 +46,7 @@ You can delete packages by using [the API](../../../api/packages.md#delete-a-pac To delete package assets in the UI, from your group or project: -1. Go to **Packages and registries > Package Registry**. +1. Go to **Deploy > Package Registry**. 1. Find the name of the package you want to delete. 1. Select the package to view additional details. 1. Find the name of the assets you would like to delete. diff --git a/doc/user/packages/package_registry/supported_functionality.md b/doc/user/packages/package_registry/supported_functionality.md index ca174c43565..d2ee5645fc1 100644 --- a/doc/user/packages/package_registry/supported_functionality.md +++ b/doc/user/packages/package_registry/supported_functionality.md @@ -57,12 +57,12 @@ Requests for packages not found in your GitLab project are forwarded to the publ | Package type | Supports request forwarding | |-------------------------------------------------------|-----------------------------| -| [Maven (with `mvn`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | -| [Maven (with `gradle`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | -| [Maven (with `sbt`)](../maven_repository/index.md) | [Yes (disabled by default)](../../admin_area/settings/continuous_integration.md#maven-forwarding) | -| [npm](../npm_registry/index.md) | [Yes](../../admin_area/settings/continuous_integration.md#npm-forwarding) | +| [Maven (with `mvn`)](../maven_repository/index.md) | [Yes (disabled by default)](../../../administration/settings/continuous_integration.md#maven-forwarding) | +| [Maven (with `gradle`)](../maven_repository/index.md) | [Yes (disabled by default)](../../../administration/settings/continuous_integration.md#maven-forwarding) | +| [Maven (with `sbt`)](../maven_repository/index.md) | [Yes (disabled by default)](../../../administration/settings/continuous_integration.md#maven-forwarding) | +| [npm](../npm_registry/index.md) | [Yes](../../../administration/settings/continuous_integration.md#npm-forwarding) | | [NuGet](../nuget_repository/index.md) | N | -| [PyPI](../pypi_repository/index.md) | [Yes](../../admin_area/settings/continuous_integration.md#pypi-forwarding) | +| [PyPI](../pypi_repository/index.md) | [Yes](../../../administration/settings/continuous_integration.md#pypi-forwarding) | | [Generic packages](../generic_packages/index.md) | N | | [Terraform](../terraform_module_registry/index.md) | N | | [Composer](../composer_repository/index.md) | N | @@ -86,7 +86,7 @@ To reduce the associated security risks, before deleting a package you can: - Verify the package is not being actively used. - Disable request forwarding: - - Instance administrators can disable forwarding in the [**Continuous Integration** section](../../admin_area/settings/continuous_integration.md#package-registry-configuration) of the Admin Area. + - Instance administrators can disable forwarding in the [**Continuous Integration** section](../../../administration/settings/continuous_integration.md#package-registry-configuration) of the Admin Area. - Group owners can disable forwarding in the **Packages and Registries** section of the group settings. ## Allow or prevent duplicates **(FREE)** diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index d7f33dd9e79..3a3409daa6a 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -19,7 +19,7 @@ projects. To view Terraform modules in your project: 1. Go to the project. -1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. On the left sidebar, select **Operate > Terraform modules**. You can search, sort, and filter modules on this page. @@ -162,7 +162,7 @@ Prerequisites: - You need to [authenticate with the API](../../../api/rest/index.md#authentication). If authenticating with a personal access token, it must be configured with the `read_api` scope. -Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` file: +Authentication tokens (Job Token or Personal Access Token) can be provided for `terraform` in your `~/.terraformrc` or `%APPDATA%/terraform.rc` file: ```terraform credentials "gitlab.com" { @@ -186,7 +186,7 @@ Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the To download a Terraform module: -1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. On the left sidebar, select **Operate > Terraform modules**. 1. Select the name of the module you want to download. 1. In the **Activity** section, select the name of the module you want to download. @@ -217,7 +217,7 @@ You can delete modules by using [the packages API](../../../api/packages.md#dele To delete a module in the UI, from your project: -1. On the left sidebar, select **Packages and registries > Terraform modules**. +1. On the left sidebar, select **Operate > Terraform modules**. 1. Find the name of the package you want to delete. 1. Select **Delete**. diff --git a/doc/user/packages/workflows/project_registry.md b/doc/user/packages/workflows/project_registry.md index 7bff6cf8234..1df244cf0c6 100644 --- a/doc/user/packages/workflows/project_registry.md +++ b/doc/user/packages/workflows/project_registry.md @@ -35,7 +35,7 @@ of each package management system to publish different package types to the same Let's take a look at how you might create one project to host all of your packages: 1. Create a new project in GitLab. The project doesn't require any code or content. -1. On the left sidebar, select **Project information**, and note the project ID. +1. On the left sidebar, select **Project overview**, and note the project ID. 1. Create an access token for authentication. All package types in the Package Registry can be published by using: - A [personal access token](../../profile/personal_access_tokens.md). diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 0b02de59ab4..cf859174c10 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -58,11 +58,11 @@ The following table lists project permissions available for each role: | Action | Guest | Reporter | Developer | Maintainer | Owner | |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----------|-----------|------------|----------| | [Analytics](analytics/index.md):<br>View [issue analytics](analytics/issue_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [value stream analytics](group/value_stream_analytics/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [DORA metrics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [CI/CD analytics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Analytics](analytics/index.md):<br>View [repository analytics](analytics/repository_analytics.md) | | ✓ | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>View licenses in [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | | [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/proxy-based.md#on-demand-scans) | | | ✓ | ✓ | ✓ | @@ -223,7 +223,7 @@ The following table lists project permissions available for each role: <!-- markdownlint-disable MD029 --> -1. On self-managed GitLab instances, users with the Guest role are able to perform this action only on public and internal projects (not on private projects). [External users](admin_area/external_users.md) must be given explicit access even if the project is internal. Users with the Guest role on GitLab.com are only able to perform this action on public projects because internal visibility is not available. +1. On self-managed GitLab instances, users with the Guest role are able to perform this action only on public and internal projects (not on private projects). [External users](../administration/external_users.md) must be given explicit access (at least the **Reporter** role) even if the project is internal. Users with the Guest role on GitLab.com are only able to perform this action on public projects because internal visibility is not available. 2. Guest users can only view the [confidential issues](project/issues/confidential_issues.md) they created themselves or are assigned to. 3. Not allowed for Guest, Reporter, Developer, Maintainer, or Owner. See [protected branches](project/protected_branches.md). 4. If the [branch is protected](project/protected_branches.md), this depends on the access given to Developers and Maintainers. @@ -401,13 +401,13 @@ The following table lists group permissions available for each role: | View group [Usage Quotas](usage_quotas.md) page | | | | | ✓ (3) | | Manage group runners | | | | | ✓ | | [Migrate groups](group/import/index.md) | | | | | ✓ | -| Manage [subscriptions, and purchase storage and units of compute](../subscriptions/gitlab_com/index.md) | | | | | ✓ | +| Manage [subscriptions, and purchase storage and compute minutes](../subscriptions/gitlab_com/index.md) | | | | | ✓ | <!-- markdownlint-disable MD029 --> 1. Groups can be set to allow either Owners, or Owners and users with the Maintainer role, to [create subgroups](group/subgroups/index.md#create-a-subgroup). 2. Default project creation role can be changed at: - - The [instance level](admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). + - The [instance level](../administration/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). - The [group level](group/index.md#specify-who-can-add-projects-to-a-group). 3. Does not apply to subgroups. 4. Developers can push commits to the default branch of a new project only if the [default branch protection](group/manage.md#change-the-default-branch-protection-of-a-group) is set to "Partially protected" or "Not protected". diff --git a/doc/user/product_analytics/index.md b/doc/user/product_analytics/index.md index 0d97c008561..e04c61b2c00 100644 --- a/doc/user/product_analytics/index.md +++ b/doc/user/product_analytics/index.md @@ -13,12 +13,12 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - Snowplow integration introduced in GitLab 15.11 [with a flag](../../administration/feature_flags.md) named `product_analytics_snowplow_support`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. FLAG: -On self-managed GitLab, by default the Snowplow integration is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_snowplow_support`. +On self-managed GitLab, by default the Snowplow integration is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_snowplow_support`. On GitLab.com, this feature is not available. This feature is not ready for production use. @@ -66,7 +66,7 @@ flowchart TB > - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flags](../../administration/feature_flags.md) named `product_analytics_dashboards` and `product_analytics_admin_settings`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flags](../../administration/feature_flags.md) named `product_analytics_dashboards` and `product_analytics_admin_settings`. On GitLab.com, this feature is not available. This feature is not ready for production use. @@ -114,7 +114,7 @@ To instrument code to collect data, use one or more of the existing SDKs: > - `product_analytics_internal_preview` replaced with `product_analytics_dashboards` in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. +On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. On GitLab.com, this feature is not available. This feature is not ready for production use. @@ -125,6 +125,29 @@ The `cube_analytics` data type connects to the Cube instance defined when [produ All filters and queries are sent to the Cube instance and the returned data is processed by the product analytics data source to be rendered by the appropriate visualizations. +### Filling missing data + +- Introduced in GitLab 16.3 behind the [feature flag](../../administration/feature_flags.md) named `product_analytics_dashboards`. Disabled by default. + +When [exporting data](#raw-data-export) or [viewing dashboards](../analytics/analytics_dashboards.md#view-project-dashboards), +if there is no data for a given day, the missing data is autofilled with `0`. + +This approach has the following benefits: + +- The visualization's day axis matches the selected date range, removing ambiguity about missing data. +- Data exports have rows for the entire date range, making data analysis easier. + +However, this approach also has the following limitations: + +- The `day` [granularity](https://cube.dev/docs/product/apis-integrations/rest-api/query-format) must be used. + All other granularities are not supported at this time. +- It only fills a date range defined by the [`inDateRange`](https://cube.dev/docs/product/apis-integrations/rest-api/query-format#indaterange) filter. + - The date selector in the UI already uses this filter. +- The filling of data ignores the query-defined limit. If you set a limit of 10 data points over 20 days, it + returns 20 data points, with the missing data filled by `0`. + +[Issue 417231](https://gitlab.com/gitlab-org/gitlab/-/issues/417231) proposes a solution to this limitation. + ## Funnel analysis Use funnel analysis to understand the flow of users through your application, and where @@ -246,3 +269,10 @@ POST /api/v4/projects/PROJECT_ID/product_analytics/request/load?queryType=multi ``` If the request is successful, the returned JSON includes an array of rows of results. + +## Troubleshooting + +### No events are collected + +Check your [instrumentation details](#enable-product-analytics), +and make sure product analytics is enabled and set up correctly. diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index c367498f66e..b27658e5e41 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -20,7 +20,7 @@ Deleting a user deletes all projects in that user namespace. > Delay between a user deleting their own account and deletion of the user record introduced in GitLab 16.0 [with a flag](../../../administration/feature_flags.md) named `delay_delete_own_user`. Enabled by default on GitLab.com. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `delay_delete_own_user`. On GitLab.com, this feature is available. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `delay_delete_own_user`. On GitLab.com, this feature is available. As a user, to delete your own account: @@ -30,9 +30,7 @@ As a user, to delete your own account: 1. Select **Delete account**. NOTE: -On GitLab.com, there is a seven day delay between a user deleting their own account and deletion of the user record. During this time, that user is [blocked](../../admin_area/moderate_users.md#block-a-user) and a new account with the same email address or username cannot be created. - -Unblocking the account does not undo the deletion because the account will still be in the deletion queue, and there is no quick method to reverse this process. +On GitLab.com, there is a seven day delay between a user deleting their own account and deletion of the user record. During this time, that user is [blocked](../../../administration/moderate_users.md#block-a-user) and a new account with the same email address or username cannot be created. Unblocking the account does not undo the deletion because the account will still be in the deletion queue, and will be deleted. Accounts with no issues, comments, notes, merge requests, or snippets are deleted immediately. Accounts under paid namespaces are deleted immediately. ## Delete users and user contributions **(FREE SELF)** @@ -71,9 +69,9 @@ When deleting users, you can either: - Personal access tokens. - Snippets. -An alternative to deleting is [blocking a user](../../admin_area/moderate_users.md#block-a-user). +An alternative to deleting is [blocking a user](../../../administration/moderate_users.md#block-a-user). -When a user is deleted from an [abuse report](../../admin_area/review_abuse_reports.md) or spam log, these associated +When a user is deleted from an [abuse report](../../../administration/review_abuse_reports.md) or spam log, these associated records are always removed. The deleting associated records option can be requested in the [API](../../../api/users.md#user-deletion) as well as @@ -91,7 +89,7 @@ ERROR: null value in column "user_id" violates not-null constraint ``` The error can be found in the [PostgreSQL log](../../../administration/logs/index.md#postgresql-logs) and -in the **Retries** section of the [background jobs view](../../admin_area/index.md#background-jobs) in the Admin Area. +in the **Retries** section of the [background jobs view](../../../administration/admin_area.md#background-jobs) in the Admin Area. If the user being deleted used the [iterations](../../group/iterations/index.md) feature, such as adding an issue to an iteration, you must use diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index b856366bb58..33888e4f06e 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -90,7 +90,7 @@ in a safe place. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212312) in GitLab 13.5 [with a flag](../../../administration/feature_flags.md) named `forti_authenticator`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per user, ask an administrator to +On self-managed GitLab, by default this feature is not available. To make it available per user, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `forti_authenticator`. On GitLab.com, this feature is not available. @@ -218,7 +218,7 @@ On your GitLab server: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212313) in GitLab 13.7 [with a flag](../../../administration/feature_flags.md) named `forti_token_cloud`. Disabled by default. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per user, ask an administrator to +On self-managed GitLab, by default this feature is not available. To make it available per user, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `forti_token_cloud`. On GitLab.com, this feature is not available. The feature is not ready for production use. @@ -277,13 +277,13 @@ Configure FortiToken Cloud in GitLab. On your GitLab server: > - Optional one-time password authentication for WebAuthn devices [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/378844) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `webauthn_without_topt`. [Enabled on GitLab.com and self-managed by default](https://gitlab.com/gitlab-org/gitlab/-/issues/232671). FLAG: -On self-managed GitLab, by default, WebAuthn devices are available. To disable the feature, ask an administrator to +On self-managed GitLab, by default, WebAuthn devices are available. To disable the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `webauthn`. If you disable the WebAuthn feature flag after WebAuthn devices have been registered, these devices are not usable until you re-enable this feature. On GitLab.com, WebAuthn devices are available. FLAG: -On self-managed GitLab, by default, optional one-time password authentication for WebAuthn devices is not available. To enable the feature, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `webauthn_without_totp`. +On self-managed GitLab, by default, optional one-time password authentication for WebAuthn devices is not available. To enable the feature, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `webauthn_without_totp`. On GitLab.com, this feature is available. WebAuthn is [supported by](https://caniuse.com/#search=webauthn) the following: @@ -465,7 +465,7 @@ a GitLab global administrator disable 2FA for your account: ## Information for GitLab administrators **(FREE SELF)** -- Take care that 2FA keeps working after [restoring a GitLab backup](../../../raketasks/backup_restore.md). +- Take care that 2FA keeps working after [restoring a GitLab backup](../../../administration/backup_restore/index.md). - To ensure 2FA authorizes correctly with a time-based one-time password (TOTP) server, synchronize your GitLab server's time using a service like NTP. Otherwise, authorization can always fail because of time differences. - The GitLab WebAuthn implementation does _not_ work when the GitLab instance is accessed from multiple hostnames @@ -501,7 +501,7 @@ This error occurs in the following scenarios: - You do not have 2FA enabled but an administrator has enabled the [enforce 2FA for all users](../../../security/two_factor_authentication.md#enforce-2fa-for-all-users) setting. - You do not have 2FA enabled, but an administrator has disabled the - [password authentication enabled for Git over HTTP(S)](../../admin_area/settings/sign_in_restrictions.md#password-authentication-enabled) + [password authentication enabled for Git over HTTP(S)](../../../administration/settings/sign_in_restrictions.md#password-authentication-enabled) setting. Instead you can authenticate: diff --git a/doc/user/profile/achievements.md b/doc/user/profile/achievements.md index 4e3248ceb10..a90144beb1b 100644 --- a/doc/user/profile/achievements.md +++ b/doc/user/profile/achievements.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w FLAG: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `achievements`. +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `achievements`. The feature is not ready for production use. Achievements are a way to reward users for their activity on GitLab. @@ -109,6 +109,23 @@ mutation achievementsCreate($file: Upload!) { } ``` +To supply the avatar file, call the mutation using `curl`: + +```shell +curl "https://gitlab.com/api/graphql" \ + -H "Authorization: Bearer <your-pat-token>" \ + -H "Content-Type: multipart/form-data" \ + -F operations='{ "query": "mutation ($file: Upload!) { achievementsCreate(input: { namespaceId: \"gid://gitlab/Namespace/<namespace-id>\", name: \"<name>\", description: \"<description>\", avatar: $file }) { achievement { id name description avatarUrl } } }", "variables": { "file": null } }' \ + -F map='{ "0": ["variables.file"] }' \ + -F 0='@/path/to/your/file.jpg' +``` + +When successful, the response returns the achievement ID: + +```shell +{"data":{"achievementsCreate":{"achievement":{"id":"gid://gitlab/Achievements::Achievement/1","name":"<name>","description":"<description>","avatarUrl":"https://gitlab.com/uploads/-/system/achievements/achievement/avatar/1/file.jpg"}}}} +``` + ## Update an achievement You can change the name, description, and avatar of an achievement at any time. diff --git a/doc/user/profile/comment_templates.md b/doc/user/profile/comment_templates.md index 4157fc852b7..a9db2d268fe 100644 --- a/doc/user/profile/comment_templates.md +++ b/doc/user/profile/comment_templates.md @@ -12,7 +12,7 @@ type: howto > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119468) in GitLab 16.0. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `saved_replies`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `saved_replies`. On GitLab.com, this feature is available. With comment templates, create and reuse text for any text area in: diff --git a/doc/user/profile/contributions_calendar.md b/doc/user/profile/contributions_calendar.md index 7adf653606e..e7f7211aeae 100644 --- a/doc/user/profile/contributions_calendar.md +++ b/doc/user/profile/contributions_calendar.md @@ -101,6 +101,6 @@ To reset your feed token: 1. On the left sidebar, select **Access Tokens**. 1. Scroll down. In the **Feed token** section, select the **reset this token** link. -1. On the confirmation box, select **OK**. +1. On the confirmation dialog, select **OK**. A new token is generated. diff --git a/doc/user/profile/index.md b/doc/user/profile/index.md index 958acaa61a9..32ea9dd2c23 100644 --- a/doc/user/profile/index.md +++ b/doc/user/profile/index.md @@ -39,7 +39,7 @@ Prerequisites: - Have a project that hosts [GitLab Pages](../project/pages/index.md). For more information, see [changing your username in the GitLab Team Handbook](https://about.gitlab.com/handbook/tools-and-tips/#change-your-username-at-gitlabcom). - Your username must be between 2 and 255 characters in length, and must not: - - Contain special characters or emojis. + - Contain special characters or emoji. - End with `.<reserved file extension>`, for example `jon.png`. However, `jonpng` is valid. To change your username: @@ -90,7 +90,7 @@ not. When visiting the public page of a user, you can only see the projects which you have privileges to. -If the [public level is restricted](../admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels), +If the [public level is restricted](../../administration/settings/visibility_and_access_controls.md#restrict-visibility-levels), user profiles are only visible to authenticated users. ## Add details to your profile with a README @@ -104,7 +104,7 @@ the README file with information, it's included on your profile page. To create a new project and add its README to your profile: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create blank project**. 1. Enter the project details: - In the **Project name** field, enter the name for your new project. @@ -250,7 +250,7 @@ Your primary email is used by default. To change your commit email: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. In the **Commit email** dropdown list, select an email address. 1. Select **Update profile settings**. @@ -261,7 +261,7 @@ Your primary email is the default email address for your login, commit email, an To change your primary email: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. In the **Email** field, enter your new email address. 1. Select **Update profile settings**. @@ -271,7 +271,7 @@ To change your primary email: You can select one of your [configured email addresses](#add-emails-to-your-user-profile) to be displayed on your public profile: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. In the **Public email** field, select one of the available email addresses. 1. Select **Update profile settings**. @@ -354,7 +354,7 @@ By default, you are signed out of GitLab after seven days (10080 minutes) of ina window, whichever comes first. GitLab administrators can -[change this default](../admin_area/settings/account_and_limit_settings.md#customize-the-default-session-duration). +[change this default](../../administration/settings/account_and_limit_settings.md#customize-the-default-session-duration). ### Stay signed in indefinitely @@ -365,7 +365,7 @@ To remain signed in indefinitely, select the **Remember me** checkbox on the Git You remain signed in because, although the server sets a session time of one week, your browser stores a secure token that enables automatic reauthentication. -GitLab administrators can [turn off the **Remember me** setting](../admin_area/settings/account_and_limit_settings.md#session-duration) for environments +GitLab administrators can [turn off the **Remember me** setting](../../administration/settings/account_and_limit_settings.md#session-duration) for environments that require sessions to expire periodically for security or compliance purposes. ### Cookies used for sign-in diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index cc0b67eed52..f378b0ae301 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -14,7 +14,7 @@ Stay informed about what's happening in GitLab with email notifications. You can receive updates about activity in issues, merge requests, epics, and designs. For the tool that GitLab administrators can use to send messages to users, read -[Email from GitLab](../admin_area/email_from_gitlab.md). +[Email from GitLab](../../administration/email_from_gitlab.md). ## Who receives notifications @@ -46,7 +46,7 @@ anyone else. To edit your notification settings: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Edit the desired global, group, or project notification settings. @@ -99,7 +99,7 @@ You can select a notification level and email address for each group. To select a notification level for a group, use either of these methods: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Locate the project in the **Groups** section. @@ -118,7 +118,7 @@ Or: You can select an email address to receive notifications for each group you belong to. You can use group notifications, for example, if you work freelance, and want to keep email about clients' projects separate. -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Locate the project in the **Groups** section. @@ -130,7 +130,7 @@ To help you stay up to date, you can select a notification level for each projec To select a notification level for a project, use either of these methods: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Locate the project in the **Projects** section. @@ -152,7 +152,7 @@ These emails are enabled by default. To opt out: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Clear the **Receive product marketing emails** checkbox. @@ -290,7 +290,7 @@ To always receive notifications on your own issues, merge requests, and so on, t NOTE: This feature is enabled by default for self-managed instances. Administrators may disable this feature -through the [Sign-in restrictions](../admin_area/settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins) section of the UI. +through the [Sign-in restrictions](../../administration/settings/sign_in_restrictions.md#email-notification-for-unknown-sign-ins) section of the UI. The feature is always enabled on GitLab.com. When a user successfully signs in from a previously unknown IP address or device, @@ -335,7 +335,7 @@ The participants are: If you no longer wish to receive any email notifications: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. On the left sidebar, select **Notifications**. 1. Clear the **Receive product marketing emails** checkbox. @@ -345,7 +345,7 @@ If you no longer wish to receive any email notifications: **Disabled**. On self-managed installations, even after doing this, your instance administrator -[can still email you](../admin_area/email_from_gitlab.md). +[can still email you](../../administration/email_from_gitlab.md). To unsubscribe, select the unsubscribe link in one of these emails. ## Email headers you can use to filter email diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index bda92ce8388..a8231460045 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -48,7 +48,7 @@ Use impersonation tokens to automate authentication as a specific user. You can create as many personal access tokens as you like. -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Access Tokens**. 1. Enter a name and expiry date for the token. @@ -79,17 +79,15 @@ for guidance on managing personal access tokens (for example, setting a short ex At any time, you can revoke a personal access token. -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Access Tokens**. 1. In the **Active personal access tokens** area, next to the key, select **Revoke**. ## View the last time a token was used -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414945) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `update_personal_access_token_usage_information_every_10_minutes`. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `update_personal_access_token_usage_information_every_10_minutes`. On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33162) in GitLab 13.2. Token usage information is updated every 24 hours. +> - The frequency of token usage information updates [changed](https://gitlab.com/gitlab-org/gitlab/-/issues/410168) in GitLab 16.1 from 24 hours to 10 minutes. Token usage information is updated every 10 minutes. GitLab considers a token used when the token is used to: @@ -98,7 +96,7 @@ Token usage information is updated every 10 minutes. GitLab considers a token us To view the last time a token was used: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select **Access Tokens**. 1. In the **Active personal access tokens** area, next to the key, view the **Last Used** date. @@ -119,7 +117,8 @@ A personal access token can perform actions based on the assigned scopes. | `read_registry` | Grants read-only (pull) access to a [Container Registry](../packages/container_registry/index.md) images if a project is private and authorization is required. Available only when the Container Registry is enabled. | | `write_registry` | Grants read-write (push) access to a [Container Registry](../packages/container_registry/index.md) images if a project is private and authorization is required. Available only when the Container Registry is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28958) in GitLab 12.10.) | | `sudo` | Grants permission to perform API actions as any user in the system, when authenticated as an administrator. | -| `admin_mode` | Grants permission to perform API actions as an administrator, when Admin Mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107875) in GitLab 15.8.) | +| `admin_mode` | Grants permission to perform API actions as an administrator, when Admin Mode is enabled. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/107875) in GitLab 15.8.) | +| `create_runner` | Grants permission to create runners. | WARNING: If you enabled [external authorization](../admin_area/settings/external_authorization.md), personal access tokens cannot access container or package registries. If you use personal access tokens to access these registries, this measure breaks this use of these tokens. Disable external authorization to use personal access tokens with container or package registries. @@ -131,7 +130,8 @@ Personal access tokens expire on the date you define, at midnight UTC. - GitLab runs a check at 01:00 AM UTC every day to identify personal access tokens that expire in the next seven days. The owners of these tokens are notified by email. - GitLab runs a check at 02:00 AM UTC every day to identify personal access tokens that expire on the current date. The owners of these tokens are notified by email. - In GitLab Ultimate, administrators can - [limit the lifetime of access tokens](../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens). + [limit the allowable lifetime of access tokens](../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens). If not set, the maximum allowable lifetime of a personal access token is 365 days. +- In GitLab Free and Premium, the maximum allowable lifetime of a personal access token is 365 days. ## Create a personal access token programmatically **(FREE SELF)** @@ -157,11 +157,11 @@ To create a personal access token programmatically: The token must be 20 characters long. The scopes must be valid and are visible [in the source code](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/auth.rb). - For example, to create a token that belongs to a user with username `automation-bot`: + For example, to create a token that belongs to a user with username `automation-bot` and expires in a year: ```ruby user = User.find_by_username('automation-bot') - token = user.personal_access_tokens.create(scopes: ['read_user', 'read_repository'], name: 'Automation token') + token = user.personal_access_tokens.create(scopes: ['read_user', 'read_repository'], name: 'Automation token', expires_at: 365.days.from_now) token.set_token('token-string-here123') token.save! ``` @@ -170,7 +170,7 @@ This code can be shortened into a single-line shell command by using the [Rails runner](../../administration/operations/rails_console.md#using-the-rails-runner): ```shell -sudo gitlab-rails runner "token = User.find_by_username('automation-bot').personal_access_tokens.create(scopes: ['read_user', 'read_repository'], name: 'Automation token'); token.set_token('token-string-here123'); token.save!" +sudo gitlab-rails runner "token = User.find_by_username('automation-bot').personal_access_tokens.create(scopes: ['read_user', 'read_repository'], name: 'Automation token', expires_at: 365.days.from_now); token.set_token('token-string-here123'); token.save!" ``` ## Revoke a personal access token programmatically **(FREE SELF)** diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index bcfe323a5fe..17dea99e5ef 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -12,14 +12,13 @@ of GitLab to their liking. To navigate to your profile's preferences: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. ## Navigation theme The GitLab navigation theme setting allows you to personalize your GitLab experience. -You can choose from several color themes that add unique colors to the top navigation -and left side navigation. +You can choose from several color themes that add unique colors to the left sidebar. Using individual color themes might help you differentiate between your different GitLab instances. @@ -158,7 +157,7 @@ You can choose one of the following options as the first day of the week: - Sunday - Monday -If you select **System Default**, the [instance default](../admin_area/settings/index.md#default-first-day-of-the-week) setting is used. +If you select **System Default**, the [instance default](../../administration/settings/index.md#default-first-day-of-the-week) setting is used. ## Time preferences diff --git a/doc/user/profile/user_passwords.md b/doc/user/profile/user_passwords.md index 50624a43893..c57a81c00bf 100644 --- a/doc/user/profile/user_passwords.md +++ b/doc/user/profile/user_passwords.md @@ -67,7 +67,7 @@ By default GitLab enforces the following password requirements: Self-managed installations can configure the following additional password requirements: - [Password minimum and maximum length limits](../../security/password_length_limits.md). -- [Password complexity requirements](../admin_area/settings/sign_up_restrictions.md#password-complexity-requirements). +- [Password complexity requirements](../../administration/settings/sign_up_restrictions.md#password-complexity-requirements). ## Block weak passwords diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 8ea762777ac..c4ef6a647db 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -121,7 +121,7 @@ or by sending requests to the [GraphQL API](../../api/graphql/getting_started.md To use your [deploy board](../../user/project/deploy_boards.md): -1. Go to **Deployments > Environments** for your project. +1. Go to **Operate > Environments** for your project. 1. Set the new weight with the dropdown list on the right side. 1. Confirm your selection. @@ -134,7 +134,7 @@ Here's an example using [GraphiQL](../../api/graphql/getting_started.md#graphiql mutation { environmentsCanaryIngressUpdate(input:{ id: "gid://gitlab/Environment/29", # Your Environment ID. You can get the ID from the URL of the environment page. - weight: 45 # The new traffic weight. e.g. If you set `45`, 45% of traffic goes to a canary deployment and 55% of traffic goes to a stable deployment. + weight: 45 # The new traffic weight. for example, If you set `45`, 45% of traffic goes to a canary deployment and 55% of traffic goes to a stable deployment. }) { errors } diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index 87554e786ab..9a30cbe94e2 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -54,7 +54,7 @@ To create new a EKS cluster for your project, group, or instance, through cluster certificates: 1. Go to your: - - Project's **Infrastructure > Kubernetes clusters** page, for a project-level cluster. + - Project's **Operate > Kubernetes clusters** page, for a project-level cluster. - Group's **Kubernetes** page, for a group-level cluster. - **Main menu > Admin > Kubernetes**, for an instance-level cluster. 1. Select **Integrate with a cluster certificate**. @@ -213,7 +213,7 @@ Otherwise, the deployed app isn't externally available outside of the cluster. GitLab creates a new pipeline, which begins to build, test, and deploy the app. After the pipeline has finished, your app runs in EKS, and is available -to users. Select **CI/CD > Environments**. +to users. Select **Operate > Environments**.  diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index bb85662d442..aaaa72d282e 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -66,7 +66,7 @@ to grant access. To add a Kubernetes cluster to your project, group, or instance: 1. Navigate to your: - 1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. + 1. Project's **{cloud-gear}** **Operate > Kubernetes clusters** page, for a project-level cluster. 1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. 1. **Main menu > Admin > Kubernetes** page, for an instance-level cluster. 1. On the **Kubernetes clusters** page, select the **Connect with a certificate** option from the **Actions** dropdown list. diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index c6561fffa0b..09dbd70d1bb 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -60,7 +60,7 @@ To create new Kubernetes clusters to your project, group, or instance, through cluster certificates: 1. Navigate to your: - - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level + - Project's **{cloud-gear}** **Operate > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - **Main menu > Admin > Kubernetes** page, for an instance-level cluster. diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index bba05f1e724..470daf499be 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -17,7 +17,7 @@ To create and manage a new cluster use [Infrastructure as Code](../../infrastruc When you successfully connect an existing cluster using cluster certificates, the cluster connection to GitLab becomes enabled. To disable it: 1. Go to your: - - Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. + - Project's **{cloud-gear}** **Operate > Kubernetes clusters** page, for a project-level cluster. - Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. - **Main menu > Admin > Kubernetes** page, for an instance-level cluster. 1. Select the name of the cluster you want to disable. diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index b321646d8d8..537b38dd547 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -17,7 +17,7 @@ To connect your cluster to GitLab, use the [GitLab agent](../../../user/clusters To manage applications, use the [Cluster Project Management Template](../../../user/clusters/management_project_template.md). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `certificate_based_clusters`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `certificate_based_clusters`. You can choose to allow GitLab to manage your cluster for you. If your cluster is managed by GitLab, resources for your projects are automatically created. See @@ -45,7 +45,7 @@ your cluster. This can cause deployment jobs to fail. To clear the cache: -1. Navigate to your project's **Infrastructure > Kubernetes clusters** page, and select your cluster. +1. Navigate to your project's **Operate > Kubernetes clusters** page, and select your cluster. 1. Expand the **Advanced settings** section. 1. Select **Clear cluster cache**. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 09a443614d0..8f64fe7dd7d 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -24,7 +24,7 @@ This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/e to add this functionality to the [agent](../index.md). FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `certificate_based_clusters`. GitLab deploy boards offer a consolidated view of the current health and status of each CI [environment](../../ci/environments/index.md) running on [Kubernetes](https://kubernetes.io), displaying the status @@ -127,7 +127,7 @@ To display the deploy boards for a specific [environment](../../ci/environments/  Once all of the above are set up and the pipeline has run at least once, -navigate to the environments page under **Deployments > Environments**. +go to the environments page under **Operate > Environments**. Deploy boards are visible by default. You can explicitly select the triangle next to their respective environment name to hide them. diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index 5bd19fec0ba..4e380d485a8 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -40,7 +40,6 @@ A deploy key is given a permission level when it is created: You can change a deploy key's permission level after creating it. Changing a project deploy key's permissions only applies for the current project. -Although a deploy key is a secret that isn't associated with a specific user, GitLab authorizes the creator of the deploy key if the Git-command triggers additional processes. For example: - When a deploy key is used to push a commit to a [protected branch](../protected_branches.md), @@ -58,6 +57,9 @@ For human interactions, use credentials tied to users such as Personal Access To To help detect a potential secret leak, you can use the [Audit Event](../../../administration/audit_event_streaming/examples.md#example-payloads-for-ssh-events-with-deploy-key) feature. +WARNING: +Deploy keys work even if the user who created them is removed from the group or project. + ## View deploy keys To view the deploy keys available to a project: @@ -128,6 +130,20 @@ To grant a public deploy key access to a project: 1. In the key's row, select **Edit** (**{pencil}**). 1. Select the **Grant write permissions to this key** checkbox. +### Edit project access permissions of a deploy key + +Prerequisites: + +- You must have at least the Maintainer role for the project. + +To edit the project access permissions of a deploy key: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Repository**. +1. Expand **Deploy keys**. +1. In the key's row, select **Edit** (**{pencil}**). +1. Select or clear the **Grant write permissions to this key** checkbox. + ## Revoke project access of a deploy key To revoke a deploy key's access to a project, you can disable it. Any service that relies on @@ -159,8 +175,10 @@ What happens to the deploy key when it is disabled depends on the following: There are a few scenarios where a deploy key fails to push to a [protected branch](../protected_branches.md). -- The owner associated to a deploy key does not have access to the protected branch. - The owner associated to a deploy key does not have [membership](../members/index.md) to the project of the protected branch. +- The owner associated to a deploy key has [project membership permissions](../../../user/permissions.md#project-members-permissions) lower than required to **View project code**. +- The deploy key does not have [read-write permissions for the project](#edit-project-access-permissions-of-a-deploy-key). +- The deploy key has been [revoked](#revoke-project-access-of-a-deploy-key). - **No one** is selected in [the **Allowed to push and merge** section](../protected_branches.md#add-protection-to-existing-branches) of the protected branch. All deploy keys are associated to an account. Since the permissions for an account can change, this might lead to scenarios where a deploy key that was working is suddenly unable to push to a protected branch. diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 11e538964a2..3fb338a75ec 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -16,7 +16,7 @@ You might want to use these templates: - For different stages of your workflow, for example, feature proposal, feature improvement, or a bug report. - For every issue or merge request for a specific project, so the layout is consistent. -- For a [Service Desk email template](service_desk.md#use-a-custom-template-for-service-desk-issues). +- For a [Service Desk email template](service_desk.md#use-a-custom-template-for-service-desk-tickets). For description templates to work, they must be: diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 71b4bff41d1..4c77323778c 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -147,7 +147,7 @@ permissions to the project: git lfs unlock --id=123 --force ``` -You can normally push files to GitLab whether they're locked or unlocked. +You can push files to GitLab whether they're locked or unlocked. NOTE: Although multi-branch file locks can be created and managed through the Git LFS @@ -209,11 +209,13 @@ To lock a file: 1. Open the file or directory in GitLab. 1. In the upper-right corner, above the file, select **Lock**. -1. On the confirmation dialog box, select **OK**. +1. On the confirmation dialog, select **OK**. If you do not have permission to lock the file, the button is not enabled. -To view the user who locked the file (if it was not you), hover over the button. +To view the user who locked a directory (if it was not you), hover over the button. Reinstatement of +similar functionality for locked files is discussed in +[issue 376222](https://gitlab.com/gitlab-org/gitlab/-/issues/376222). ### View and remove existing locks diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index dc17a3646a8..b957ff93a4c 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -36,7 +36,7 @@ When importing: - [Bitbucket Cloud integration](../../../integration/bitbucket.md) must be enabled. If that integration is not enabled, ask your GitLab administrator to enable it. The Bitbucket Cloud integration is enabled by default on GitLab.com. -- [Bitbucket Cloud import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your +- [Bitbucket Cloud import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The Bitbucket Cloud import source is enabled by default on GitLab.com. - At least the Maintainer role on the destination group to import to. @@ -70,7 +70,7 @@ For user contributions to be mapped, each user must complete the following befor > Ability to re-import projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23905) in GitLab 15.9. 1. Sign in to GitLab. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Import project**. 1. Select **Bitbucket Cloud**. 1. Log in to Bitbucket and grant GitLab access to your Bitbucket account. diff --git a/doc/user/project/import/bitbucket_server.md b/doc/user/project/import/bitbucket_server.md index fce9ca7d147..6226e4c1296 100644 --- a/doc/user/project/import/bitbucket_server.md +++ b/doc/user/project/import/bitbucket_server.md @@ -31,7 +31,7 @@ You can import Bitbucket repositories to GitLab. > Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. -- [Bitbucket Server import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [Bitbucket Server import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The Bitbucket Server import source is enabled by default on GitLab.com. - At least the Maintainer role on the destination group to import to. @@ -41,7 +41,7 @@ You can import Bitbucket repositories to GitLab. To import your Bitbucket repositories: 1. Sign in to GitLab. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Import project**. 1. Select **Bitbucket Server**. 1. Log in to Bitbucket and grant GitLab access to your Bitbucket account. @@ -93,7 +93,7 @@ repository imports under the namespace of the user who started the import proces FLAG: On self-managed GitLab and GitLab.com, by default this feature is not available. To make it -available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) +available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `bitbucket_server_user_mapping_by_username`. This feature is not ready for production use. With this feature enabled, the importer tries to find a user in the GitLab user database with the diff --git a/doc/user/project/import/fogbugz.md b/doc/user/project/import/fogbugz.md index 30c4249e0d6..18758ba99e6 100644 --- a/doc/user/project/import/fogbugz.md +++ b/doc/user/project/import/fogbugz.md @@ -19,7 +19,7 @@ users. > Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. -- [FogBugz import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [FogBugz import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The FogBugz import source is enabled by default on GitLab.com. - At least the Maintainer role on the destination group to import to. @@ -29,7 +29,7 @@ users. To import your project from FogBugz: 1. Sign in to GitLab. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Import project**. 1. Select **FogBugz**. 1. Enter your FogBugz URL, email address, and password. diff --git a/doc/user/project/import/gitea.md b/doc/user/project/import/gitea.md index 62cda62c2fe..22c89084c56 100644 --- a/doc/user/project/import/gitea.md +++ b/doc/user/project/import/gitea.md @@ -32,7 +32,7 @@ on the issue about the original Gitea author. > Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. - Gitea version 1.0.0 or later. -- [Gitea import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [Gitea import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The Gitea import source is enabled by default on GitLab.com. - At least the Maintainer role on the destination group to import to. diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 509029a3099..afc36f309be 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -35,7 +35,7 @@ When importing projects: - The organization the repository belongs to must not impose restrictions of a [third-party application access policy](https://docs.github.com/en/organizations/managing-oauth-access-to-your-organizations-data/about-oauth-app-access-restrictions) on the GitLab instance you import to. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For an overview of the import process, see [Migrating from GitHub to GitLab](https://youtu.be/VYOXuOg9tQI). +For an overview of the import process, see [How to migrate from GitHub to GitLab including Actions](https://www.youtube.com/watch?v=0Id5oMl1Kqs). ## Prerequisites @@ -43,7 +43,7 @@ For an overview of the import process, see [Migrating from GitHub to GitLab](htt To import projects from GitHub: -- [GitHub import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [GitHub import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The GitHub import source is enabled by default on GitLab.com. - You must have at least the Maintainer role on the destination group to import to. @@ -68,7 +68,7 @@ If you are importing from GitHub Enterprise to a self-managed GitLab instance: - You must first enable the [GitHub integration](../../../integration/github.md). - GitHub must be enabled as an import source in the - [Admin Area](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). + [Admin Area](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources). - For GitLab 15.10 and earlier, you must add `github.com` and `api.github.com` entries in the [allowlist for local requests](../../../security/webhooks.md#allow-outbound-requests-to-certain-ip-addresses-and-domains). @@ -78,7 +78,7 @@ If you are importing from GitHub.com to a self-managed GitLab instance: - You don't need to enable the [GitHub integration](../../../integration/github.md). - GitHub must be enabled as an import source in the - [Admin Area](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources). + [Admin Area](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources). ## Import your GitHub repository into GitLab @@ -96,9 +96,11 @@ NOTE: If you are using a self-managed GitLab instance or if you are importing from GitHub Enterprise, this process requires that you have configured [GitHub integration](../../../integration/github.md). -1. From the top navigation bar, select **+** and select **New project**. -1. Select the **Import project** tab and then select **GitHub**. -1. Select the first button to **List your GitHub repositories**. You are redirected to a page on [GitHub](https://github.com) to authorize the GitLab application. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. +1. Select **Import project** and then **GitHub**. +1. Now you can either: + - Add a personal access token and select **Authenticate**. + - If GitHub is [configured](../../../integration/github.md) for the instance, select **Authorize with GitHub**. 1. Select **Authorize GitlabHQ**. You are redirected back to the GitLab Import page and all of your GitHub repositories are listed. 1. Continue on to [selecting which repositories to import](#select-which-repositories-to-import). diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index 9fdb8eed5aa..6f9c64b73cb 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -92,7 +92,7 @@ over a series of Docker pulls and pushes. Re-run any CI pipelines to retrieve an ## Migrate between two self-managed GitLab instances To migrate from an existing self-managed GitLab instance to a new self-managed GitLab instance, -you should [back up](../../../raketasks/backup_restore.md) +you should [back up](../../../administration/backup_restore/index.md) the existing instance and restore it on the new instance. For example, you could use this method to migrate a self-managed instance from an old server to a new server. The backups produced don't depend on the operating system running GitLab. You can therefore use @@ -114,7 +114,7 @@ You can view all project imports created by you. This list includes the followin To view project import history: 1. Sign in to GitLab. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Import project**. 1. In the upper-right corner, select **History**. 1. If there are any errors for a particular import, you can see them by selecting **Details**. diff --git a/doc/user/project/import/manifest.md b/doc/user/project/import/manifest.md index 7b3550d2dc9..980c051eac7 100644 --- a/doc/user/project/import/manifest.md +++ b/doc/user/project/import/manifest.md @@ -19,7 +19,7 @@ repositories like the Android Open Source Project (AOSP). > Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. -- [Manifest import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [Manifest import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The Manifest import source is enabled by default on GitLab.com. - GitLab must use PostgreSQL for its database, because [subgroups](../../group/subgroups/index.md) are needed for the manifest import diff --git a/doc/user/project/import/repo_by_url.md b/doc/user/project/import/repo_by_url.md index 230113581f8..c3c516042f7 100644 --- a/doc/user/project/import/repo_by_url.md +++ b/doc/user/project/import/repo_by_url.md @@ -12,7 +12,7 @@ You can import your existing repositories by providing the Git URL. > Requirement for Maintainer role instead of Developer role introduced in GitLab 16.0 and backported to GitLab 15.11.1 and GitLab 15.10.5. -- [Repository by URL import source](../../admin_area/settings/visibility_and_access_controls.md#configure-allowed-import-sources) +- [Repository by URL import source](../../../administration/settings/visibility_and_access_controls.md#configure-allowed-import-sources) must be enabled. If not enabled, ask your GitLab administrator to enable it. The Repository by URL import source is enabled by default on GitLab.com. - At least the Maintainer role on the destination group to import to. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 67b96efe9a4..b7bdf47ae49 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -12,7 +12,7 @@ You can create a project in many ways in GitLab. To create a blank project: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create blank project**. 1. Enter the project details: - In the **Project name** field, enter the name of your project. The name must start with a lowercase or uppercase letter (`a-zA-Z`), digit (`0-9`), emoji, or underscore (`_`). It can also contain dots (`.`), pluses (`+`), dashes (`-`), or spaces. @@ -41,7 +41,7 @@ Anyone can [contribute a built-in template](../../development/project_templates. To create a project from a built-in template: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. From the list of templates: @@ -68,10 +68,10 @@ For this reason, the creation date of imported objects can be older than the cre Custom project templates are available at: -- The [instance-level](../../user/admin_area/custom_project_templates.md) +- The [instance-level](../../administration/custom_project_templates.md) - The [group-level](../../user/group/custom_project_templates.md) -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create from template**. 1. Select the **Instance** or **Group** tab. 1. From the list of templates: @@ -96,7 +96,7 @@ HIPAA Audit Protocol published by the U.S Department of Health and Human Service To create a project from the HIPAA Audit Protocol template: -1. On the left sidebar, at the top, select **Create new...** (**{plus}**) and **New project/repository**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. Locate the **HIPAA Audit Protocol** template: diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 5ca6fcebdd6..533824dd58a 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -28,6 +28,13 @@ To view project insights: 1. Select **Analyze > Insights**. 1. To view a report, select the **Select report** dropdown list. +### Access Insights reports with deep links + +You can direct users to a specific report in Insights by using the deep-linked URL. + +To create a deep link, append the report key to the end of the Insights report URL. +For example, a GitLab report with the key `bugsCharts` has the deep link URL `https://gitlab.com/gitlab-org/gitlab/insights/#/bugsCharts`. + ## Configure project insights Prerequisites: diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 8790c7213e5..bfa8a905699 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -4,95 +4,110 @@ group: Import and Integrate info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# GitLab for Slack app **(FREE SAAS)** +# GitLab for Slack app **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/358872) for self-managed instances in GitLab 16.2. NOTE: -This feature is only configurable on GitLab.com. -For self-managed GitLab instances, use -[Slack slash commands](slack_slash_commands.md) and [Slack notifications](slack.md) instead. -For more information about our plans to make this feature configurable for all GitLab installations, -see [epic 1211](https://gitlab.com/groups/gitlab-org/-/epics/1211). +This page contains information about configuring the GitLab for Slack app on GitLab.com. For administrator documentation, see [GitLab for Slack app administration](../../admin_area/settings/slack_app.md). + +The GitLab for Slack app is a native Slack app that provides [slash commands](#slash-commands) and [notifications](#slack-notifications) +in your Slack workspace. GitLab links your Slack user with your GitLab user so that any command +you run in Slack is run by your linked GitLab user. + +## Install the GitLab for Slack app -Slack provides a native application that you can enable with your project's -integrations on GitLab.com. +Prerequisites: -## Installation +- You must have the [appropriate permissions to add apps to your Slack workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). +- On self-managed GitLab, an administrator must [enable the integration](../../admin_area/settings/slack_app.md). In GitLab 15.0 and later, the GitLab for Slack app uses [granular permissions](https://medium.com/slack-developer-blog/more-precision-less-restrictions-a3550006f9c3). Although functionality has not changed, you should [reinstall the app](#update-the-gitlab-for-slack-app). -### Through the Slack App Directory +### From project integration settings -To enable the GitLab for Slack app for your workspace, -install the [GitLab application](https://slack-platform.slack.com/apps/A676ADMV5-gitlab) -from the [Slack App Directory](https://slack.com/apps). +To install the GitLab for Slack app from project integration settings: -On the [GitLab for Slack app landing page](https://gitlab.com/-/profile/slack/edit), -you can select a GitLab project to link with your Slack workspace. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Integrations**. +1. Select **GitLab for Slack app**. +1. Select **Install GitLab for Slack app**. +1. On the Slack confirmation page, select **Allow**. -### Through GitLab project settings +To update the app in your Slack workspace to the latest version, +you can also select **Reinstall GitLab for Slack app**. -Alternatively, you can configure the GitLab for Slack app with your project's -integration settings. +### From the Slack App Directory **(FREE SAAS)** -You must have the appropriate permissions for your Slack -workspace to be able to install a new application. See -[Add apps to your Slack workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). +On GitLab.com, you can also install the GitLab for Slack app from the +[Slack App Directory](https://slack-platform.slack.com/apps/A676ADMV5-gitlab). -To enable the GitLab integration for your Slack workspace: +To install the GitLab for Slack app from the Slack App Directory: -1. Go to your project's **Settings > Integration > GitLab for Slack app** (only - visible on GitLab.com). -1. Select **Install GitLab for Slack app**. -1. Select **Allow** on Slack's confirmation screen. +1. Go to the [GitLab for Slack page](https://gitlab.com/-/profile/slack/edit). +1. Select a GitLab project to link with your Slack workspace. -To update the app in your Slack workspace to the latest version, -you can also select **Reinstall GitLab for Slack app**. +## Update the GitLab for Slack app + +When GitLab releases new features for the GitLab for Slack app, you might have to manually update your app to use the new features. + +To update your GitLab for Slack app: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find a project for + which the GitLab for Slack app is configured. +1. Select **Settings > Integrations**. +1. Select **GitLab for Slack app**. +1. Select **Reinstall GitLab for Slack app**. + +The GitLab for Slack app is updated for all projects that use the integration. + +Alternatively, you can [configure the integration](https://about.gitlab.com/solutions/slack/) again. ## Slash commands -After installing the GitLab for Slack app, everyone in your Slack workspace can use slash commands. +You can use slash commands to run common GitLab operations. Replace `<project>` with a project full path. + +**For the GitLab for Slack app**: + +- You must authorize your Slack user on GitLab.com when you run your first slash command. +- You can [create a shorter project alias](#create-a-project-alias-for-slash-commands) for slash commands. + +**For [Slack slash commands](slack_slash_commands.md) on self-managed GitLab, [Mattermost slash commands](mattermost_slash_commands.md), and [ChatOps](../../../ci/chatops/index.md)**, replace `/gitlab` with the slash command trigger name configured for your integration. -Replace `<project>` with the project full path, or create a shorter [project alias](#create-a-project-alias-for-slash-commands) for the slash commands. +The following slash commands are available: -| Command | Effect | -| ------- | ------ | +| Command | Description | +| ------- | ----------- | | `/gitlab help` | Shows all available slash commands. | -| `/gitlab <project> issue new <title> <shift+return> <description>` | Creates a new issue with the title `<title>` and description `<description>`. | +| `/gitlab <project> issue new <title>` <kbd>Shift</kbd>+<kbd>Enter</kbd> `<description>` | Creates a new issue with the title `<title>` and description `<description>`. | | `/gitlab <project> issue show <id>` | Shows the issue with the ID `<id>`. | | `/gitlab <project> issue close <id>` | Closes the issue with the ID `<id>`. | -| `/gitlab <project> issue search <query>` | Shows up to 5 issues matching `<query>`. | +| `/gitlab <project> issue search <query>` | Shows up to five issues matching `<query>`. | | `/gitlab <project> issue move <id> to <project>` | Moves the issue with the ID `<id>` to `<project>`. | -| `/gitlab <project> issue comment <id> <shift+return> <comment>` | Adds a new comment with the comment body `<comment>` to the issue with the ID `<id>`. | +| `/gitlab <project> issue comment <id>` <kbd>Shift</kbd>+<kbd>Enter</kbd> `<comment>` | Adds a new comment with the comment body `<comment>` to the issue with the ID `<id>`. | | `/gitlab <project> deploy <from> to <to>` | [Deploys](#the-deploy-slash-command) from the `<from>` environment to the `<to>` environment. | | `/gitlab <project> run <job name> <arguments>` | Executes the [ChatOps](../../../ci/chatops/index.md) job `<job name>` on the default branch. | -| `/gitlab incident declare` | **Beta** Opens modal to [create a new incident](../../../operations/incident_management/slack.md). | +| `/gitlab incident declare` | Opens a dialog to [create a new incident from Slack](../../../operations/incident_management/slack.md) (Beta). | -### The deploy slash command +### The `deploy` slash command -To deploy to an environment, GitLab tries to find a deployment -manual action in the pipeline. +To deploy to an environment, GitLab tries to find a deployment manual action in the pipeline. -If there's only one action for a given environment, it is triggered. -If more than one action is defined, GitLab finds an action -name that matches the environment name to deploy to. +If only one action is defined for a given environment, it is triggered. +If more than one action is defined, GitLab tries to find an action name +that matches the environment name to deploy to. The command returns an error if no matching action is found. -### User authorization - -When you perform your first slash command, you must -authorize your Slack user on GitLab.com. - ### Create a project alias for slash commands -By default, slash commands expect a project full path. To use a shorter alias -instead: +By default, slash commands expect a project full path. To create a shorter project alias in the GitLab for Slack app: -1. Go to your project's home page. -1. Go to **Settings > Integrations** (only visible on GitLab.com). -1. On the **Integrations** page, select **GitLab for Slack app**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Settings > Integrations**. +1. Select **GitLab for Slack app**. 1. The current **Project Alias**, if any, is displayed. To edit this value, select **Edit**. 1. Enter your desired alias, and select **Save changes**. @@ -101,19 +116,19 @@ instead: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/381012) in GitLab 15.9. -With Slack notifications, GitLab can send messages to Slack workspace channels for certain GitLab [events](#events-for-slack-notifications) (for example, when an issue is created). +With Slack notifications, GitLab can send messages to Slack workspace channels for certain GitLab [events](#notification-events). ### Configure notifications To configure Slack notifications: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find a project for - which the GitLab for Slack app has been [installed](#installation). + which the GitLab for Slack app is [installed](#install-the-gitlab-for-slack-app). 1. Select **Settings > Integrations**. 1. Select **GitLab for Slack app**. 1. In the **Trigger** section, select the checkbox for each GitLab event you want to receive a notification for in Slack. For a full list, see - [Events for Slack notifications](#events-for-slack-notifications). + [Notification events](#notification-events). 1. For each checkbox you select, enter the name of the channel that receives the notifications (for example, `#my-channel`). - To send notifications to multiple Slack channels, enter up to 10 channel names separated by commas (for example, `#channel-one, #channel-two`). @@ -136,7 +151,7 @@ To receive notifications to a private Slack channel, you must add the GitLab for 1. Mention the app in the channel by typing `@GitLab` and pressing <kbd>Enter</kbd>. 1. Select **Add to Channel**. -### Events for Slack notifications +### Notification events The following events are available for Slack notifications: @@ -153,25 +168,17 @@ The following events are available for Slack notifications: | **Wiki page** | A wiki page is created or updated. | | **Deployment** | A deployment starts or finishes. | | **Alert** | A new, unique alert is recorded. | +| **Group mention in public** | A group is mentioned in a public context. | +| **Group mention in private** | A group is mentioned in a confidential context. | | [**Vulnerability**](../../application_security/vulnerabilities/index.md) | A new, unique vulnerability is recorded. | ## Troubleshooting -### Update the GitLab for Slack app - -New releases of the app might require permissions to be authorized before some features work in your Slack workspace. You should ensure the app is up to date in your Slack workspace to enjoy all the latest features. +### GitLab for Slack app does not appear in the list of integrations -To update your GitLab for Slack app: - -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find a project for - which the GitLab for Slack app has been configured. -1. Select **Settings > Integrations**. -1. Select **GitLab for Slack app**. -1. Select **Reinstall GitLab for Slack app**. - -The GitLab for Slack app is updated for all projects that use the integration. +The GitLab for Slack app might not appear in the list of integrations. To have the GitLab for Slack app on your self-managed instance, an administrator must [enable the integration](../../admin_area/settings/slack_app.md). On GitLab.com, the GitLab for Slack app is available by default. -Alternatively, you can [configure a new Slack integration](https://about.gitlab.com/solutions/slack/). +The GitLab for Slack app is enabled at the project level only. Support for the app at the group and instance levels is proposed in [issue 391526](https://gitlab.com/gitlab-org/gitlab/-/issues/391526). ### Project or alias not found @@ -186,7 +193,11 @@ As a workaround, ensure: - The project full path is correct. - If using a [project alias](#create-a-project-alias-for-slash-commands), the alias is correct. -- The GitLab for Slack app integration is [enabled for the project](#through-gitlab-project-settings). +- The GitLab for Slack app is [enabled for the project](#from-project-integration-settings). + +### Slash commands return `/gitlab failed with the error "dispatch_failed"` in Slack + +Slash commands might return `/gitlab failed with the error "dispatch_failed"` in Slack. To resolve this issue, ensure an administrator has properly configured the [GitLab for Slack app settings](../../admin_area/settings/slack_app.md) on your self-managed instance. ### Notifications are not received to a channel diff --git a/doc/user/project/integrations/index.md b/doc/user/project/integrations/index.md index c70a58a24d2..fd59f54a066 100644 --- a/doc/user/project/integrations/index.md +++ b/doc/user/project/integrations/index.md @@ -72,7 +72,6 @@ You can configure the following integrations. | Packagist | Keep your PHP dependencies updated on Packagist. | **{check-circle}** Yes | | [Pipelines emails](pipeline_status_emails.md) | Send the pipeline status to a list of recipients by email. | **{dotted-circle}** No | | [Pivotal Tracker](pivotal_tracker.md) | Add commit messages as comments to Pivotal Tracker stories. | **{dotted-circle}** No | -| [Prometheus](prometheus.md) | Monitor application metrics. | **{dotted-circle}** No | | [Pumble](pumble.md) | Send event notifications to a Pumble channel. | **{dotted-circle}** No | | Pushover | Get real-time notifications on your device. | **{dotted-circle}** No | | [Redmine](redmine.md) | Use Redmine as the issue tracker. | **{dotted-circle}** No | diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index a8d8323a651..6a09a1cfbcd 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -6,12 +6,14 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Mattermost slash commands **(FREE)** -You can use slash commands to run common GitLab operations, like creating an issue, -from a [Mattermost](https://mattermost.com/) chat environment. +You can use [slash commands](gitlab_slack_application.md#slash-commands) to run common GitLab operations, +like creating an issue, from a [Mattermost](https://mattermost.com/) chat environment. GitLab can also send events (such as `issue created`) to Mattermost as part of the separately configured [Mattermost notifications](mattermost.md). +For a list of available slash commands, see [Slash commands](gitlab_slack_application.md#slash-commands). + ## Configuration options GitLab provides different ways to configure Mattermost slash commands. For any of these options, @@ -109,7 +111,7 @@ Your slash command can now communicate with your GitLab project. Prerequisite: -- To run [slash commands](#available-slash-commands), you must have +- To run [slash commands](gitlab_slack_application.md#slash-commands), you must have [permission](../../permissions.md#project-members-permissions) to perform the action in the GitLab project. @@ -120,21 +122,10 @@ To interact with GitLab using Mattermost slash commands: You can see all authorized chat accounts in your Mattermost profile page under **Chat**. -## Available slash commands - -The available slash commands for Mattermost are: - -| Command | Description | Example | -| ------- | ----------- | ------- | -| `/<trigger> issue new <title>` <kbd>Shift</kbd>+<kbd>Enter</kbd> `<description>` | Create a new issue in the project that `<trigger>` is tied to. `<description>` is optional. | `/gitlab issue new We need to change the homepage` | -| `/<trigger> issue show <issue-number>` | Show the issue with ID `<issue-number>` from the project that `<trigger>` is tied to. | `/gitlab issue show 42` | -| `/<trigger> deploy <environment> to <environment>` | Start the CI/CD job that deploys from one environment to another (for example, `staging` to `production`). CI/CD must be [properly configured](../../../ci/yaml/index.md). | `/gitlab deploy staging to production` | -| `/<trigger> help` | View a list of available slash commands. | `/gitlab help` | - ## Related topics -- [Mattermost slash commands](https://developers.mattermost.com/integrate/slash-commands/) -- [Linux package Mattermost](../../../integration/mattermost/index.md) +- [Mattermost Linux package](../../../integration/mattermost/index.md) +- [Slash commands at Mattermost](https://developers.mattermost.com/integrate/slash-commands/) ## Troubleshooting diff --git a/doc/user/project/integrations/mlflow_client.md b/doc/user/project/integrations/mlflow_client.md index cffa2649cc8..ce092d10d20 100644 --- a/doc/user/project/integrations/mlflow_client.md +++ b/doc/user/project/integrations/mlflow_client.md @@ -1,104 +1,11 @@ --- -stage: Create -group: Incubation -info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. +redirect_to: '../ml/experiment_tracking/mlflow_client.md' +remove_date: '2023-10-12' --- -# MLflow client integration **(FREE)** +This document was moved to [another location](../ml/experiment_tracking/mlflow_client.md). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.11 as an [Experiment](../../../policy/experiment-beta-support.md#experiment) release [with a flag](../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. - -NOTE: -Model experiment tracking is an [experimental feature](../../../policy/experiment-beta-support.md). -Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. - -[MLflow](https://mlflow.org/) is a popular open source tool for Machine Learning Experiment Tracking. -GitLab works as a backend to the MLflow Client, [logging experiments](../ml/experiment_tracking/index.md). -Setting up your integrations requires minimal changes to existing code. - -GitLab plays the role of a MLflow server. Running `mlflow server` is not necessary. - -## Enable MLflow client integration - -Prerequisites: - -- A [personal access token](../../../user/profile/personal_access_tokens.md) for the project, with minimum access level of `api`. -- The project ID. To find the project ID: - 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. - 1. Select **Settings > General**. - -To enable MLflow client integration: - -1. Set the tracking URI and token environment variables on the host that runs the code. - This can be your local environment, CI pipeline, or remote host. For example: - - ```shell - export MLFLOW_TRACKING_URI="http://<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow" - export MLFLOW_TRACKING_TOKEN="<your_access_token>" - ``` - -1. If your training code contains the call to `mlflow.set_tracking_uri()`, remove it. - -When running the training code, MLflow creates experiments, runs, log parameters, metrics, metadata -and artifacts on GitLab. - -After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. -Runs are registered as: - -- Model Candidates, which can be explored by selecting an experiment. -- Tags, which are registered as metadata. - -## Associating a candidate to a CI/CD job - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119454) in GitLab 16.1. - -If your training code is being run from a CI/CD job, GitLab can use that information to enhance -candidate metadata. To do so, add the following snippet to your training code within the run -execution context: - -```python -with mlflow.start_run(run_name=f"Candidate {index}"): - # Your training code - - # Start of snippet to be included - if os.getenv('GITLAB_CI'): - mlflow.set_tag('gitlab.CI_JOB_ID', os.getenv('CI_JOB_ID')) - # End of snippet to be included -``` - -## Supported MLflow client methods and caveats - -GitLab supports these methods from the MLflow client. Other methods might be supported but were not -tested. More information can be found in the [MLflow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html). - -| Method | Supported | Version Added | Comments | -|--------------------------|------------------|----------------|----------| -| `get_experiment` | Yes | 15.11 | | -| `get_experiment_by_name` | Yes | 15.11 | | -| `set_experiment` | Yes | 15.11 | | -| `get_run` | Yes | 15.11 | | -| `start_run` | Yes | 15.11 | | -| `log_artifact` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. -| `log_artifacts` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. -| `log_batch` | Yes | 15.11 | | -| `log_metric` | Yes | 15.11 | | -| `log_metrics` | Yes | 15.11 | | -| `log_param` | Yes | 15.11 | | -| `log_params` | Yes | 15.11 | | -| `log_figure` | Yes | 15.11 | | -| `log_image` | Yes | 15.11 | | -| `log_text` | Yes with caveat | 15.11 | (15.11) Does not support directories. -| `log_dict` | Yes with caveat | 15.11 | (15.11) Does not support directories. -| `set_tag` | Yes | 15.11 | | -| `set_tags` | Yes | 15.11 | | -| `set_terminated` | Yes | 15.11 | | -| `end_run` | Yes | 15.11 | | -| `update_run` | Yes | 15.11 | | -| `log_model` | Partial | 15.11 | (15.11) Saves the artifacts, but not the model data. `artifact_path` must be empty. - -## Limitations - -- The API GitLab supports is the one defined at MLflow version 1.28.0. -- API endpoints not listed above are not supported. -- During creation of experiments and runs, ExperimentTags are stored, even though they are not displayed. -- MLflow Model Registry is not supported. +<!-- This redirect file can be deleted after <2023-10-12>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 14183a47625..55000a0a992 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -79,6 +79,8 @@ The following triggers are available for Slack notifications: | **Wiki page** | A wiki page is created or updated. | | **Deployment** | A deployment starts or finishes. | | **Alert** | A new, unique alert is recorded. | +| **Group mention in public** | A group is mentioned in a public context. | +| **Group mention in private** | A group is mentioned in a confidential context. | | [**Vulnerability**](../../application_security/vulnerabilities/index.md) | A new, unique vulnerability is recorded. | ## Troubleshooting diff --git a/doc/user/project/integrations/slack_slash_commands.md b/doc/user/project/integrations/slack_slash_commands.md index 74bd12561fb..c1e04f2aa63 100644 --- a/doc/user/project/integrations/slack_slash_commands.md +++ b/doc/user/project/integrations/slack_slash_commands.md @@ -10,16 +10,18 @@ NOTE: This feature is only configurable on self-managed GitLab instances. For GitLab.com, use the [GitLab for Slack app](gitlab_slack_application.md) instead. -If you want to control and view GitLab content while you're -working in Slack, you can use Slack slash commands. -To use Slack slash commands, you must configure both Slack and GitLab. +You can use [slash commands](gitlab_slack_application.md#slash-commands) to run common GitLab operations, +like creating an issue, from a [Slack](https://slack.com/) chat environment. +To use slash commands in Slack, you must configure both Slack and GitLab. -GitLab can also send events (for example, `issue created`) to Slack as notifications. -The [Slack notifications integration](slack.md) is configured separately. +GitLab can also send events (such as `issue created`) to Slack as part of the +separately configured [Slack notifications](slack.md). -## Configure GitLab and Slack +For a list of available slash commands, see [Slash commands](gitlab_slack_application.md#slash-commands). -Slack slash command integrations are scoped to a project. +## Configure the integration + +Slack slash commands are scoped to a project. To configure Slack slash commands: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > Integrations**. @@ -35,7 +37,3 @@ Slack slash command integrations are scoped to a project. 1. On the Slack configuration page, select **Save Integration** and copy the **Token**. 1. Go back to the GitLab configuration page and paste in the **Token**. 1. Ensure the **Active** checkbox is selected and select **Save changes**. - -## Slash commands - -You can now use the available [Slack slash commands](../../../integration/slash_commands.md). diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md index dfff537e4a1..8d66f3e48dd 100644 --- a/doc/user/project/integrations/webhook_events.md +++ b/doc/user/project/integrations/webhook_events.md @@ -24,6 +24,7 @@ Event type | Trigger [Subgroup event](#subgroup-events) | A subgroup is created or removed from a group. [Feature flag event](#feature-flag-events) | A feature flag is turned on or off. [Release event](#release-events) | A release is created or updated. +[Emoji event](#emoji-events) | An emoji is awarded or revoked. NOTE: If an author has no public email listed in their @@ -61,6 +62,7 @@ Payload example: "before": "95790bf891e76fee5e1747ab589903a6a1f80f22", "after": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "ref": "refs/heads/master", + "ref_protected": true, "checkout_sha": "da1560886d4f094c3e6c9ef40349f7d38b5d27d7", "user_id": 4, "user_name": "John Smith", @@ -151,6 +153,7 @@ Payload example: "before": "0000000000000000000000000000000000000000", "after": "82b3d5ae55f7080f1e6022629cdb57bfae7cccc7", "ref": "refs/tags/v1.0.0", + "ref_protected": true, "checkout_sha": "82b3d5ae55f7080f1e6022629cdb57bfae7cccc7", "user_id": 1, "user_name": "John Smith", @@ -1468,13 +1471,8 @@ Payload example: ### Number of retries -> `retries_count` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382046) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `job_webhook_retries_count`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named -`job_webhook_retries_count`. -On GitLab.com, this feature is not available. +> - `retries_count` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382046) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `job_webhook_retries_count`. Disabled by default. +> - `retries_count` [enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/382046) in GitLab 16.2. `retries_count` is an integer that indicates if the job is a retry. `0` means that the job has not been retried. `1` means that it's the first retry. @@ -1850,3 +1848,149 @@ Payload example: } } ``` + +## Emoji events + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123952) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `emoji_webhooks`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `emoji_webhooks`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +NOTE: +To have the `emoji_webhooks` flag enabled on GitLab.com, see [issue 417288](https://gitlab.com/gitlab-org/gitlab/-/issues/417288). + +An emoji event is triggered when an emoji is awarded or revoked on: + +- Issues +- Merge requests +- Project snippets +- Comments on: + - Issues + - Merge requests + - Project snippets + - Commits + +The available values for `object_attributes.action` in the payload are: + +- `award` +- `revoke` + +Request header: + +```plaintext +X-Gitlab-Event: Emoji Hook +``` + +Payload example: + +```json +{ + "object_kind": "emoji", + "event_type": "award", + "user": { + "id": 1, + "name": "Blake Bergstrom", + "username": "root", + "avatar_url": "http://example.com/uploads/-/system/user/avatar/1/avatar.png", + "email": "[REDACTED]" + }, + "project_id": 6, + "project": { + "id": 6, + "name": "Flight", + "description": "Velit fugit aperiam illum deleniti odio sequi.", + "web_url": "http://example.com/flightjs/Flight", + "avatar_url": null, + "git_ssh_url": "ssh://git@example.com/flightjs/Flight.git", + "git_http_url": "http://example.com/flightjs/Flight.git", + "namespace": "Flightjs", + "visibility_level": 20, + "path_with_namespace": "flightjs/Flight", + "default_branch": "master", + "ci_config_path": null, + "homepage": "http://example.com/flightjs/Flight", + "url": "ssh://git@example.com/flightjs/Flight.git", + "ssh_url": "ssh://git@example.com/flightjs/Flight.git", + "http_url": "http://example.com/flightjs/Flight.git" + }, + "object_attributes": { + "user_id": 1, + "created_at": "2023-07-04 20:44:11 UTC", + "id": 1, + "name": "thumbsup", + "awardable_type": "Note", + "awardable_id": 363, + "updated_at": "2023-07-04 20:44:11 UTC", + "action": "award", + "awarded_on_url": "http://example.com/flightjs/Flight/-/issues/42#note_363" + }, + "note": { + "attachment": null, + "author_id": 1, + "change_position": null, + "commit_id": null, + "created_at": "2023-07-04 15:09:55 UTC", + "discussion_id": "c3d97fd471f210a5dc8b97a409e3bea95ee06c14", + "id": 363, + "line_code": null, + "note": "Testing 123", + "noteable_id": 635, + "noteable_type": "Issue", + "original_position": null, + "position": null, + "project_id": 6, + "resolved_at": null, + "resolved_by_id": null, + "resolved_by_push": null, + "st_diff": null, + "system": false, + "type": null, + "updated_at": "2023-07-04 19:58:46 UTC", + "updated_by_id": null, + "description": "Testing 123", + "url": "http://example.com/flightjs/Flight/-/issues/42#note_363" + }, + "issue": { + "author_id": 1, + "closed_at": null, + "confidential": false, + "created_at": "2023-07-04 14:59:43 UTC", + "description": "Issue description!", + "discussion_locked": null, + "due_date": null, + "id": 635, + "iid": 42, + "last_edited_at": null, + "last_edited_by_id": null, + "milestone_id": null, + "moved_to_id": null, + "duplicated_to_id": null, + "project_id": 6, + "relative_position": 18981, + "state_id": 1, + "time_estimate": 0, + "title": "New issue!", + "updated_at": "2023-07-04 15:09:55 UTC", + "updated_by_id": null, + "weight": null, + "health_status": null, + "url": "http://example.com/flightjs/Flight/-/issues/42", + "total_time_spent": 0, + "time_change": 0, + "human_total_time_spent": null, + "human_time_change": null, + "human_time_estimate": null, + "assignee_ids": [ + 1 + ], + "assignee_id": 1, + "labels": [ + + ], + "state": "opened", + "severity": "unknown" + } +} +``` diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 5c8fc5703dd..9f24f3b49ff 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -126,7 +126,7 @@ Endpoints should follow these best practices: > - [Disabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/390157) in GitLab 15.10 [with a flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `auto_disabling_web_hooks`. On GitLab.com, this feature is available. Project or group webhooks that fail four consecutive times are automatically disabled. @@ -279,6 +279,7 @@ For more information about supported events for webhooks, see [webhook events](w > - `X-Gitlab-Event-UUID` header [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329743) in GitLab 14.8. > - `X-Gitlab-Instance` header [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31333) in GitLab 15.5. +> - `X-Gitlab-Webhook-UUID` header [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230830) in GitLab 16.2. Webhook requests to your endpoint include the following headers: @@ -286,6 +287,7 @@ Webhook requests to your endpoint include the following headers: | ------ | ------ | ------ | | `User-Agent` | In the format `"Gitlab/<VERSION>"`. | `"GitLab/15.5.0-pre"` | | `X-Gitlab-Instance` | Hostname of the GitLab instance that sent the webhook. | `"https://gitlab.com"` | +| `X-Gitlab-Webhook-UUID` | Unique ID per webhook. If a webhook request fails and retries, the second request has a new ID. | `"02affd2d-2cba-4033-917d-ec22d5dc4b38"` | | `X-Gitlab-Event` | Name of the webhook type. Corresponds to [event types](webhook_events.md) but in the format `"<EVENT> Hook"`. | `"Push Hook"` | | `X-Gitlab-Event-UUID` | Unique ID per webhook that is not recursive. A hook is recursive if triggered by an earlier webhook that hit the GitLab instance. Recursive webhooks have the same value for this header. | `"13792a34-cac6-4fda-95a8-c58e00a3954e"` | diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index f5d0382ccd8..bca1ee5245c 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -422,7 +422,7 @@ Prerequisites: To set a WIP limit for a list, in an issue board: -1. On the top of the list you want to edit, select **List actions** (**{ellipsis_v}**) **> Edit list settings**. +1. On the top of the list you want to edit, select **Edit list settings** (**{settings}**). The list settings sidebar opens on the right. 1. Next to **Work in progress limit**, select **Edit**. 1. Enter the maximum number of issues. @@ -499,10 +499,10 @@ Prerequisites: To remove a list from an issue board: -1. On the top of the list you want to remove, select **List actions** (**{ellipsis_v}**). +1. On the top of the list you want to remove, select **Edit list settings** (**{settings}**). The list settings sidebar opens on the right. -1. Select **Remove list**. A confirmation dialog appears. -1. Select **Remove list** again. +1. Select **Remove list**. +1. On the confirmation dialog, select **Remove list** again. ### Add issues to a list diff --git a/doc/user/project/issues/confidential_issues.md b/doc/user/project/issues/confidential_issues.md index 7e5f26d3526..6c2e6cb2132 100644 --- a/doc/user/project/issues/confidential_issues.md +++ b/doc/user/project/issues/confidential_issues.md @@ -26,7 +26,7 @@ When you create a confidential issue in a project, the project becomes listed in To create a confidential issue: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. On the left sidebar, at the top, select **Create new...** (**{plus}**). +1. On the left sidebar, at the top, select **Create new** (**{plus}**). 1. From the dropdown list, select **New issue**. 1. Complete the [fields](create_issues.md#fields-in-the-new-issue-form). - Select the **This issue is confidential...** checkbox. diff --git a/doc/user/project/issues/create_issues.md b/doc/user/project/issues/create_issues.md index 3e7cfc12da7..8cc9ab71ca7 100644 --- a/doc/user/project/issues/create_issues.md +++ b/doc/user/project/issues/create_issues.md @@ -99,16 +99,16 @@ Prerequisites: To create an issue from a project issue board: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Issues > Boards**. -1. At the top of a board list, select **List actions** (**{ellipsis_v}**) **> Create new issue**. +1. Select **Plan > Issue boards**. +1. At the top of a board list, select **Create new issue** (**{plus-square}**). 1. Enter the issue's title. 1. Select **Create issue**. To create an issue from a group issue board: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. -1. Select **Issues > Boards**. -1. At the top of a board list, select **List actions** (**{ellipsis_v}**) **> Create new issue**. +1. Select **Plan > Issue boards**. +1. At the top of a board list, select **Create new issue** (**{plus-square}**). 1. Enter the issue's title. 1. Under **Projects**, select the project in the group that the issue should belong to. 1. Select **Create issue**. diff --git a/doc/user/project/issues/design_management.md b/doc/user/project/issues/design_management.md index 9c04a40cb32..6013abc4892 100644 --- a/doc/user/project/issues/design_management.md +++ b/doc/user/project/issues/design_management.md @@ -202,12 +202,12 @@ Archived designs are not permanently lost. You can browse ## Markdown and rich text editors for descriptions -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388449) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388449) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/375172) in GitLab 16.2. FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. -On GitLab.com, this feature is not available. -This feature is not ready for production use. +On self-managed GitLab, by default the rich text editor is available. To hide it, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `content_editor_on_issues`. +On GitLab.com, this feature is available. When this feature is enabled, you can use the Markdown and rich text editor in design descriptions. It's the same editor you use for comments across GitLab. @@ -251,7 +251,7 @@ Prerequisites: To delete a comment from a design: 1. On the comment you want to delete, select **More actions** **{ellipsis_v}** **> Delete comment**. -1. On the confirmation dialog box, select **Delete comment**. +1. On the confirmation dialog, select **Delete comment**. ## Resolve a discussion thread on a design diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index a43dd65ed74..b2cc555d3b7 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -16,7 +16,7 @@ You can use issues for many purposes, customized to your needs and workflow. - Accept feature proposals, questions, support requests, or bug reports. - Elaborate on code implementations. -For more information about using issues, see the GitLab blog post: +For more information about issues, see the GitLab blog post: [Always start a discussion with an issue](https://about.gitlab.com/blog/2016/03/03/start-with-an-issue/). Issues are always associated with a specific project. If you have multiple diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 6f1c14b2b80..6bf8e2eeb97 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -155,7 +155,7 @@ Prerequisites: To do it: -1. Optional (but recommended). [Create a backup](../../../raketasks/backup_restore.md) before +1. Optional (but recommended). [Create a backup](../../../administration/backup_restore/index.md) before attempting any changes in the console. 1. Open the [Rails console](../../../administration/operations/rails_console.md). 1. Run the following script. Make sure to change `project`, `admin_user`, and `target_project` to @@ -215,9 +215,8 @@ To close an issue, you can either: 1. Select **Plan > Issues**, then select your issue to view it. 1. At the top of the issue, select **Close issue**. -<!-- Delete when the `moved_mr_sidebar` feature flag is removed --> If you don't see this action at the top of an issue, your project or instance might have -enabled a feature flag for [moved actions](../merge_requests/index.md#move-sidebar-actions). +enabled a feature flag to [moved it in the actions menu](#move-the-close-button-into-the-actions-menu). ### Reopen a closed issue @@ -228,6 +227,9 @@ Prerequisites: To reopen a closed issue, at the top of the issue, select **Reopen issue**. A reopened issue is no different from any other open issue. +If you don't see this action at the top of an issue, your project or instance might have +enabled a feature flag to [moved it in the actions menu](#move-the-close-button-into-the-actions-menu). + ### Closing issues automatically You can close issues automatically by using certain words, called a _closing pattern_, @@ -328,6 +330,24 @@ Prerequisites: Learn how to change the default [issue closing pattern](../../../administration/issue_closing_pattern.md). of your installation. +<!-- Delete when the `move_close_into_dropdown` feature flag is removed +and update steps for closing and reopening issues, incidents, and epics --> +### Move the close button into the actions menu + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125173) in GitLab 16.2 [with a flag](../../../administration/feature_flags.md) named `move_close_into_dropdown`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `move_close_into_dropdown`. +On GitLab.com, this feature is not available. + +When this feature flag is enabled, in the upper-right corner, +**Issue actions** (**{ellipsis_v}**) contains the **Close issue** and **Reopen issue** actions. + +In GitLab 16.2 and later, similar action menus are also available on incidents and epics. + +When this feature flag is disabled, **Close issue** and **Reopen issue** are +on the top bar, outside of the actions menu. + ## Change the issue type Prerequisites: @@ -445,9 +465,6 @@ To filter the list of issues: 1. Repeat this process to filter by multiple attributes. Multiple attributes are joined by a logical `AND`. -GitLab displays the results on-screen, but you can also -[retrieve them as an RSS feed](../../search/index.md#retrieve-search-results-as-feed). - ### Filter with the OR operator > - OR filtering for author and assignee was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23532) in GitLab 15.6 [with a flag](../../../administration/feature_flags.md) named `or_issuable_queries`. Disabled by default. @@ -456,7 +473,7 @@ GitLab displays the results on-screen, but you can also FLAG: On self-managed GitLab, by default this feature is available. -To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `or_issuable_queries`. +To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `or_issuable_queries`. On GitLab.com, this feature is available. When this feature is enabled, you can use the OR operator (**is one of: `||`**) diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index 829e44eae9a..4c0566c2386 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -76,6 +76,10 @@ When you sort by **Popularity**, the issue order changes to sort descending by t number of upvotes ([emoji reactions](../../award_emojis.md) with the "thumbs up") on each issue. You can use this to identify issues that are in high demand. +The total number of votes is not summed up. An issue with 18 upvotes and 5 +downvotes is considered more popular than an issue with 17 upvotes and no +downvotes. + ## Sorting by priority When you sort by **Priority**, the issue order changes to sort in this order: diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 7a1d9d6e6e3..37b24cda5aa 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -69,7 +69,7 @@ If a user is: ### Membership and visibility rights -Depending on their membership type, members of groups or projects are granted different visibility levels +Depending on their membership type, members of groups or projects are granted different [visibility levels](../../../user/public_access.md) and rights into the group or project. | Action | Direct group member | Inherited group member | Direct shared group member | Inherited shared group member | @@ -80,7 +80,7 @@ and rights into the group or project. | View milestones of groups higher in the hierarchy | ✓ | ✓ | ✓ | ✓ | | Be shared into other groups | ✓ | | | | | Be shared into other projects | ✓ | ✓ | | | -| Share the group with other members | ✓ | | | | +| Share the group with other members | ✓ | ✓ | ✓ | ✓ | In the following example, `User` is a: @@ -235,7 +235,7 @@ To remove a member from a project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Manage > Members**. 1. Next to the project member you want to remove, select **Remove member**. -1. Optional. In the confirmation box, select the +1. Optional. On the confirmation dialog, select the **Also unassign this user from related issues and merge requests** checkbox. 1. To prevent leaks of sensitive information from private projects, verify the member has not forked the private repository or created webhooks. Existing forks continue to receive diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index aadc27fb2d3..15c5935648f 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -76,6 +76,7 @@ In addition: - On the group's page, the project is listed on the **Shared projects** tab. - On the project's **Members** page, the group is listed on the **Groups** tab. - Each user is assigned a maximum role. +- Members who have the **Project Invite** badge next to their profile on the usage quota page count towards the billable members of the shared project's top-level group. ## Maximum role diff --git a/doc/user/project/merge_requests/allow_collaboration.md b/doc/user/project/merge_requests/allow_collaboration.md index d5851050fd4..4cc8f50dd70 100644 --- a/doc/user/project/merge_requests/allow_collaboration.md +++ b/doc/user/project/merge_requests/allow_collaboration.md @@ -109,5 +109,5 @@ To see the pipeline status from the merge request page of a forked project going back to the original project: 1. Create a group containing all the upstream members. -1. Go to the **Project information > Members** page in the forked project and invite the newly-created +1. Go to the **Manage > Members** page in the forked project and invite the newly-created group to the forked project. diff --git a/doc/user/project/merge_requests/approvals/index.md b/doc/user/project/merge_requests/approvals/index.md index 68d5665139a..0dcf5f37d65 100644 --- a/doc/user/project/merge_requests/approvals/index.md +++ b/doc/user/project/merge_requests/approvals/index.md @@ -109,23 +109,22 @@ Without the approvals, the work cannot merge. Required approvals enable multiple > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334698) in GitLab 15.1. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/389905) in GitLab 15.11 [with a flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. Disabled by default. -> - [Enabled by default on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/405023) in GitLab 16.1. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/405023) in GitLab 16.2. Feature flag `invalid_scan_result_policy_prevents_merge` removed. FLAG: On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. +an administrator can [disable the feature flag](../../../../administration/feature_flags.md) named `invalid_scan_result_policy_prevents_merge`. -Whenever an approval rule cannot be satisfied, the rule is displayed as **(!) Auto approved**. This applies to the following conditions: +Whenever an approval rule cannot be satisfied, the rule is displayed as **Auto approved**. This applies to the following conditions: - The only eligible approver is the author of the merge request. - No eligible approvers (either groups or users) have been assigned to the approval rule. - The number of required approvals is more than the number of eligible approvers. -These rules are automatically approved to unblock their respective merge requests, -unless they were created through a security policy. - -Invalid approval rules created through a security policy are presented with **(!) Action Required** -and are not automatically approved, blocking their respective merge requests. +These rules are automatically approved to unblock their respective merge requests, unless they were +created through a [scan result policy](../../../application_security/policies/scan-result-policies.md). +Invalid approval rules created through a scan result policy are presented with +**Action required** and are not automatically approved, blocking their respective merge requests. ## Related topics diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index bc5d4353ffc..169c7a09875 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -42,7 +42,7 @@ To add a merge request approval rule: - To apply the rule to a specific branch, select it from the list. 1. Set the number of required approvals in **Approvals required**. A value of `0` makes [the rule optional](#configure-optional-approval-rules), and any number greater than `0` - creates a required rule. + creates a required rule. Maximum number of required approvals is `100`. 1. To add users or groups as approvers, search for users or groups that are [eligible to approve](#eligible-approvers), and select **Add**. GitLab suggests approvers based on previous authors of the files changed by the merge request. @@ -261,7 +261,12 @@ For more information, see [Coverage check approval rule](../../../../ci/testing/ ## Security Approvals **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/357021) in GitLab 15.0. +> - Bot comment for approvals [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/411656) in GitLab 16.2 [with a flag](../../../../administration/feature_flags.md) named `security_policy_approval_notification`. Enabled by default. + +FLAG: +On self-managed GitLab, by default the bot comment for approvals is available. To hide the feature, an administrator can [disable the feature flag](../../../../administration/feature_flags.md) named `security_policy_approval_notification`. +On GitLab.com, the bot comment for approvals is available. You can use [scan result policies](../../../application_security/policies/scan-result-policies.md#scan-result-policy-editor) to define security approvals based on the status of vulnerabilities in the merge request and the default branch. Details for each security policy is shown in the Security Approvals section of your Merge Request configuration. @@ -269,6 +274,8 @@ Details for each security policy is shown in the Security Approvals section of y The security approval rules are applied to all merge requests until the pipeline is complete. The application of the security approval rules prevents users from merging in code before the security scans run. After the pipeline is complete, the security approval rules are checked to determine if the security approvals are still required. +In case the scanners in the pipeline identify an issue and security approvals are required, a bot comment is generated +on the merge request to indicate which steps are needed to proceed.  diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 6c079dc9319..eed8cbcd94d 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -112,7 +112,7 @@ permission enables an electronic signature for approvals, such as the one define [Code of Federal Regulations (CFR) Part 11](https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?CFRPart=11&showFR=1&subpartNode=21:1.0.1.1.8.3)): 1. Enable password authentication for the web interface, as described in the - [sign-in restrictions documentation](../../../admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). + [sign-in restrictions documentation](../../../../administration/settings/sign_in_restrictions.md#password-authentication-enabled). 1. On the left sidebar, select **Settings > Merge requests**. 1. In the **Merge request approvals** section, scroll to **Approval settings** and select **Require user password to approve**. diff --git a/doc/user/project/merge_requests/changes.md b/doc/user/project/merge_requests/changes.md index 94f506ba556..79599580f3e 100644 --- a/doc/user/project/merge_requests/changes.md +++ b/doc/user/project/merge_requests/changes.md @@ -51,7 +51,7 @@ clear your browser's cookies or change this behavior again. To view one file at a time for all of your merge requests: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Preferences**. 1. Scroll to the **Behavior** section and select the **Show one file at a time on merge request's Changes tab** checkbox. 1. Select **Save changes**. @@ -146,11 +146,8 @@ per conflicted file on the merge request diff: ## Add a comment to a merge request file -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121429) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `comment_on_files`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `comment_on_files`. -On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123515) in GitLab 16.1 [with a flag](../../../administration/feature_flags.md) named `comment_on_files`. Enabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/125130) in GitLab 16.2. You can add comments to a merge request diff file. These comments persist across rebases and file changes. diff --git a/doc/user/project/merge_requests/drafts.md b/doc/user/project/merge_requests/drafts.md index 6f309d2db24..ff5421d5785 100644 --- a/doc/user/project/merge_requests/drafts.md +++ b/doc/user/project/merge_requests/drafts.md @@ -59,7 +59,7 @@ when you mark a merge request as ready, notifications are triggered to When viewing or searching in your project's merge requests list, you can include or exclude draft merge requests: -1. Go to your project and select **Merge requests**. +1. Go to your project and select **Code > Merge requests**. 1. In the navigation bar, select **Open**, **Merged**, **Closed**, or **All** to filter by merge request status. 1. Select the search box to display a list of filters and select **Draft**, or diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 38125f623eb..dfa62628e46 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -77,7 +77,7 @@ or: or: -1. On the left sidebar, at the top, select **Merge requests** (**{merge-request}**). +1. On the left sidebar, select **Code > Merge requests** (**{merge-request}**). 1. From the dropdown list, select **Assigned**. ## Filter the list of merge requests @@ -111,9 +111,6 @@ To filter the list of merge requests: 1. Select a **Sort direction**, either **{sort-lowest}** for descending order, or **{sort-highest}** for ascending order. -GitLab displays the results on-screen, but you can also -[retrieve them as an RSS feed](../../search/index.md#retrieve-search-results-as-feed). - ### By environment or deployment date > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44041) in GitLab 13.6. @@ -263,13 +260,9 @@ after merging does not retarget open merge requests. This improvement is <!-- When the `moved_mr_sidebar` feature flag is removed, delete this topic and update the steps for these actions like in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/87727/diffs?diff_id=522279685#5d9afba799c4af9920dab533571d7abb8b9e9163 --> -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85584) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/85584) in GitLab 14.10 [with a flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. Enabled by default. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/373757) to also move actions on issues, incidents, and epics in GitLab 16.0. -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available per project or for your entire instance, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `moved_mr_sidebar`. -On GitLab.com, this feature is enabled in the following projects: `gitlab-org/gitlab`, `gitlab-com/www-gitlab-com`, and `gitlab-org/customers-gitlab-com`. - When this feature flag is enabled, in the upper-right corner, **Merge request actions** (**{ellipsis_v}**) contains the following actions: @@ -317,7 +310,7 @@ For a web developer writing a webpage for your company's website: FLAG: On self-managed GitLab, by default this feature is not available. -To make it available per user, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `mr_activity_filters` for individual or groups of users. +To make it available per user, an administrator can [enable the feature flag](../../../administration/feature_flags.md) named `mr_activity_filters` for individual or groups of users. On GitLab.com, this feature is enabled for GitLab team members only. To understand the history of a merge request, filter its activity feed to show you diff --git a/doc/user/project/merge_requests/reviews/img/suggest_changes_v16_2.png b/doc/user/project/merge_requests/reviews/img/suggest_changes_v16_2.png Binary files differnew file mode 100644 index 00000000000..3465085c311 --- /dev/null +++ b/doc/user/project/merge_requests/reviews/img/suggest_changes_v16_2.png diff --git a/doc/user/project/merge_requests/reviews/index.md b/doc/user/project/merge_requests/reviews/index.md index 86468af06a2..54ebfd9eecb 100644 --- a/doc/user/project/merge_requests/reviews/index.md +++ b/doc/user/project/merge_requests/reviews/index.md @@ -218,7 +218,7 @@ When bulk-editing merge requests in a project, you can edit the following attrib To update multiple project merge requests at the same time: -1. In a project, go to **Merge requests**. +1. In a project, go to **Code > Merge requests**. 1. Select **Bulk edit**. A sidebar on the right-hand side of your screen appears with editable fields. 1. Select the checkboxes next to each merge request you want to edit. @@ -236,7 +236,7 @@ When bulk editing merge requests in a group, you can edit the following attribut To update multiple group merge requests at the same time: -1. In a group, go to **Merge requests**. +1. In a group, go to **Code > Merge requests**. 1. Select **Bulk edit**. A sidebar on the right-hand side of your screen appears with editable fields. 1. Select the checkboxes next to each merge request you want to edit. diff --git a/doc/user/project/merge_requests/reviews/suggestions.md b/doc/user/project/merge_requests/reviews/suggestions.md index 24197c5c313..f949dba85dd 100644 --- a/doc/user/project/merge_requests/reviews/suggestions.md +++ b/doc/user/project/merge_requests/reviews/suggestions.md @@ -7,9 +7,6 @@ type: index, reference # Suggest changes **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25381) custom commit messages for suggestions in GitLab 13.9 [with a flag](../../../../administration/feature_flags.md) named `suggestions_custom_commit`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/297404) in GitLab 13.10. Feature flag `suggestions_custom_commit` removed. - Reviewers can suggest code changes with a Markdown syntax in merge request diff threads. The merge request author (or other users with the appropriate role) can apply any or all of the suggestions from the GitLab UI. Applying suggestions adds a commit to the @@ -43,8 +40,6 @@ merge request, authored by the user who suggested the changes. ### Multi-line suggestions -> [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/232339) in GitLab 13.11: suggestions in multi-line comments also become multi-line. - When you review a merge request diff, you can propose changes to multiple lines (up to 200) in a single suggestion, by either: @@ -72,6 +67,23 @@ Suggestions for multiple lines are limited to 100 lines _above_ and 100 lines _below_ the commented diff line. This allows for up to 200 changed lines per suggestion. +#### Using the rich text editor + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/388449) in GitLab 16.1 [with a flag](../../../../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/375172) in GitLab 16.2. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../../administration/feature_flags.md) named `content_editor_on_issues`. +On GitLab.com, this feature is available. + +When you insert suggestions, you can use the WYSIWYG +[rich text editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/) to move +up and down the source file's line numbers in the UI. + +To add or subtract changed lines, next to **From line**, select **+** or **-**. + + + ## Apply suggestions Prerequisites: @@ -155,10 +167,7 @@ For example, to customize the commit message to output ## Batch suggestions -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25486) in GitLab 13.1 as an [Experiment](../../../../policy/experiment-beta-support.md#experiment) with a flag named `batch_suggestions`, disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/227799) in GitLab 13.2. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) in GitLab 13.11. [Feature flag `batch_suggestions`](https://gitlab.com/gitlab-org/gitlab/-/issues/320755) removed. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326168) custom commit messages for batch suggestions in GitLab 14.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326168) custom commit messages for batch suggestions in GitLab 14.4. To reduce the number of commits added to your branch, you can apply multiple suggestions in a single commit. diff --git a/doc/user/project/milestones/burndown_and_burnup_charts.md b/doc/user/project/milestones/burndown_and_burnup_charts.md index c7ed6069cb6..59b11e78622 100644 --- a/doc/user/project/milestones/burndown_and_burnup_charts.md +++ b/doc/user/project/milestones/burndown_and_burnup_charts.md @@ -32,12 +32,14 @@ For an overview, check the video demonstration on [Mapping work versus time with To view a project's burndown chart: -1. In a project, navigate to **Issues > Milestones**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Plan > Milestones**. 1. Select a milestone from the list. To view a group's burndown chart: -1. In a group, navigate to **Issues > Milestones**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Plan > Milestones**. 1. Select a milestone from the list. ### Use cases for burndown charts @@ -110,12 +112,14 @@ Burnup charts show the assigned and completed work for a milestone. To view a project's burnup chart: -1. In a project, navigate to **Issues > Milestones**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Plan > Milestones**. 1. Select a milestone from the list. To view a group's burnup chart: -1. In a group, navigate to **Issues > Milestones**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your group. +1. Select **Plan > Milestones**. 1. Select a milestone from the list. ### How burnup charts work diff --git a/doc/user/project/ml/experiment_tracking/img/candidate_detail_ci_v16_12.png b/doc/user/project/ml/experiment_tracking/img/candidate_detail_ci_v16_12.png Binary files differnew file mode 100644 index 00000000000..fb4f8d5dc66 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/img/candidate_detail_ci_v16_12.png diff --git a/doc/user/project/ml/experiment_tracking/index.md b/doc/user/project/ml/experiment_tracking/index.md index 81c4bc20301..4e9e736f067 100644 --- a/doc/user/project/ml/experiment_tracking/index.md +++ b/doc/user/project/ml/experiment_tracking/index.md @@ -6,14 +6,18 @@ info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering pr # Machine learning model experiments **(FREE)** -FLAG: -On self-managed GitLab, model experiment tracking is disabled by default. -To enable the feature, ask an administrator to [enable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. -On GitLab.com, this feature is in private testing only. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/9341) in GitLab 15.11 as an [Experiment](../../../../policy/experiment-beta-support.md#experiment) release [with a flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. To enable the feature, an administrator can [enable the feature flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/95373) in GitLab 16.2. NOTE: Model experiment tracking is an [experimental feature](../../../../policy/experiment-beta-support.md). Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. +ACCESS LEVEL: +Model experiments [visibility level](../../../public_access.md) can be set to public, private or disabled. This options can +be configured under `Settings > General > Visibility, project features, permissions > Model experiments`. Users must have +at least [Reporter role](../../../permissions.md#roles) to modify or delete experiments +and candidate data. + When creating machine learning models, data scientists often experiment with different parameters, configurations, and feature engineering to improve the performance of the model. Keeping track of all this metadata and the associated artifacts so that the data scientist can later replicate the experiment is not trivial. Machine learning experiment @@ -58,16 +62,12 @@ Some example parameters: ## Track new experiments and candidates Experiment and trials can only be tracked through the -[MLflow](https://www.mlflow.org/docs/latest/tracking.html) client integration. -See [MLflow client integration](../../integrations/mlflow_client.md) for more information +[MLflow](https://www.mlflow.org/docs/latest/tracking.html) client compatibility. +See [MLflow client compatibility](mlflow_client.md) for more information on how to use GitLab as a backend for the MLflow Client. ## Explore model candidates -Prerequisites: - -- You must have at least the Developer role to view experiment data. - To list the current active experiments, either go to `https/-/ml/experiments` or: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. @@ -82,6 +82,14 @@ limitations. After an artifact is logged for a candidate, all artifacts logged f package registry. The package name for a candidate is `ml_experiment_<experiment_id>`, where the version is the candidate IID. The link to the artifacts can also be accessed from the **Experiment Candidates** list or **Candidate detail**. +## View CI information + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119788) in 16.1 + +Candidates can be associated to the CI job that created them, allowing quick links to the merge request, pipeline, and user that triggered the pipeline: + + + ## Related topics - Development details in [epic 8560](https://gitlab.com/groups/gitlab-org/-/epics/8560). diff --git a/doc/user/project/ml/experiment_tracking/mlflow_client.md b/doc/user/project/ml/experiment_tracking/mlflow_client.md new file mode 100644 index 00000000000..7fd5c7cca92 --- /dev/null +++ b/doc/user/project/ml/experiment_tracking/mlflow_client.md @@ -0,0 +1,109 @@ +--- +stage: Create +group: Incubation +info: Machine Learning Experiment Tracking is a GitLab Incubation Engineering program. No technical writer assigned to this group. +--- + +# MLflow client compatibility **(FREE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/8560) in GitLab 15.11 as an [Experiment](../../../../policy/experiment-beta-support.md#experiment) release [with a flag](../../../../administration/feature_flags.md) named `ml_experiment_tracking`. Disabled by default. + +NOTE: +Model experiment tracking is an [experimental feature](../../../../policy/experiment-beta-support.md). +Refer to <https://gitlab.com/gitlab-org/gitlab/-/issues/381660> for feedback and feature requests. + +[MLflow](https://mlflow.org/) is a popular open source tool for Machine Learning Experiment Tracking. +GitLab [Model experiment tracking](index.md) is compatible with MLflow Client, +[logging experiments](index.md). The setup requires minimal changes to existing code. + +GitLab plays the role of a MLflow server. Running `mlflow server` is not necessary. + +## Enable MLflow client integration + +Prerequisites: + +- A [personal](../../../../user/profile/personal_access_tokens.md), [project](../../../../user/project/settings/project_access_tokens.md), or [group](../../../../user/group/settings/group_access_tokens.md) access token with at least the Developer role and the `api` permission. +- The project ID. To find the project ID: + 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. + 1. Select **Settings > General**. + +To use MLflow client compatibility from a local environment: + +1. Set the tracking URI and token environment variables on the host that runs the code. + This can be your local environment, CI pipeline, or remote host. For example: + + ```shell + export MLFLOW_TRACKING_URI="<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow" + export MLFLOW_TRACKING_TOKEN="<your_access_token>" + ``` + +1. If your training code contains the call to `mlflow.set_tracking_uri()`, remove it. + +When running the training code, MLflow creates experiments, runs, log parameters, metrics, metadata +and artifacts on GitLab. + +After experiments are logged, they are listed under `/<your project>/-/ml/experiments`. +Runs are registered as: + +- Model Candidates, which can be explored by selecting an experiment. +- Tags, which are registered as metadata. + +## Associating a candidate to a CI/CD job + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/119454) in GitLab 16.1. + +If your training code is being run from a CI/CD job, GitLab can use that information to enhance +candidate metadata. To associate a candidate to a CI/CD job: + +1. In the [Project CI variables](../../../../ci/variables/index.md), include the following variables: + - `MLFLOW_TRACKING_URI`: `"<your gitlab endpoint>/api/v4/projects/<your project id>/ml/mlflow"` + - `MLFLOW_TRACKING_TOKEN`: `<your_access_token>` + +1. In your training code within the run execution context, add the following code snippet: + + ```python + with mlflow.start_run(run_name=f"Candidate {index}"): + # Your training code + + # Start of snippet to be included + if os.getenv('GITLAB_CI'): + mlflow.set_tag('gitlab.CI_JOB_ID', os.getenv('CI_JOB_ID')) + # End of snippet to be included + ``` + +## Supported MLflow client methods and caveats + +GitLab supports these methods from the MLflow client. Other methods might be supported but were not +tested. More information can be found in the [MLflow Documentation](https://www.mlflow.org/docs/1.28.0/python_api/mlflow.html). + +| Method | Supported | Version Added | Comments | +|--------------------------|------------------|----------------|----------| +| `get_experiment` | Yes | 15.11 | | +| `get_experiment_by_name` | Yes | 15.11 | | +| `set_experiment` | Yes | 15.11 | | +| `get_run` | Yes | 15.11 | | +| `start_run` | Yes | 15.11 | | +| `log_artifact` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_artifacts` | Yes with caveat | 15.11 | (15.11) `artifact_path` must be empty string. Does not support directories. +| `log_batch` | Yes | 15.11 | | +| `log_metric` | Yes | 15.11 | | +| `log_metrics` | Yes | 15.11 | | +| `log_param` | Yes | 15.11 | | +| `log_params` | Yes | 15.11 | | +| `log_figure` | Yes | 15.11 | | +| `log_image` | Yes | 15.11 | | +| `log_text` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `log_dict` | Yes with caveat | 15.11 | (15.11) Does not support directories. +| `set_tag` | Yes | 15.11 | | +| `set_tags` | Yes | 15.11 | | +| `set_terminated` | Yes | 15.11 | | +| `end_run` | Yes | 15.11 | | +| `update_run` | Yes | 15.11 | | +| `log_model` | Partial | 15.11 | (15.11) Saves the artifacts, but not the model data. `artifact_path` must be empty. + +## Limitations + +- The API GitLab supports is the one defined at MLflow version 1.28.0. +- API endpoints not listed above are not supported. +- During creation of experiments and runs, ExperimentTags are stored, even though they are not displayed. +- MLflow Model Registry is not supported. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index 606f0474cb1..0601e236a08 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -54,10 +54,7 @@ this document for an [overview on DNS records](dns_concepts.md). To add your custom domain to GitLab Pages: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. In the upper-right corner, select **New Domain**. 1. In **Domain**, enter the domain name. 1. Optional. In **Certificate**, turn off the **Automatic certificate management using Let's Encrypt** toggle to add an [SSL/TLS certificate](#adding-an-ssltls-certificate-to-pages). You can also add the certificate and key later. @@ -168,10 +165,7 @@ If you're using Cloudflare, check After you have added all the DNS records: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Next to the domain name, select **Edit**. 1. In **Verification status**, select **Retry verification** (**{retry}**). @@ -263,10 +257,7 @@ meet these requirements. - To add the certificate at the time you add a new domain: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. - 1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). + 1. On the left sidebar, select **Deploy > Pages**. 1. In the upper-right corner, select **New Domain**. 1. In **Domain**, enter the domain name. 1. In **Certificate**, turn off the **Automatic certificate management using Let's Encrypt** toggle to add an [SSL/TLS certificate](#adding-an-ssltls-certificate-to-pages). @@ -275,17 +266,11 @@ meet these requirements. - To add the certificate to a domain previously added: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. - 1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). + 1. On the left sidebar, select **Deploy > Pages**. 1. Next to the domain name, select **Edit**. 1. In **Certificate**, turn off the **Automatic certificate management using Let's Encrypt** toggle to add an [SSL/TLS certificate](#adding-an-ssltls-certificate-to-pages). 1. Select **Save changes**. -NOTE: -The Pages menu entry may also be located at **Deployments > Pages**, [more information](../index.md#menu-position-test) - 1. Add the PEM certificate to its corresponding field. 1. If your certificate is missing its intermediate, copy and paste the root certificate (usually available from your CA website) @@ -309,10 +294,7 @@ domain (as long as you've set a valid certificate for it). To enable this setting: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Select the **Force HTTPS (requires valid certificates)** checkbox. 1. Select **Save changes**. diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 15152efb80c..878613c3b9c 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -43,10 +43,7 @@ For **self-managed** GitLab instances, make sure your administrator has Once you've met the requirements, enable Let's Encrypt integration: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Next to the domain name, select **Edit**. 1. Turn on the **Automatic certificate management using Let's Encrypt** toggle. @@ -73,10 +70,7 @@ associated Pages domain. GitLab also renews it automatically. If you get an error **Something went wrong while obtaining the Let's Encrypt certificate**, first, make sure that your pages site is set to "Everyone" in your project's **Settings > General > Visibility**. This allows the Let's Encrypt Servers reach your pages site. Once this is confirmed, you can try obtaining the certificate again by following these steps: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Next to the domain name, select **Edit**. 1. In **Verification status**, select **Retry verification** (**{retry}**). 1. If you're still getting the same error: @@ -91,10 +85,7 @@ If you get an error **Something went wrong while obtaining the Let's Encrypt cer If you've enabled Let's Encrypt integration, but a certificate is absent after an hour and you see the message, "GitLab is obtaining a Let's Encrypt SSL certificate for this domain. This process can take some time. Please try again later.", try to remove and add the domain for GitLab Pages again by following these steps: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Next to the domain name, select **Remove**. 1. [Add the domain again, and verify it](index.md#1-add-a-custom-domain). 1. [Enable Let's Encrypt integration for your domain](#enabling-lets-encrypt-integration-for-your-custom-domain). diff --git a/doc/user/project/pages/getting_started/pages_ci_cd_template.md b/doc/user/project/pages/getting_started/pages_ci_cd_template.md index 167c72fdc89..02e662d15b3 100644 --- a/doc/user/project/pages/getting_started/pages_ci_cd_template.md +++ b/doc/user/project/pages/getting_started/pages_ci_cd_template.md @@ -25,12 +25,10 @@ these steps, you may have to do additional configuration for the Pages site to g If everything is configured correctly, the site can take approximately 30 minutes to deploy. -To view the pipeline, go to **CI/CD > Pipelines**. +To view the pipeline, go to **Build > Pipelines**. -When the pipeline is finished, go to **Settings > Pages** to find the link to +When the pipeline is finished, go to **Deploy > Pages** to find the link to your Pages website. -If this path is not visible, select **Deployments > Pages**. -[This location is part of an experiment](../index.md#menu-position-test). For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. diff --git a/doc/user/project/pages/getting_started/pages_forked_sample_project.md b/doc/user/project/pages/getting_started/pages_forked_sample_project.md index e8de80ac137..8332d96d99e 100644 --- a/doc/user/project/pages/getting_started/pages_forked_sample_project.md +++ b/doc/user/project/pages/getting_started/pages_forked_sample_project.md @@ -20,14 +20,12 @@ To fork a sample project and create a Pages website: 1. View the sample projects by navigating to the [GitLab Pages examples](https://gitlab.com/pages) group. 1. Select the name of the project you want to [fork](../../repository/forking_workflow.md#create-a-fork). 1. In the upper-right corner, select **Fork**, then choose a namespace to fork to. -1. For your project, on the left sidebar, select **CI/CD > Pipelines** and then **Run pipeline**. +1. For your project, on the left sidebar, select **Build > Pipelines** and then **Run pipeline**. GitLab CI/CD builds and deploys your site. The site can take approximately 30 minutes to deploy. -When the pipeline is finished, go to **Settings > Pages** to find the link to +When the pipeline is finished, go to **Deploy > Pages** to find the link to your Pages website. -If this path is not visible, select **Deployments > Pages**. -[This location is part of an experiment](../index.md#menu-position-test). For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. diff --git a/doc/user/project/pages/getting_started/pages_from_scratch.md b/doc/user/project/pages/getting_started/pages_from_scratch.md index c62ead69216..51f53860fee 100644 --- a/doc/user/project/pages/getting_started/pages_from_scratch.md +++ b/doc/user/project/pages/getting_started/pages_from_scratch.md @@ -174,12 +174,10 @@ After you have completed the preceding steps, deploy your website: 1. Save and commit the `.gitlab-ci.yml` file. -1. Go to **CI/CD > Pipelines** to watch the pipeline. -1. When the pipeline is finished, go to **Settings > Pages** to find the link to +1. Go to **Build > Pipelines** to watch the pipeline. +1. When the pipeline is finished, go to **Deploy > Pages** to find the link to your Pages website. - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). When this `pages` job completes successfully, a special `pages:deploy` job appears in the pipeline view. It prepares the content of the website for the GitLab Pages daemon. GitLab runs it in the background and doesn't use a runner. diff --git a/doc/user/project/pages/getting_started/pages_new_project_template.md b/doc/user/project/pages/getting_started/pages_new_project_template.md index cb1da3fb21f..ae0dd9507ad 100644 --- a/doc/user/project/pages/getting_started/pages_new_project_template.md +++ b/doc/user/project/pages/getting_started/pages_new_project_template.md @@ -12,21 +12,19 @@ You can create a new project from a template and run the CI/CD pipeline to gener Use a template when you want to test GitLab Pages or start a new project that's already configured to generate a Pages site. -1. From the top navigation, select the **+** button and select **New project**. +1. On the left sidebar, at the top, select **Create new** (**{plus}**) and **New project/repository**. 1. Select **Create from Template**. 1. Next to one of the templates starting with **Pages**, select **Use template**.  1. Complete the form and select **Create project**. -1. From the left sidebar, navigate to your project's **CI/CD > Pipelines** +1. On the left sidebar, select **Build > Pipelines** and select **Run pipeline** to trigger GitLab CI/CD to build and deploy your site. -When the pipeline is finished, go to **Settings > Pages** to find the link to +When the pipeline is finished, go to **Deploy > Pages** to find the link to your Pages website. -If this path is not visible, select **Deployments > Pages**. -[This location is part of an experiment](../index.md#menu-position-test). For every change pushed to your repository, GitLab CI/CD runs a new pipeline that immediately publishes your changes to the Pages site. diff --git a/doc/user/project/pages/getting_started/pages_ui.md b/doc/user/project/pages/getting_started/pages_ui.md index 16d5cfcca4a..94dab9dfd17 100644 --- a/doc/user/project/pages/getting_started/pages_ui.md +++ b/doc/user/project/pages/getting_started/pages_ui.md @@ -34,10 +34,8 @@ a pipeline deploys your Pages website. To complete the setup and generate a GitLab Pages deployment: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. +1. On the left sidebar, select **Deploy > Pages**. - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](../index.md#menu-position-test). A **Get Started with Pages** form appears. If this form is not available, see [Troubleshooting](#if-the-get-started-with-pages-form-is-not-available). 1. For **Step 1**, enter an image name and verify that your files are in a `public` folder. @@ -54,7 +52,7 @@ To complete the setup and generate a GitLab Pages deployment: 1. For **Step 4**, add a commit message and select **Commit**. This commit triggers your first GitLab Pages deployment. -To view the running pipeline, go to **CI/CD > Pipelines**. +To view the running pipeline, go to **Build > Pipelines**. To view the artifacts that were created during the deployment, view the job, and on the right side, select **Download artifacts**. diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 9b299b46f75..e3a32a9afe9 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -34,13 +34,6 @@ Pages does not support dynamic server-side processing, for instance, as `.php` a For more information, see [Static vs dynamic websites](https://about.gitlab.com/blog/2016/06/03/ssg-overview-gitlab-pages-part-1-dynamic-x-static/). -## Menu Position Test - -NOTE: -We are currently conducting an A/B test where some users may see the Pages -Menu entry under "Deployments" instead of "Settings". We think that this may -be a more accurate position. Feel free to add any feedback to [the experiment issue](https://gitlab.com/gitlab-org/gitlab/-/issues/373547). - ## Getting started To create a GitLab Pages website: diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 73bfe018a7d..d00af81c10e 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -23,8 +23,6 @@ In brief, this is what you need to upload your website in GitLab Pages: 1. Domain of the instance: domain name that is used for GitLab Pages (ask your administrator). 1. GitLab CI/CD: a `.gitlab-ci.yml` file with a specific job named [`pages`](../../../ci/yaml/index.md#pages) in the root directory of your repository. -1. A directory called `public` in your site's repository containing the content - to be published. 1. GitLab Runner enabled for the project. ## GitLab Pages on GitLab.com @@ -67,10 +65,7 @@ You can configure redirects for your site using a `_redirects` file. For more in To remove your pages: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. - - If this path is not visible, select **Deployments > Pages**. - [This location is part of an experiment](index.md#menu-position-test). +1. On the left sidebar, select **Deploy > Pages**. 1. Select **Remove pages**. ## Subdomains of subdomains @@ -97,19 +92,14 @@ you can create your project first and access it under `http(s)://namespace.examp ## Enable unique domains > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/9347) in GitLab 15.9 [with a flag](../../../administration/feature_flags.md) named `pages_unique_domain`. Disabled by default. -> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/388151) in GitLab 15.11. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `pages_unique_domain`. -On GitLab.com, by default this feature is available. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/388151) in GitLab 15.11. By default, every project in a group shares the same domain, for example, `group.gitlab.io`. This means that cookies are also shared for all projects in a group. To ensure your project uses a unique Pages domain, enable the unique domains feature for the project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. Select **Settings > Pages**. +1. On the left sidebar, select **Deploy > Pages**. 1. Select the **Use unique domain** checkbox. 1. Select **Save changes**. @@ -278,6 +268,35 @@ instead. Here are some examples of what happens given the above Pages site: When `public/data/index.html` exists, it takes priority over the `public/data.html` file for both the `/data` and `/data/` URL paths. +## Customize the default folder + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/859) in GitLab 16.1 with a Pages flag named `FF_CONFIGURABLE_ROOT_DIR`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/1073) in GitLab 16.1. +> - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/890) in GitLab 16.2. + +By default, the [artifact](../../../ci/jobs/job_artifacts.md) folder +that contains the static files of your site needs to have the name `public`. + +To change that folder name to any other value, add a `publish` property to your +`pages` job configuration in `.gitlab-ci.yml`. + +The following example publishes a folder named `dist` instead: + +```yaml +pages: + script: + - npm run build + artifacts: + paths: + - dist + publish: dist +``` + +If you're using a folder name other than `public`you must specify +the directory to be deployed with Pages both as an artifact, and under the +`publish` property. The reason you need both is that you can define multiple paths +as artifacts, and GitLab doesn't know which one you want to deploy. + ## Known issues For a list of known issues, see the GitLab [public issue tracker](https://gitlab.com/gitlab-org/gitlab/-/issues?label_name[]=Category%3APages). diff --git a/doc/user/project/pages/public_folder.md b/doc/user/project/pages/public_folder.md index 6ee8ea17aee..68023b87560 100644 --- a/doc/user/project/pages/public_folder.md +++ b/doc/user/project/pages/public_folder.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Pages public folder **(FREE)** -All the files that should be accessible by the browser must be in a root-level folder called `public`. +> With GitLab 16.1 we introduced the ability to configure the published folder in `.gitlab-ci.yml`, so you longer need to change your framework config. For more information, see how to [set a custom folder to be deployed with Pages](introduction.md#customize-the-default-folder). Follow these instructions to configure the `public` folder for the following frameworks. diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 6958b69061a..78384249abc 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -116,7 +116,7 @@ The protected branch displays in the list of protected branches. FLAG: On self-managed GitLab, by default this feature is not available. -To make it available, ask an administrator to +To make it available, an administrator can [enable the feature flag](../../administration/feature_flags.md) named `group_protected_branches`. On GitLab.com, this feature is not available. @@ -360,8 +360,7 @@ branches by using the GitLab web interface: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Code > Branches**. 1. Next to the branch you want to delete, select **Delete** (**{remove}**). -1. On the confirmation dialog, type the branch name. -1. Select **Yes, delete protected branch**. +1. On the confirmation dialog, enter the branch name and select **Yes, delete protected branch**. Protected branches can only be deleted by using GitLab either from the UI or API. This prevents accidentally deleting a branch through local Git commands or diff --git a/doc/user/project/push_options.md b/doc/user/project/push_options.md index 368a59e69b0..2dfdb77df9c 100644 --- a/doc/user/project/push_options.md +++ b/doc/user/project/push_options.md @@ -37,6 +37,7 @@ You can use push options to skip a CI/CD pipeline, or pass CI/CD variables. | ------------------------------ | ------------------------------------------------------------------------------------------- |---------------------- | | `ci.skip` | Do not create a CI pipeline for the latest push. Only skips branch pipelines and not [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md). This does not skip pipelines for CI integrations, such as Jenkins. | [11.7](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15643) | | `ci.variable="<name>=<value>"` | Provide [CI/CD variables](../../ci/variables/index.md) to be used in a CI pipeline, if one is created due to the push. Only passes variables to branch pipelines and not [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md). | [12.6](https://gitlab.com/gitlab-org/gitlab/-/issues/27983) | +| `integrations.skip_ci` | Skip push events for CI integrations, such as Atlassian Bamboo, Buildkite, Drone, Jenkins, and JetBrains TeamCity. | [16.2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/123837) | An example of using `ci.skip`: @@ -50,6 +51,12 @@ An example of passing some CI/CD variables for a pipeline: git push -o ci.variable="MAX_RETRIES=10" -o ci.variable="MAX_TIME=600" ``` +An example of using `integrations.skip_ci`: + +```shell +git push -o integrations.skip_ci +``` + ## Push options for merge requests You can use Git push options to perform certain actions for merge requests at the same diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 3e8ce009740..066da445eeb 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -163,8 +163,10 @@ The following quick actions can be applied through the description field when ed | `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Clear [health status](issues/managing_issues.md#health-status). | `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, and `2`. | `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. -| `/type` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Converts work item to specified type. Available options for `<type>` include `Issue`, `Task`, `Objective` and `Key Result`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385227) in GitLab 16.0. +| `/type` | **{check-circle}** Yes | **{dotted-circle}** Yes | **{dotted-circle}** Yes | Converts work item to specified type. Available options for `<type>` include `issue`, `task`, `objective` and `key result`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/385227) in GitLab 16.0. | `/promote_to <type>` | **{check-circle}** Yes | **{dotted-circle}** No | **{check-circle}** Yes | Promotes work item to specified type. Available options for `<type>`: `issue` (promote a task) and `objective` (promote a key result). [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412534) in GitLab 16.1. +| `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412277) in GitLab 16.2. +| `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to-do item as done. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/412277) in GitLab 16.2. ## Commit messages diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 6bcc57d5916..40fb6969def 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -38,7 +38,7 @@ When you create a release, or after, you can: To view a list of releases: -- On the left sidebar, select **Deployments > Releases**, or +- On the left sidebar, select **Deploy > Releases**, or - On the project's overview page, if at least one release exists, select the number of releases. @@ -75,7 +75,7 @@ Prerequisites: To create a release in the Releases page: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. On the left sidebar, select **Build > Releases** and select **New release**. +1. On the left sidebar, select **Deploy > Releases** and select **New release**. 1. From the [**Tag name**](release_fields.md#tag-name) dropdown list, either: - Select an existing Git tag. Selecting an existing tag that is already associated with a release results in a validation error. @@ -194,7 +194,7 @@ Prerequisites: In the UI: -1. On the left sidebar, select **Deployments > Releases**. +1. On the left sidebar, select **Deploy > Releases**. 1. In the upper-right corner of the release you want to modify, select **Edit this release** (the pencil icon). 1. On the **Edit Release** page, change the release's details. 1. Select **Save changes**. @@ -236,17 +236,17 @@ the [Releases API](../../../api/releases/index.md#create-a-release). In the user interface, to associate milestones to a release: -1. On the left sidebar, select **Deployments > Releases**. +1. On the left sidebar, select **Deploy > Releases**. 1. In the upper-right corner of the release you want to modify, select **Edit this release** (the pencil icon). 1. From the **Milestones** list, select each milestone you want to associate. You can select multiple milestones. 1. Select **Save changes**. -On the **Deployments > Releases** page, the **Milestone** is listed in the top +On the **Deploy > Releases** page, the **Milestone** is listed in the top section, along with statistics about the issues in the milestones.  -Releases are also visible on the **Issues > Milestones** page, and when you select a milestone +Releases are also visible on the **Plan > Milestones** page, and when you select a milestone on this page. Here is an example of milestones with no releases, one release, and two releases. @@ -266,7 +266,7 @@ You can be notified by email when a new release is created for your project. To subscribe to notifications for releases: -1. On the left sidebar, select **Project information**. +1. On the left sidebar, select **Project overview**. 1. Select **Notification setting** (the bell icon). 1. In the list, select **Custom**. 1. Select the **New release** checkbox. @@ -302,8 +302,8 @@ deploy_to_production: To set a deploy freeze window in the UI, complete these steps: 1. Sign in to GitLab as a user with the Maintainer role. -1. On the left sidebar, select **Project information**. -1. In the left navigation menu, go to **Settings > CI/CD**. +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. On the left sidebar, select **Settings > CI/CD**. 1. Scroll to **Deploy freezes**. 1. Select **Expand** to see the deploy freeze table. 1. Select **Add deploy freeze** to open the deploy freeze modal. diff --git a/doc/user/project/releases/release_evidence.md b/doc/user/project/releases/release_evidence.md index 3269bf8f44b..990945b1a2b 100644 --- a/doc/user/project/releases/release_evidence.md +++ b/doc/user/project/releases/release_evidence.md @@ -97,7 +97,7 @@ Evidence collection snapshots are visible on the Releases page, along with the t When you create a release, if [job artifacts](../../../ci/yaml/index.md#artifactsreports) are included in the last pipeline that ran, they are automatically included in the release as release evidence. -Although job artifacts normally expire, artifacts included in release evidence do not expire. +Although job artifacts usually expire, artifacts included in release evidence do not expire. To enable job artifact collection you must specify both: diff --git a/doc/user/project/remote_development/connect_machine.md b/doc/user/project/remote_development/connect_machine.md index f981918c0ea..53c5d342f74 100644 --- a/doc/user/project/remote_development/connect_machine.md +++ b/doc/user/project/remote_development/connect_machine.md @@ -26,7 +26,7 @@ To connect a remote machine to the Web IDE, you must: To generate Let's Encrypt certificates: -1. Create an `A` record to point a domain to your remote machine (for example, from `example.remote.gitlab.dev` to `1.2.3.4`). +1. Create an `A` record to point a domain to your remote machine (for example, from `example.remote.gitlab.dev` to `10.0.2.2`). 1. Install [Certbot](https://certbot.eff.org/) to enable HTTPS: ```shell diff --git a/doc/user/project/remote_development/index.md b/doc/user/project/remote_development/index.md index 6a9d0c73e9a..ccb1745d490 100644 --- a/doc/user/project/remote_development/index.md +++ b/doc/user/project/remote_development/index.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The feature is not ready for production use. WARNING: This feature is in [Beta](../../../policy/experiment-beta-support.md#beta) and subject to change without notice. diff --git a/doc/user/project/repository/branches/default.md b/doc/user/project/repository/branches/default.md index 100450aefe7..ae978e2123d 100644 --- a/doc/user/project/repository/branches/default.md +++ b/doc/user/project/repository/branches/default.md @@ -96,7 +96,7 @@ unless a subgroup configuration overrides it. ## Protect initial default branches **(FREE SELF)** -> Full protection after initial push [added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118729) in GitLab 16.0. +> Full protection after initial push [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/118729) in GitLab 16.0. GitLab administrators and group owners can define [branch protections](../../../project/protected_branches.md) to apply to every repository's [default branch](#default-branch) diff --git a/doc/user/project/repository/branches/index.md b/doc/user/project/repository/branches/index.md index e43efca600a..3e9957806c8 100644 --- a/doc/user/project/repository/branches/index.md +++ b/doc/user/project/repository/branches/index.md @@ -128,7 +128,7 @@ On this page, you can: > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/363170) in GitLab 15.11 FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../feature_flags.md) named `branch_rules`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../feature_flags.md) named `branch_rules`. On GitLab.com, this feature is available. Branches in your repository can be [protected](../../protected_branches.md) in multiple ways. You can: diff --git a/doc/user/project/repository/code_suggestions.md b/doc/user/project/repository/code_suggestions.md index dd1fa3f4c8b..95d5873a535 100644 --- a/doc/user/project/repository/code_suggestions.md +++ b/doc/user/project/repository/code_suggestions.md @@ -5,191 +5,232 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: index, reference --- -# Code Suggestions (Beta) **(FREE SAAS)** +# Code Suggestions (Beta) **(FREE)** -> - [Introduced](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#code-suggestions-available-in-closed-beta) in GitLab 15.9 as [Beta](/ee/policy/experiment-beta-support.md#beta) for early access Ultimate customers. -> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/408104) as opt-in with GitLab 15.11 as [Beta](/ee/policy/experiment-beta-support.md#beta). +> - [Introduced](https://about.gitlab.com/releases/2023/02/22/gitlab-15-9-released/#code-suggestions-available-in-closed-beta) in GitLab 15.9 as [Beta](../../../policy/experiment-beta-support.md#beta) for early access Ultimate customers on GitLab.com. +> - [Enabled](https://gitlab.com/gitlab-org/gitlab/-/issues/408104) as opt-in with GitLab 15.11 as [Beta](../../../policy/experiment-beta-support.md#beta). > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/408158) from GitLab Ultimate to GitLab Premium in 16.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/410801) from GitLab Premium to GitLab Free in 16.0. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121079) in GitLab 16.1. -> - [Default to third-party AI services](https://gitlab.com/groups/gitlab-org/-/epics/10562) in GitLab 16.1. +> - [Introduced support for Google Vertex AI Codey APIs](https://gitlab.com/groups/gitlab-org/-/epics/10562) in GitLab 16.1. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10653) in GitLab 16.1 as [Beta](../../../policy/experiment-beta-support.md#beta) on self-managed GitLab. +> - Code suggestions in the GitLab WebIDE enabled for all GitLab-hosted customers. +> - [Removed support for GitLab native model](https://gitlab.com/groups/gitlab-org/-/epics/10752) in GitLab 16.2. WARNING: -This feature is in [Beta](/ee/policy/experiment-beta-support.md#beta). -Due to high demand, this feature may have unscheduled downtime and Code Suggestions in IDEs may be delayed. -Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). -Beta users should read about the [known limitations](#known-limitations). We look forward to hearing your feedback. +This feature is in [Beta](../../../policy/experiment-beta-support.md#beta). +Beta users should read about the [known limitations](#known-limitations). We look forward to hearing your [feedback](#feedback). -Code Suggestions use generative AI to suggest code while you're developing. -Use Code Suggestions to write code more efficiently by viewing Code Suggestions -as you type. Depending on the cursor position, the extension either: +Write code more efficiently by using generative AI to suggest code while you're developing. -- Provides entire code snippets, like generating functions. -- Completes the current line. +Code Suggestions are available: -To accept a suggestion, press <kbd>Tab</kbd>. +- To users of GitLab SaaS (by default) and self-managed GitLab Enterprise Edition (when requested). Code Suggestions are not available for GitLab Community Edition. +- In VS Code and Microsoft Visual Studio when you have the corresponding GitLab extension installed. +- In the GitLab WebIDE -Code Suggestions are available in Visual Studio Code when you have the GitLab Workflow extension installed. [Additional IDE extension support](https://gitlab.com/groups/gitlab-org/-/epics/10542) is planned for the near future. +<div class="video-fallback"> + <a href="https://www.youtube.com/watch?v=WnxBYxN2-p4">View an end-to-end demo of Code Suggestions in VS Code</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe> +</figure> + +Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). +Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). ## Supported languages -Code Suggestions may produce [low-quality or incomplete suggestions](#model-accuracy-and-quality). +The best results from Code Suggestions are expected [for languages the Google Vertex AI Codey APIs](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_coding_languages) directly support: -Language support varies depending on which AI model serves Code Suggestions. To use Code Suggestions entirely within GitLab cloud infrastructure, disable third-party AI services. To receive higher quality suggestions, [enable third-party AI services](#third-party-ai-services-controls). +- C++ +- C# +- Go +- Google SQL +- Java +- JavaScript +- Kotlin +- PHP +- Python +- Ruby +- Rust +- Scala +- Swift +- TypeScript -The best results from Code Suggestions are expected for these languages: + Supported [code infrastructure interfaces](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview#supported_code_infrastructure_interfaces) include: -- **Third-party AI services (Google Codey)**: Go, Google Cloud CLI, Google SQL, Java, JavaScript, Kubernetes Resource Model (KRM), Python, Terraform, TypeScript. -- **GitLab first-party AI model**: C/C++, C#, Go, Java, JavaScript, Python, PHP, Ruby, Rust, Scala, TypeScript. +- Google Cloud CLI +- Kubernetes Resource Model (KRM) +- Terraform -Suggestions may be mixed for other languages. Using natural language code comments to request completions may also not function as expected. +Suggestion quality for other languages and using natural language code comments to request completions may not yet result in high-quality suggestions. -Suggestions are best when writing new code. Editing existing functions or 'fill in the middle' of a function may not perform as expected. +## Enable Code Suggestions on GitLab SaaS **(FREE SAAS)** -We are making improvements to the Code Suggestions underlying AI model weekly to improve the quality of suggestions. Please remember that AI is non-deterministic, so you may not get the same suggestion week to week. +Code Suggestions can be enabled [for all members of a group](../../group/manage.md#enable-code-suggestions). -Usage of Code Suggestions is governed by the [GitLab Testing Agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). Learn about [data usage when using Code Suggestions](#code-suggestions-data-usage). +Each individual user must also choose to enable Code Suggestions. -## Enable Code Suggestions for an individual user +### Enable Code Suggestions for an individual user -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121079) in GitLab 16.1 as [Beta](/ee/policy/experiment-beta-support.md#beta). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121079) in GitLab 16.1 as [Beta](../../../policy/experiment-beta-support.md#beta). Each user can enable Code Suggestions for themselves: 1. On the left sidebar, select your avatar. -1. On the left sidebar, select **Preferences**. -1. In the **Code Suggestions** section, select **Enable Code Suggestions**. +1. Select **Preferences**. +1. In the **Code Suggestions** section, select the **Enable Code Suggestions** checkbox. 1. Select **Save changes**. +If Code Suggestions is enabled for the group, the group setting overrides the user setting. + NOTE: -If Code Suggestions is [enabled for the group](../../group/manage.md#enable-code-suggestions), the group setting overrides the user setting. +This setting controls Code Suggestions for all IDEs. Support for [more granular control per IDE](https://gitlab.com/groups/gitlab-org/-/epics/10624) is proposed. -## Enable Code Suggestions in WebIDE +## Enable Code Suggestions on self-managed GitLab **(FREE SELF)** -Prerequisites: +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/10653) in GitLab 16.1 as [Beta](../../../policy/experiment-beta-support.md#beta). -- Code Suggestions must be [enabled for the top-level group](../../group/manage.md#enable-code-suggestions). -- Code Suggestions must be [enabled for your user account](#enable-code-suggestions-for-an-individual-user). -- You must be a GitLab team member. +To enable Code Suggestions on a self-managed GitLab EE instance, you must: -Code Suggestions work automatically in the [GitLab WebIDE](../../project/web_ide/index.md) if the above prerequisites are met. To disable Code Suggestions in the WebIDE, disable the user account setting. +- Be an administrator. +- Have a [GitLab SaaS account](https://gitlab.com/users/sign_up). +You do not need to have a GitLab SaaS subscription. -NOTE: -Disabling in the WebIDE will also disable in any other IDEs you use locally like VS Code. Support for [more granular control per IDE](https://gitlab.com/groups/gitlab-org/-/epics/10624) is proposed. +Then, you will: -## Enable Code Suggestions in VS Code +1. Enable Code Suggestions for your SaaS account. +1. Enable Code Suggestions for the instance. +1. [Request early access](#request-access-to-code-suggestions) to the Code Suggestions Beta. -Prerequisites: +### Enable Code Suggestions for your SaaS account -- Code Suggestions must be [enabled for the top-level group](../../group/manage.md#enable-code-suggestions). -- Code Suggestions must be [enabled for your user account](#enable-code-suggestions-for-an-individual-user). -- Completed the [setup instructions](https://gitlab.com/gitlab-org/gitlab-vscode-extension#setup) for the GitLab Visual Studio Code Extension. +To enable Code Suggestions for your GitLab SaaS account: -To enable Code Suggestions in VS Code: +1. Create a [personal access token](../../profile/personal_access_tokens.md#create-a-personal-access-token) + with the `api` scope. +1. On the left sidebar, select your avatar. +1. Select **Preferences**. +1. In the **Code Suggestions** section, select **Enable Code Suggestions**. +1. Select **Save changes**. -1. Download and install the - [GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow) - for Visual Studio Code. -1. Complete the [setup instructions](https://gitlab.com/gitlab-org/gitlab-vscode-extension#setup) for the extension. -1. After your GitLab account connects successfully, in the left sidebar, select **Extensions**. -1. Find the **GitLab workflow** extension, select **Settings** (**{settings}**), and select **Extension Settings**. -1. Enable **GitLab > AI Assisted Code Suggestions**. +### Enable Code Suggestions for the instance -Start typing and receive suggestions for your GitLab projects. +You must enable Code Suggestions for the instance. When you do this, you: -<div class="video-fallback"> - See an end-to-end demo: <a href="https://www.youtube.com/watch?v=WnxBYxN2-p4">Get started with GitLab Code Suggestions in VS Code</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube-nocookie.com/embed/WnxBYxN2-p4" frameborder="0" allowfullscreen> </iframe> -</figure> +- Agree to the [GitLab testing agreement](https://about.gitlab.com/handbook/legal/testing-agreement/). +- Acknowledge that GitLab: + - Sends data from the instance, including personal data, to GitLab.com infrastructure. -## Enable Code Suggestions in other IDEs and editors +To enable Code Suggestions for your self-managed GitLab instance: -We have experimental support for Code Suggestions in Visual Studio, JetBrains, Neovim, Emacs, Sublime, etc. +1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). +1. Select **Admin Area**. +1. On the left sidebar, select **Settings > General**. +1. Expand **Code Suggestions** and: + - Select **Turn on Code Suggestions for this instance**. + - In **Personal access token**, enter your GitLab SaaS personal access token. +1. Select **Save changes**. -More details in this [blog](https://about.gitlab.com/blog/2023/06/01/extending-code-suggestions/). +This setting is visible only in self-managed GitLab instances. -## Why aren't Code Suggestions displayed? +WARNING: +If you clear the **Turn on code suggestions for this instance** checkbox, the users in your instance can still use Code Suggestions for up to one hour, until the issued JSON web token (JWT) expires. -If Code Suggestions are not displayed, try the following troubleshooting steps. +### Request access to Code Suggestions -In GitLab, ensure Code Suggestions is enabled: +GitLab provisions access on a customer-by-customer basis for Code Suggestions +on self-managed instances. To request access: -- [For your user account](#enable-code-suggestions-for-an-individual-user). -- [For *all* top-level groups your account belongs to](../../group/manage.md#enable-code-suggestions). If you don't have a role that lets you view the top-level group's settings, contact a group owner. +1. Sign into your GitLab SaaS account. +1. Comment on [issue 415393](https://gitlab.com/gitlab-org/gitlab/-/issues/415393) + and tag your customer success manager. -In VS Code: +After GitLab has provisioned access to Code Suggestions for your instance, +the users in your instance can now enable Code Suggestions. -- Ensure [your IDE is configured properly](#enable-code-suggestions-in-vs-code). +## Enable Code Suggestions in other IDEs and editors -To confirm that your account is enabled, go to [https://gitlab.com/api/v4/ml/ai-assist](https://gitlab.com/api/v4/ml/ai-assist). A response of `user_is_allowed` should return `true`. +We have experimental support for Code Suggestions in JetBrains, Neovim, Emacs, Sublime, etc. -### Authentication troubleshooting +More details in this [blog](https://about.gitlab.com/blog/2023/06/01/extending-code-suggestions/). -If the above steps do not solve your issue, the problem may be related to the recent changes in authentication, specifically the token system. To resolve the issue, please follow these troubleshooting steps: +## Use Code Suggestions -- Remove the existing PAT from your GitLab account settings. -- Reauthorize your GitLab account in VSCode using OAuth. -- Test the code suggestions feature with different file extensions to verify if the issue is resolved. +Prerequisites: -## Third-party AI services controls +- For self-managed GitLab, Code Suggestions must be enabled [for the instance](#enable-code-suggestions-on-self-managed-gitlab). +- For GitLab SaaS, Code Suggestions must be enabled [for the top-level group](../../group/manage.md#enable-code-suggestions) and [for your user account](#enable-code-suggestions-for-an-individual-user). +- To use VS Code, ensure you have installed [the VS Code GitLab Workflow extension](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow). +- To use Microsoft Visual Studio, ensure you have installed [the Visual Studio GitLab extension](https://marketplace.visualstudio.com/items?itemName=GitLab.GitLabExtensionForVisualStudio). -Organizations can opt to use Code Suggestions entirely within GitLab cloud infrastructure. This option can be controlled with the top-level group [Third-party AI setting](../../group/manage.md#enable-third-party-ai-features). +To use Code Suggestions: + +1. Author your code. As you type, suggestions are displayed. Depending on the cursor position, the extension either: -Having the third-party AI setting enabled will allow Code Suggestions to use third-party AI services, which is likely to produce higher quality results. Please note that language support varies between the two options and will change over time. + - Provides entire code snippets, like generating functions. + - Completes the current line. -To use Code Suggestions entirely within GitLab’s cloud infrastructure, disable third-party AI services. You can disable Code Suggestions entirely in [your user profile settings](#enable-code-suggestions-for-an-individual-user). +1. To accept a suggestion, press <kbd>Tab</kbd>. + +Suggestions are best when writing new code. Editing existing functions or 'fill in the middle' of a function may not perform as expected. -## Stability and performance +GitLab is making improvements to the Code Suggestions to improve the quality. AI is non-deterministic, so you may not get the same suggestion every time with the same input. -This feature is currently in [Beta](/ee/policy/experiment-beta-support.md#beta). -While the Code Suggestions inference API operates completely within the GitLab.com enterprise infrastructure, -we expect a high demand for this Beta feature, which may cause degraded performance or unexpected downtime -of the feature. We have built this feature to gracefully degrade and have controls in place to allow us to +This feature is currently in [Beta](../../../policy/experiment-beta-support.md#beta). +Code Suggestions depends on both Google Vertex AI Codey APIs and the GitLab Code Suggestions service. We have built this feature to gracefully degrade and have controls in place to allow us to mitigate abuse or misuse. GitLab may disable this feature for any or all customers at any time at our discretion. ## Code Suggestions data usage -Code Suggestions is a generative artificial intelligence (AI) model hosted on GitLab.com. +Code Suggestions is a generative artificial intelligence (AI) model. Your personal access token enables a secure API connection to GitLab.com. -This API connection securely transmits a context window from VS Code to the Code Suggestions ML model for inference, -and the generated suggestion is transmitted back to VS Code. +This API connection securely transmits a context window from your IDE/editor to the Code Suggestions GitLab hosted service which calls Google Vertex AI Codey APIs, +and the generated suggestion is transmitted back to your IDE/editor. -Depending on your settings, different ML models will be used to provide Code Suggestions. GitLab currently leverages [Google Cloud's Vertex AI Codey API models](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview) for third-party AI powered Code Suggestions. The sections below refer only to GitLab first-party AI Model. +GitLab currently leverages [Google Cloud's Vertex AI Codey API models](https://cloud.google.com/vertex-ai/docs/generative-ai/code/code-models-overview). ### Data privacy -This section applies only to customers who have third-party AI services disabled. +No new additional data is collected to enable this feature. Private non-public GitLab customer data is +not used as training data. -Code Suggestions operate completely in the GitLab.com infrastructure, providing the same level of -[security](https://about.gitlab.com/security/) as any other feature of GitLab.com, and processing any personal -data in accordance with our [Privacy Statement](https://about.gitlab.com/privacy/). +Learn more about Google Vertex AI Codey APIs [Data Governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance) -No new additional data is collected to enable this feature. The content of your GitLab hosted source code is -not used as training data. Source code inference against the Code Suggestions model is not used to re-train the model. -Your data also never leaves GitLab.com. All training and inference is done in GitLab.com infrastructure. +### Inference window context -[Read more about the security of GitLab.com](https://about.gitlab.com/security/faq/). +Code Suggestions currently inferences against the currently opened file and has a context window of 2,048 tokens and 8,192 character limits. This limit includes content before and after the cursor, the file name, and the extension type. + +Learn more about Google Vertex AI [code-gecko](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models). + +### Self-managed instance data privacy + +A self-managed GitLab instance does not generate the code suggestion. After successful +authentication to the self-managed instance, a token is generated. + +The IDE/editor then uses this token to securely transmit data directly to +GitLab.com's Code Suggestions service for processing. + +The Code Suggestion service then securely returns an AI-generated code suggestion. + +Neither GitLab nor Google Vertex AI Codey APIs have any visibility into a self-managed customer's code other than +what is sent to generate the code suggestion. ### Training data -This section applies only to customers who have third-party AI services disabled. +Code suggestions are routed through Google Vertex AI Codey APIs. Learn more about Google Vertex AI Codey APIs [Data Governance](https://cloud.google.com/vertex-ai/docs/generative-ai/data-governance) and [Responsible AI](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/responsible-ai). + +Google Vertex AI Codey APIs are not trained on private non-public GitLab customer or user data. -Code Suggestions uses open source pre-trained base models from the -[CodeGen family](https://openreview.net/forum?id=iaYcJKpY2B_) including CodeGen-MULTI and CodeGen-NL. -We then re-train and fine-tune these base models with a customized open source dataset to enable multi-language -support and additional use cases. This customized dataset contains non-preprocessed open source code in 13 -programming languages from [The Pile](https://pile.eleuther.ai/) and the -[Google BigQuery source code dataset](https://cloud.google.com/blog/topics/public-datasets/github-on-bigquery-analyze-all-the-open-source-code). -We then process this raw dataset against heuristics that aim to increase the quality of the dataset. +Google has [shared the following](https://ai.google/discover/foundation-models/) about the data Codey models are trained on: -The Code Suggestions model is not trained on GitLab customer or user data. +> Codey is our family of foundational coding models built on PaLM 2. Codey was fine-tuned on a large dataset of high quality, permissively licensed code from external sources ## Progressive enhancement -This feature is designed as a progressive enhancement to developers IDEs. +This feature is designed as a progressive enhancement to developer's IDEs. Code Suggestions offer a completion if the machine learning engine can generate a recommendation. In the event of a connection issue or model inference failure, the feature gracefully degrades. Code Suggestions do not prevent you from writing code in your IDE. @@ -203,25 +244,16 @@ To use Code Suggestions: - On GitLab.com, you must have an internet connection and be able to access GitLab. - In GitLab 16.1 and later, on self-managed GitLab, you must have an internet connection. -[Self-managed support via a proxy to GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/10528) has been proposed. - ### Model accuracy and quality -Regardless of whether third-party AI services are enabled, while in Beta, Code Suggestions can generate low-quality, incomplete, and possibly insecure code. +Code Suggestions can generate low-quality, incomplete, and possibly insecure code. We strongly encourage all beta users to leverage GitLab native [Code Quality Scanning](../../../ci/testing/code_quality.md) and [Security Scanning](../../application_security/index.md) capabilities. -GitLab uses a customized open source dataset to fine-tune the model to support multiple languages. -Based on the languages you code in, GitLab routes the request to a targeted inference and prompt engine -to get relevant suggestions. - -GitLab is actively refining these models to: - -- Improve the quality of recommendations. -- Add support for more languages. -- Add protections to limit personal data, insecure code, and other unwanted behavior - that the model may have learned from training data. +GitLab currently does not retrain Google Vertex AI Codey APIs. GitLab makes no claims +to the accuracy or quality of code suggestions generated by Google Vertex AI Codey API. +Read more about [Google Vertex AI foundation model capabilities](https://cloud.google.com/vertex-ai/docs/generative-ai/learn/models). ## Known limitations @@ -243,3 +275,64 @@ We are also aware of specific situations that can produce unexpected or incohere ## Feedback Report issues in the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/405152). + +## Troubleshooting + +### Code Suggestions aren't displayed + +If Code Suggestions are not displayed, try the following troubleshooting steps. + +In GitLab, ensure Code Suggestions is enabled: + +- [For your user account](#enable-code-suggestions-for-an-individual-user). +- [For *all* top-level groups your account belongs to](../../group/manage.md#enable-code-suggestions). If you don't have a role that lets you view the top-level group's settings, contact a group owner. + +To confirm that your account is enabled, go to [https://gitlab.com/api/v4/ml/ai-assist](https://gitlab.com/api/v4/ml/ai-assist). A response of `user_is_allowed` should return `true`. + +#### Code Suggestions not displayed in VS Code or GitLab WebIDE + +Check all the steps in [Code Suggestions aren't displayed](#code-suggestions-arent-displayed) first. + +If you are a self-managed user, ensure that Code Suggestions for the [GitLab WebIDE](../../project/web_ide/index.md) are enabled. The same settings apply to VS Code as local IDE. + +1. On the left sidebar, select **Extensions > GitLab Workflow**. +1. Select **Settings** (**{settings}**), and then select **Extension Settings**. +1. In **GitLab > AI Assisted Code Suggestions**, select the **Enable code completion (Beta)** + checkbox. + +If the settings are enabled, but Code Suggestions are still not displayed, try the following steps: + +1. Enable the `Debug` checkbox in the GitLab Workflow **Extension Settings**. +1. Open the extension log in **View > Output** and change the dropdown list to **GitLab Workflow** as the log filter. The command palette command is `GitLab: Show Extension Logs`. +1. Disable and re-enable the **Enable code completion (Beta)** checkbox. +1. Verify that the debug log contains similar output: + +```shell +2023-07-14T17:29:00:763 [debug]: Disabling code completion +2023-07-14T17:29:01:802 [debug]: Enabling code completion +2023-07-14T17:29:01:802 [debug]: AI Assist: Using server: https://codesuggestions.gitlab.com/v2/completions +``` + +#### Code Suggestions not displayed in Microsoft Visual Studio + +Check all the steps in [Code Suggestions aren't displayed](#code-suggestions-arent-displayed) first. + +1. Ensure you have properly [set up the extension](https://gitlab.com/gitlab-org/editor-extensions/gitlab-visual-studio-extension#setup). +1. From the **Tools > Options** menu, find the **GitLab** option. Ensure **Log Level** is set to **Debug**. +1. Open the extension log in **View > Output** and change the dropdown list to **GitLab Extension** as the log filter. +1. Verify that the debug log contains similar output: + +```shell +14:48:21:344 GitlabProposalSource.GetCodeSuggestionAsync +14:48:21:344 LsClient.SendTextDocumentCompletionAsync("GitLab.Extension.Test\TestData.cs", 34, 0) +14:48:21:346 LS(55096): time="2023-07-17T14:48:21-05:00" level=info msg="update context" +``` + +### Authentication troubleshooting + +If the above steps do not solve your issue, the problem may be related to the recent changes in authentication, +specifically the token system. To resolve the issue: + +1. Remove the existing personal access token from your GitLab account settings. +1. Reauthorize your GitLab account in VS Code using OAuth. +1. Test the code suggestions feature with different file extensions to verify if the issue is resolved. diff --git a/doc/user/project/repository/git_blame.md b/doc/user/project/repository/git_blame.md index baac2a788be..235c1f34d1a 100644 --- a/doc/user/project/repository/git_blame.md +++ b/doc/user/project/repository/git_blame.md @@ -21,7 +21,7 @@ Prerequisites: To view the blame for a file: -1. Go to your project's **Repository > Files**. +1. Go to your project's **Code > Repository**. 1. Select the file you want to review. 1. In the upper-right corner, select **Blame**. diff --git a/doc/user/project/repository/git_history.md b/doc/user/project/repository/git_history.md index a9228b8cabd..edfcc4b1dc2 100644 --- a/doc/user/project/repository/git_history.md +++ b/doc/user/project/repository/git_history.md @@ -11,7 +11,7 @@ description: "Documentation on Git file history." Git file History provides information about the commit history associated with a file. To use it: -1. Go to your project's **Repository > Files**. +1. Go to your project's **Code > Repository**. 1. In the upper-right corner, select **History**. When you select **History**, this information is displayed: diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index 839ab33808b..8d8639400bd 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -119,7 +119,7 @@ If you don't already have a GPG key, create one: To add a GPG key to your user settings: 1. Sign in to GitLab. -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. Select **GPG Keys** (**{key}**). 1. In **Key**, paste your _public_ key. @@ -230,7 +230,7 @@ You can review commits for a merge request, or for an entire project: 1. Select **Code > Commits**. 1. To review commits for a merge request: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. - 1. Select **Merge requests**, then select your merge request. + 1. Select **Code > Merge requests**, then select your merge request. 1. Select **Commits**. 1. Identify the commit you want to review. Signed commits show either a **Verified** or **Unverified** badge, depending on the verification status of the GPG @@ -253,7 +253,7 @@ If a GPG key becomes compromised, revoke it. Revoking a key changes both future To revoke a GPG key: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. Select **GPG Keys** (**{key}**). 1. Select **Revoke** next to the GPG key you want to delete. @@ -268,7 +268,7 @@ When you remove a GPG key from your GitLab account: To remove a GPG key from your account: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. Select **GPG Keys** (**{key}**). 1. Select **Remove** (**{remove}**) next to the GPG key you want to delete. @@ -286,7 +286,7 @@ If you must unverify both future and past commits, - [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) - [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) - [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) - - [Review existing GPG keys in your instance](../../../admin_area/credentials_inventory.md#review-existing-gpg-keys) + - [Review existing GPG keys in your instance](../../../../administration/credentials_inventory.md#review-existing-gpg-keys) ## Troubleshooting diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 9f0c46591b9..7383772a45b 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -132,7 +132,7 @@ change. This can occur, for example, if Git or a third-party library that GitLab ## Repository languages For the default branch of each repository, GitLab determines which programming languages -are used. This information is displayed on the **Project information** page. +are used. This information is displayed on the **Project overview** page.  @@ -140,7 +140,7 @@ When new files are added, this information can take up to five minutes to update ### Add repository languages -Not all files are detected and listed on the **Project information** page. Documentation, +Not all files are detected and listed on the **Project overview** page. Documentation, vendor code, and most markup languages are excluded. You can change this behavior by overriding the default settings. @@ -224,10 +224,10 @@ To render an OpenAPI file: FLAG: On self-managed GitLab, by default GitLab uses the `du -sk` command to determine the size of a repository. GitLab can use either `git-rev-list` (enabled with feature flag `gitaly_revlist_for_repo_size`) or `git-cat-file` (enabled with feature flag -`gitaly_catfile_repo_size`) instead. To switch between different calculation methods, ask an administrator to +`gitaly_catfile_repo_size`) instead. To switch between different calculation methods, an administrator can [enable or disable](../../../administration/feature_flags.md) these feature flags. -The **Project information** page shows the size of all files in the repository. The size is +The **Project overview** page shows the size of all files in the repository. The size is updated, at most, every 15 minutes. The file size includes repository files, artifacts, and LFS. The size can differ slightly from one instance to another due to compression, housekeeping, and other factors. @@ -237,7 +237,7 @@ Administrators can set a [repository size limit](../../admin_area/settings/accou ## Repository contributor statistics -All code contributors are displayed under your project's **Repository > Contributor statistics**. +All code contributors are displayed under your project's **Analyze > Contributor statistics**. The graph shows the contributor with the most commits to the fewest. @@ -248,7 +248,7 @@ The graph shows the contributor with the most commits to the fewest. A repository graph displays a visual history of the repository network, including branches and merges. This graph can help you visualize the Git flow strategy used in the repository. -Go to your project's **Repository > Graph**. +Go to your project's **Code > Repository graph**.  diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index 733310a0b4d..58c6c343282 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -80,13 +80,7 @@ To use this option, select **Only mirror protected branches** when you create a > - Mirroring branches matching a regex [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/102608) in GitLab 15.8 [with a flag](../../../../administration/feature_flags.md) named `mirror_only_branches_match_regex`. Disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/381667) in GitLab 16.0. - -FLAG: -On self-managed GitLab, by default the field `mirror_branch_regex` is available. -To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) -named `mirror_only_branches_match_regex`. -On GitLab.com, this feature is available. - +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/410354) in GitLab 16.2. Feature flag `mirror_only_branches_match_regex` removed. To mirror only branches with names matching an [re2 regular expression](https://github.com/google/re2/wiki/Syntax), enter a regular expression into the **Mirror specific branches** field. Branches with names that do not match the regular expression are not mirrored. @@ -101,6 +95,9 @@ You can also manually trigger an update: - According to [the pull mirroring interval limit](../../../../administration/instance_limits.md#pull-mirroring-interval) set by the administrator on self-managed instances. +NOTE: +[GitLab Silent Mode](../../../../administration/silent_mode/index.md) disables both push and pull updates. + ### Force an update While mirrors are scheduled to update automatically, you can force an immediate update unless: @@ -209,8 +206,8 @@ Older versions of SSH may require you to remove `-E md5` from the command. ## Related topics - Configure a [Pull Mirroring Interval](../../../../administration/instance_limits.md#pull-mirroring-interval) -- [Disable mirrors for a project](../../../admin_area/settings/visibility_and_access_controls.md#enable-project-mirroring) -- [Secrets file and mirroring](../../../../raketasks/backup_restore.md#when-the-secrets-file-is-lost) +- [Disable mirrors for a project](../../../../administration/settings/visibility_and_access_controls.md#enable-project-mirroring) +- [Secrets file and mirroring](../../../../administration/backup_restore/backup_gitlab.md#when-the-secrets-file-is-lost) ## Troubleshooting @@ -224,11 +221,16 @@ If you receive this message while mirroring to a GitHub repository: 13:Received RST_STREAM with error code 2 ``` -Your GitHub settings might be set to block pushes that expose your email address -used in commits. To fix this problem, either: +One of these issues might be occurring: -- Set your GitHub email address to public. -- Disable the [Block command line pushes that expose my email](https://github.com/settings/emails) setting. +1. Your GitHub settings might be set to block pushes that expose your email address + used in commits. To fix this problem, either: + - Set your GitHub email address to public. + - Disable the [Block command line pushes that expose my email](https://github.com/settings/emails) + setting. +1. Your repository exceeds GitHub's file size limit of 100 MB. To fix this problem, + check the file size limit configured for on GitHub, and consider using + [Git Large File Storage](https://git-lfs.github.com) to manage large files. ### Deadline Exceeded @@ -320,7 +322,7 @@ fail nor succeed. They also do not leave a clear log. To check for this problem: end ``` -1. After you run the command, the [background jobs page](../../../admin_area/index.md#background-jobs) +1. After you run the command, the [background jobs page](../../../../administration/admin_area.md#background-jobs) should show new mirroring jobs being scheduled, especially when [triggered manually](#update-a-mirror). diff --git a/doc/user/project/repository/mirror/pull.md b/doc/user/project/repository/mirror/pull.md index 1463e0de0f1..56e85157c03 100644 --- a/doc/user/project/repository/mirror/pull.md +++ b/doc/user/project/repository/mirror/pull.md @@ -15,13 +15,16 @@ branches, tags, and commits from an upstream repository to yours. Unlike [push mirrors](push.md), pull mirrors retrieve changes from an upstream (remote) repository on a scheduled basis. To prevent the mirror from diverging from the upstream repository, don't push commits directly to the downstream mirror. Push commits to -the upstream repository instead. Changes in the remote repository are pulled into the GitLab repository, either: +the upstream repository instead. Changes in the remote repository are pulled into the GitLab repository: -- Automatically in a certain period of time. Self-managed instances can - configure [pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval). +- Automatically, 30 minutes after a previous pull. This cannot be disabled. - When an administrator [force-updates the mirror](index.md#force-an-update). - When an [API call triggers an update](#trigger-an-update-by-using-the-api). +UI and API updates are subject to default +[pull mirroring intervals](../../../../administration/instance_limits.md#pull-mirroring-interval) +of 5 minutes. This interval can be configured by self-managed instances. + By default, if any branch or tag on the downstream pull mirror diverges from the local repository, GitLab stops updating the branch. This prevents data loss. Deleted branches and tags in the upstream repository are not reflected in the @@ -52,13 +55,14 @@ After you configure a GitLab repository as a pull mirror: ## Configure pull mirroring -Prerequisite: +Prerequisites: - If your remote repository is on GitHub and you have [two-factor authentication (2FA) configured](https://docs.github.com/en/authentication/securing-your-account-with-two-factor-authentication-2fa), create a [personal access token for GitHub](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) with the `repo` scope. If 2FA is enabled, this personal access token serves as your GitHub password. +- [GitLab Silent Mode](../../../../administration/silent_mode/index.md) is not enabled. 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > Repository**. diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index 9da8ce7acc5..26a60002f0e 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -30,6 +30,9 @@ it is deleted from the remote mirror on the next push. Branches with unmerged changes are kept. If a branch diverges, the **Mirroring repositories** section displays an error. +[GitLab Silent Mode](../../../../administration/silent_mode/index.md) disables pushing to, +and pulling from, remote mirrors. + ## Configure push mirroring To set up push mirroring for an existing project: @@ -77,8 +80,12 @@ through the [remote mirrors API](../../../../api/remote_mirrors.md). To configure a mirror from GitLab to GitHub: -1. Create a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) - with `public_repo` selected. +1. Create a [GitHub fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#fine-grained-personal-access-tokens) + with at least read and write permissions on the [repository contents](https://docs.github.com/en/rest/overview/permissions-required-for-fine-grained-personal-access-tokens#repository-permissions-for-contents). If your + repository contains a `.github/workflows` directory, you must also grant + read and write access for the [Workflows](https://docs.github.com/en/rest/overview/permissions-required-for-fine-grained-personal-access-tokens#repository-permissions-for-workflows). + For a more fine-grained access, you can configure your token to only apply + to the specific repository. 1. Enter a **Git repository URL** with this format, changing the variables as needed: ```plaintext diff --git a/doc/user/project/repository/push_rules.md b/doc/user/project/repository/push_rules.md index fbadc9b84a3..81896d64815 100644 --- a/doc/user/project/repository/push_rules.md +++ b/doc/user/project/repository/push_rules.md @@ -71,11 +71,13 @@ Use these rules for your commit messages. - **Require expression in commit messages**: Messages must match the expression. To allow any commit message, leave empty. - Uses multiline mode, which can be disabled by using `(?-m)`. + Uses multiline mode, which can be disabled by using `(?-m)`. Some validation examples: + + - `JIRA\-\d+` requires every commit to reference a Jira issue, like `Refactored css. Fixes JIRA-123`. + - `[[:^punct:]]\b$` rejects a commit if the final character is a punctuation mark. + The word boundary character (`\b`) prevents false negatives, because Git adds a + newline character (`\n`) to the end of the commit message. - For example, if every commit should reference a Jira issue - (like `Refactored css. Fixes JIRA-123.`), the regular expression would be - `JIRA\-\d+`. - **Reject expression in commit messages**: Commit messages must not match the expression. To allow any commit message, leave empty. Uses multiline mode, which can be disabled by using `(?-m)`. @@ -230,7 +232,7 @@ These examples use regex (regular expressions) string boundary characters to mat the beginning of a string (`^`), and its end (`$`). They also include instances where either the directory path or the filename can include `.` or `/`. Both of these special regex characters must be escaped with a backslash `\\` if you want -to use them as normal characters in a match condition. +to use them as standard characters in a match condition. - **Prevent pushing `.exe` files to any location in the repository** - This regex matches any filename that contains `.exe` at the end: diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index ce3a5ee9916..334db91cd82 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -235,8 +235,8 @@ When using repository cleanup, note: Repository size limits: -- Can [be set by an administrator](../../admin_area/settings/account_and_limit_settings.md#account-and-limit-settings). -- Can [be set by an administrator](../../admin_area/settings/account_and_limit_settings.md) on self-managed instances. +- Can [be set by an administrator](../../../administration/settings/account_and_limit_settings.md#account-and-limit-settings). +- Can [be set by an administrator](../../../administration/settings/account_and_limit_settings.md) on self-managed instances. - Are [set for GitLab.com](../../gitlab_com/index.md#account-and-limit-settings). When a project has reached its size limit, you cannot: diff --git a/doc/user/project/repository/ssh_signed_commits/index.md b/doc/user/project/repository/ssh_signed_commits/index.md index 8f29845fd9b..85a8917022e 100644 --- a/doc/user/project/repository/ssh_signed_commits/index.md +++ b/doc/user/project/repository/ssh_signed_commits/index.md @@ -169,7 +169,7 @@ If an SSH key becomes compromised, revoke it. Revoking a key changes both future To revoke an SSH key: -1. In the upper-right corner, select your avatar. +1. On the left sidebar, select your avatar. 1. Select **Edit profile**. 1. On the left sidebar, select (**{key}**) **SSH Keys**. 1. Select **Revoke** next to the SSH key you want to delete. diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index b8e9de41062..4079f821064 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -49,7 +49,7 @@ Prerequisite: To create a requirement: -1. In a project, go to **Issues > Requirements**. +1. In a project, go to **Plan > Requirements**. 1. Select **New requirement**. 1. Enter a title and description and select **Create requirement**. @@ -124,7 +124,7 @@ You can search for a requirement from the requirements list page based on the fo To search for a requirement: -1. In a project, go to **Issues > Requirements > List**. +1. In a project, go to **Plan > Requirements > List**. 1. Select the **Search or filter results** field. A dropdown list appears. 1. Select the requirement author or status from the dropdown list or enter plain text to search by requirement title. 1. Press <kbd>Enter</kbd> on your keyboard to filter the list. @@ -239,7 +239,7 @@ Before you import your file: To import requirements: -1. In a project, go to **Issues > Requirements**. +1. In a project, go to **Plan > Requirements**. - For a project with requirements, in the upper-right corner, select the import icon (**{import}**). - For a project without requirements, in the middle of the page, select **Import CSV**. @@ -300,7 +300,7 @@ Prerequisite: To export requirements: -1. In a project, go to **Issues > Requirements**. +1. In a project, go to **Plan > Requirements**. 1. In the upper-right corner, select **Export as CSV** (**{export}**). A confirmation modal appears. diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index 8d525965f96..b508fbcf676 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -43,8 +43,8 @@ Meanwhile: ## Configure Service Desk -To start using Service Desk for a project, you must first turn it on. -By default, Service Desk is turned off. +By default, Service Desk is active in new projects. +If it's not active, you can do it in the project's settings. Prerequisites: @@ -64,7 +64,7 @@ To enable Service Desk in your project: 1. Expand **Service Desk**. 1. Turn on the **Activate Service Desk** toggle. 1. Optional. Complete the fields. - - [Add a suffix](#configure-a-custom-email-address-suffix) to your Service Desk email address. + - [Add a suffix](#configure-a-suffix-for-service-desk-alias-email) to your Service Desk email address. - If the list below **Template to append to all Service Desk issues** is empty, create a [description template](description_templates.md) in your repository. 1. Select **Save changes**. @@ -81,81 +81,73 @@ To improve your Service Desk project's security, you should: - [Enable Akismet](../../integration/akismet.md) on your GitLab instance to add spam checking to this service. Unblocked email spam can result in many spam issues being created. -### Create customized email templates +### Customize emails sent to the requester > - Moved from GitLab Premium to GitLab Free in 13.2. > - `UNSUBSCRIBE_URL`, `SYSTEM_HEADER`, `SYSTEM_FOOTER`, and `ADDITIONAL_TEXT` placeholders [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285512) in GitLab 15.9. +> - `%{ISSUE_DESCRIPTION}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223751) in GitLab 16.0. +> - `%{ISSUE_URL}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408793) in GitLab 16.1. -An email is sent to the author when: - -- A user submits a new issue using Service Desk. -- A new note is created on a Service Desk issue. - -You can customize the body of these email messages with templates. +An email is sent to the requester when: -#### Email header and footer **(FREE SELF)** +- A requester submits a new ticket by emailing Service Desk. +- A new public comment is added on a Service Desk ticket. + - Editing a comment does not trigger a new email to be sent. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344819) in GitLab 15.9. +You can customize the body of these email messages with Service Desk email templates. The templates +can include [GitLab Flavored Markdown](../markdown.md) and [some HTML tags](../markdown.md#inline-html). +For example, you can format the emails to include a header and footer in accordance with your +organization's brand guidelines. You can also include the following placeholders to display dynamic +content specific to the Service Desk ticket or your GitLab instance. -Instance administrators can add a small header or footer to the GitLab instance and make them -visible in the email template. For more information, see -[System header and footer messages](../admin_area/appearance.md#system-header-and-footer-messages). +| Placeholder | `thank_you.md` | `new_note.md` | Description +| ---------------------- | ---------------------- | ---------------------- | ----------- +| `%{ISSUE_ID}` | **{check-circle}** Yes | **{check-circle}** Yes | Ticket IID. +| `%{ISSUE_PATH}` | **{check-circle}** Yes | **{check-circle}** Yes | Project path appended with the ticket IID. +| `%{ISSUE_URL}` | **{check-circle}** Yes | **{check-circle}** Yes | URL of the ticket. External participants can only view the ticket if the project is public and ticket is not confidential (Service Desk tickets are confidential by default). +| `%{ISSUE_DESCRIPTION}` | **{check-circle}** Yes | **{check-circle}** Yes | Ticket description. If a user has edited the description, it may contain sensitive information that is not intended to be delivered to external participants. Use this placeholder with care and ideally only if you never modify descriptions or your team is aware of the template design. +| `%{UNSUBSCRIBE_URL}` | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe URL. +| `%{NOTE_TEXT}` | **{dotted-circle}** No | **{check-circle}** Yes | The new comment added to the ticket by a user. Take care to include this placeholder in `new_note.md`. Otherwise, the requesters may never see the updates on their Service Desk ticket. #### Thank you email -> - `%{ISSUE_DESCRIPTION}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223751) in GitLab 16.0. -> - `%{ISSUE_URL}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408793) in GitLab 16.1. - -When a user submits an issue through Service Desk, GitLab sends a **thank you email**. +When a requester submits an issue through Service Desk, GitLab sends a **thank you email**. +Without additional configuration, GitLab sends the default thank you email. -To create a custom email template, in the `.gitlab/service_desk_templates/` -directory in your repository, create a file named `thank_you.md`. +To create a custom thank you email template: -You can use these placeholders to be automatically replaced in each email: +1. In the `.gitlab/service_desk_templates/` directory of your repository, create a file named `thank_you.md`. +1. Populate the Markdown file with text, [GitLab Flavored Markdown](../markdown.md), + [some selected HTML tags](../markdown.md#inline-html), and placeholders to customize the reply + to Service Desk requesters. -- `%{ISSUE_ID}`: Issue IID. -- `%{ISSUE_PATH}`: Project path appended with the issue IID. -- `%{ISSUE_URL}`: URL to the issue. External participants can only view the issue if the project is public - and issue is not confidential (Service Desk issues are confidential by default). -- `%{ISSUE_DESCRIPTION}`: Issue description based on the original email. -- `%{UNSUBSCRIBE_URL}`: Unsubscribe URL. -- `%{SYSTEM_HEADER}`: [System header message](../admin_area/appearance.md#system-header-and-footer-messages). -- `%{SYSTEM_FOOTER}`: [System footer message](../admin_area/appearance.md#system-header-and-footer-messages). -- `%{ADDITIONAL_TEXT}`: [Custom additional text](../admin_area/settings/email.md#custom-additional-text). +#### New note email -Because Service Desk issues are created as [confidential](issues/confidential_issues.md) (only project members can see them), -the response email does not contain the issue link. +When a Service Desk ticket has a new public comment, GitLab sends a **new note email**. +Without additional configuration, GitLab sends the content of the comment. -#### New note email +To keep your emails on brand, you can create a custom new note email template. To do so: -> - `%{ISSUE_DESCRIPTION}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223751) in GitLab 16.0. -> - `%{ISSUE_URL}` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/408793) in GitLab 16.1. +1. In the `.gitlab/service_desk_templates/` directory in your repository, create a file named `new_note.md`. +1. Populate the Markdown file with text, [GitLab Flavored Markdown](../markdown.md), + [some selected HTML tags](../markdown.md#inline-html), and placeholders to customize the new note + email. Be sure to include the `%{NOTE_TEXT}` in the template to make sure the email recipient can + read the contents of the comment. -When a user-submitted issue receives a new comment, GitLab sends a **new note email**. +#### Instance-level email header, footer, and additional text **(FREE SELF)** -To create a custom email template, in the `.gitlab/service_desk_templates/` -directory in your repository, create a file named `new_note.md`. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344819) in GitLab 15.9. -You can use these placeholders to be automatically replaced in each email: +Instance administrators can add a header, footer or additional text to the GitLab instance and apply +them to all emails sent from GitLab. If you're using a custom `thank_you.md` or `new_note.md`, to include +this content, add `%{SYSTEM_HEADER}`, `%{SYSTEM_FOOTER}`, or `%{ADDITIONAL_TEXT}` to your templates. -- `%{ISSUE_ID}`: Issue IID. -- `%{ISSUE_PATH}`: Project path appended with the issue IID. -- `%{ISSUE_URL}`: URL to the issue. External participants can only view the issue if the project is public - and issue is not confidential (Service Desk issues are confidential by default). -- `%{ISSUE_DESCRIPTION}`: Issue description at the time email is generated. - If a user has edited the description, it might contain sensitive information that is not intended - to be delivered to external participants. Use this placeholder only if you never modify - descriptions or your team is aware of the template design. -- `%{NOTE_TEXT}`: Note text. -- `%{UNSUBSCRIBE_URL}`: Unsubscribe URL. -- `%{SYSTEM_HEADER}`: [System header message](../admin_area/appearance.md#system-header-and-footer-messages). -- `%{SYSTEM_FOOTER}`: [System footer message](../admin_area/appearance.md#system-header-and-footer-messages). -- `%{ADDITIONAL_TEXT}`: [Custom additional text](../admin_area/settings/email.md#custom-additional-text). +For more information, see [System header and footer messages](../../administration/appearance.md#system-header-and-footer-messages) and [custom additional text](../../administration/settings/email.md#custom-additional-text). -### Use a custom template for Service Desk issues +### Use a custom template for Service Desk tickets You can select one [description template](description_templates.md#create-an-issue-template) -**per project** to be appended to every new Service Desk issue's description. +**per project** to be appended to every new Service Desk ticket's description. You can set description templates at various levels: @@ -200,27 +192,26 @@ To edit the custom email display name: 1. Below **Email display name**, enter a new name. 1. Select **Save changes**. -### Use a custom email address **(FREE SELF)** +### Use an additional Service Desk alias email **(FREE SELF)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284656) in GitLab 13.8. -You can use a custom email address with Service Desk. +You can use an additional alias email address for Service Desk on an instance level. To do this, you must configure -a [custom mailbox](#configure-a-custom-mailbox). You can also configure a -[custom suffix](#configure-a-custom-email-address-suffix). +a [`service_desk_email`](#configure-service-desk-alias-email) in the instance configuration. You can also configure a +[custom suffix](#configure-a-suffix-for-service-desk-alias-email) that replaces the default `-issue-` portion on the sub-addressing part. -#### Configure a custom mailbox +#### Configure Service Desk alias email NOTE: -On GitLab.com a custom mailbox is already configured with `contact-project+%{key}@incoming.gitlab.com` as the email address, you can still configure the -[custom suffix](#configure-a-custom-email-address-suffix) in project settings. +On GitLab.com a custom mailbox is already configured with `contact-project+%{key}@incoming.gitlab.com` as the email address. You can still configure the +[custom suffix](#configure-a-suffix-for-service-desk-alias-email) in project settings. Service Desk uses the [incoming email](../../administration/incoming_email.md) -configuration by default. However, by using the `service_desk_email` configuration, -you can customize the mailbox used by Service Desk. This allows you to have -a separate email address for Service Desk by also configuring a [custom suffix](#configure-a-custom-email-address-suffix) +configuration by default. However, to have a separate email address for Service Desk, +configure `service_desk_email` with a [custom suffix](#configure-a-suffix-for-service-desk-alias-email) in project settings. Prerequisites: @@ -413,14 +404,19 @@ read about [Helm IMAP secrets](https://docs.gitlab.com/charts/installation/secre ##### Microsoft Graph -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214900) in GitLab 13.11. > - Alternative Azure deployments [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5978) in GitLab 14.9. +> - [Introduced for self-compiled (source) installs](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/116494) in GitLab 15.11. -Service Desk can be configured to read Microsoft Exchange Online mailboxes with the Microsoft +`service_desk_email` can be configured to read Microsoft Exchange Online mailboxes with the Microsoft Graph API instead of IMAP. Set up an OAuth 2.0 application for Microsoft Graph [the same way as for incoming email](../../administration/incoming_email.md#microsoft-graph). -- Example for Linux package installations: +::Tabs + +:::TabTitle Linux package (Omnibus) + +1. Edit `/etc/gitlab/gitlab.rb` and add the following lines, substituting + the values you want: ```ruby gitlab_rails['service_desk_email_enabled'] = true @@ -430,33 +426,200 @@ Graph API instead of IMAP. Set up an OAuth 2.0 application for Microsoft Graph gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" gitlab_rails['service_desk_email_inbox_method'] = 'microsoft_graph' gitlab_rails['service_desk_email_inbox_options'] = { - 'tenant_id': '<YOUR-TENANT-ID>', - 'client_id': '<YOUR-CLIENT-ID>', - 'client_secret': '<YOUR-CLIENT-SECRET>', - 'poll_interval': 60 # Optional + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional + } + ``` + + For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), + configure the `azure_ad_endpoint` and `graph_endpoint` settings. For example: + + ```ruby + gitlab_rails['service_desk_email_inbox_options'] = { + 'azure_ad_endpoint': 'https://login.microsoftonline.us', + 'graph_endpoint': 'https://graph.microsoft.us', + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional } ``` +:::TabTitle Helm chart (Kubernetes) + +1. Create the [Kubernetes Secret containing the OAuth 2.0 application client secret](https://docs.gitlab.com/charts/installation/secrets.html#microsoft-graph-client-secret-for-service-desk-emails): + + ```shell + kubectl create secret generic service-desk-email-client-secret --from-literal=secret=<YOUR-CLIENT_SECRET> + ``` + +1. Create the [Kubernetes Secret for the GitLab Service Desk email auth token](https://docs.gitlab.com/charts/installation/secrets.html#gitlab-service-desk-email-auth-token). + Replace `<name>` with the name of the [Helm release name](https://helm.sh/docs/intro/using_helm/) for the GitLab installation: + + ```shell + kubectl create secret generic <name>-service-desk-email-auth-token --from-literal=authToken=$(head -c 512 /dev/urandom | LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 32 | base64) + ``` + +1. Export the Helm values: + + ```shell + helm get values gitlab > gitlab_values.yaml + ``` + +1. Edit `gitlab_values.yaml`: + + ```yaml + global: + appConfig: + serviceDeskEmail: + enabled: true + address: "project_contact+%{key}@example.onmicrosoft.com" + user: "project_contact@example.onmicrosoft.com" + mailbox: inbox + inboxMethod: microsoft_graph + azureAdEndpoint: https://login.microsoftonline.com + graphEndpoint: https://graph.microsoft.com + tenantId: "YOUR-TENANT-ID" + clientId: "YOUR-CLIENT-ID" + clientSecret: + secret: service-desk-email-client-secret + key: secret + deliveryMethod: webhook + authToken: + secret: <name>-service-desk-email-auth-token + key: authToken + ``` + + For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), +configure the `azureAdEndpoint` and `graphEndpoint` settings. These fields are case-sensitive: + + ```yaml + global: + appConfig: + serviceDeskEmail: + [..] + azureAdEndpoint: https://login.microsoftonline.us + graphEndpoint: https://graph.microsoft.us + [..] + ``` + +1. Save the file and apply the new values: + + ```shell + helm upgrade -f gitlab_values.yaml gitlab gitlab/gitlab + ``` + +:::TabTitle Docker + +1. Edit `docker-compose.yml`: + + ```yaml + version: "3.6" + services: + gitlab: + environment: + GITLAB_OMNIBUS_CONFIG: | + gitlab_rails['service_desk_email_enabled'] = true + gitlab_rails['service_desk_email_address'] = "project_contact+%{key}@example.onmicrosoft.com" + gitlab_rails['service_desk_email_email'] = "project_contact@example.onmicrosoft.com" + gitlab_rails['service_desk_email_mailbox_name'] = "inbox" + gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" + gitlab_rails['service_desk_email_inbox_method'] = 'microsoft_graph' + gitlab_rails['service_desk_email_inbox_options'] = { + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional + } + ``` + +1. Save the file and restart GitLab: + + ```shell + docker compose up -d + ``` + For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), -configure the `azure_ad_endpoint` and `graph_endpoint` settings. +configure the `azure_ad_endpoint` and `graph_endpoint` settings: -- Example for Microsoft Cloud for US Government: +1. Edit `docker-compose.yml`: -```ruby -gitlab_rails['service_desk_email_inbox_options'] = { - 'azure_ad_endpoint': 'https://login.microsoftonline.us', - 'graph_endpoint': 'https://graph.microsoft.us', - 'tenant_id': '<YOUR-TENANT-ID>', - 'client_id': '<YOUR-CLIENT-ID>', - 'client_secret': '<YOUR-CLIENT-SECRET>', - 'poll_interval': 60 # Optional -} -``` + ```yaml + version: "3.6" + services: + gitlab: + environment: + GITLAB_OMNIBUS_CONFIG: | + gitlab_rails['service_desk_email_enabled'] = true + gitlab_rails['service_desk_email_address'] = "project_contact+%{key}@example.onmicrosoft.com" + gitlab_rails['service_desk_email_email'] = "project_contact@example.onmicrosoft.com" + gitlab_rails['service_desk_email_mailbox_name'] = "inbox" + gitlab_rails['service_desk_email_log_file'] = "/var/log/gitlab/mailroom/mail_room_json.log" + gitlab_rails['service_desk_email_inbox_method'] = 'microsoft_graph' + gitlab_rails['service_desk_email_inbox_options'] = { + 'azure_ad_endpoint': 'https://login.microsoftonline.us', + 'graph_endpoint': 'https://graph.microsoft.us', + 'tenant_id': '<YOUR-TENANT-ID>', + 'client_id': '<YOUR-CLIENT-ID>', + 'client_secret': '<YOUR-CLIENT-SECRET>', + 'poll_interval': 60 # Optional + } + ``` + +1. Save the file and restart GitLab: + + ```shell + docker compose up -d + ``` + +:::TabTitle Self-compiled (source) -The Microsoft Graph API is not yet supported in source installations. -For more information, see [issue 326169](https://gitlab.com/gitlab-org/gitlab/-/issues/326169). +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + service_desk_email: + enabled: true + address: "project_contact+%{key}@example.onmicrosoft.com" + user: "project_contact@example.onmicrosoft.com" + mailbox: "inbox" + delivery_method: webhook + log_path: "log/mailroom.log" + secret_file: .gitlab-mailroom-secret + inbox_method: "microsoft_graph" + inbox_options: + tenant_id: "<YOUR-TENANT-ID>" + client_id: "<YOUR-CLIENT-ID>" + client_secret: "<YOUR-CLIENT-SECRET>" + poll_interval: 60 # Optional + ``` + + For Microsoft Cloud for US Government or [other Azure deployments](https://learn.microsoft.com/en-us/graph/deployments), + configure the `azure_ad_endpoint` and `graph_endpoint` settings. For example: + + ```yaml + service_desk_email: + enabled: true + address: "project_contact+%{key}@example.onmicrosoft.com" + user: "project_contact@example.onmicrosoft.com" + mailbox: "inbox" + delivery_method: webhook + log_path: "log/mailroom.log" + secret_file: .gitlab-mailroom-secret + inbox_method: "microsoft_graph" + inbox_options: + azure_ad_endpoint: "https://login.microsoftonline.us" + graph_endpoint: "https://graph.microsoft.us" + tenant_id: "<YOUR-TENANT-ID>" + client_id: "<YOUR-CLIENT-ID>" + client_secret: "<YOUR-CLIENT-SECRET>" + poll_interval: 60 # Optional + ``` + +::EndTabs -#### Configure a custom email address suffix +#### Configure a suffix for Service Desk alias email You can set a custom suffix in your project's Service Desk settings. @@ -467,7 +630,7 @@ When configured, the custom suffix creates a new Service Desk email address, con Prerequisites: -- You must have configured a [custom mailbox](#configure-a-custom-mailbox). +- You must have configured a [Service Desk alias email](#configure-service-desk-alias-email). 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Settings > General**. @@ -702,7 +865,7 @@ HTML emails show HTML formatting, such as: > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/386860) in GitLab 15.10. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. +On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `service_desk_new_note_email_native_attachments`. On GitLab.com, this feature is available. If a comment contains any attachments and their total size is less than or equal to 10 MB, these diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 6cc2b51f077..8330b8c14bc 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -79,7 +79,7 @@ For example: Before you can migrate projects on a self-managed GitLab instance using file exports, GitLab administrators must: -1. [Enable file exports](../../admin_area/settings/visibility_and_access_controls.md#enable-project-export) on the source +1. [Enable file exports](../../../administration/settings/visibility_and_access_controls.md#enable-project-export) on the source instance. 1. Enable file exports as an import source for the destination instance. On GitLab.com, file exports are already enabled as an import source. @@ -186,7 +186,7 @@ Items that are **not** exported include: - Links to related merge requests Migrating projects with file exports uses the same export and import mechanisms as creating projects from templates at the [group](../../group/custom_project_templates.md) and -[instance](../../admin_area/custom_project_templates.md) levels. Therefore, the list of exported items is the same. +[instance](../../../administration/custom_project_templates.md) levels. Therefore, the list of exported items is the same. ## Import a project and its data @@ -243,7 +243,7 @@ If you have a larger project, consider [using a Rake task](../../../administrati Administrators can set the maximum import file size one of two ways: - With the `max_import_size` option in the [Application settings API](../../../api/settings.md#change-application-settings). -- In the [Admin Area UI](../../admin_area/settings/account_and_limit_settings.md#max-import-size). +- In the [Admin Area UI](../../../administration/settings/account_and_limit_settings.md#max-import-size). The default is `0` (unlimited). diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 8deb05c45ef..4a7e7d2fe7d 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -41,26 +41,16 @@ To assign topics to a project: 1. Select **Save changes**. If you're an instance administrator, you can administer all project topics from the -[Admin Area's Topics page](../../admin_area/index.md#administering-topics). +[Admin Area's Topics page](../../../administration/admin_area.md#administering-topics). -## Add a compliance framework to a project **(PREMIUM)** - -[Compliance frameworks](../../group/compliance_frameworks.md) can be assigned to projects within group that has a -compliance framework using either: +NOTE: +The assigned topics are visible only to users with access to the project, but everyone can see which topics exist on the GitLab instance. Do not include sensitive information in the name of a topic. -- The GitLab UI: - 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. - 1. Select **Settings** > **General**. - 1. Expand **Compliance frameworks**. - 1. Select a compliance framework. - 1. Select **Save changes**. -- In [GitLab 14.2](https://gitlab.com/gitlab-org/gitlab/-/issues/333249) and later, using the - [GraphQL API](../../../api/graphql/reference/index.md#mutationprojectsetcomplianceframework). If you create - compliance frameworks on subgroups with GraphQL, the framework is created on the root ancestor if the user has the - correct permissions. The GitLab UI presents a read-only view to discourage this behavior. +## Add a compliance framework to a project **(PREMIUM)** -NOTE: -Frameworks can not be added to projects in personal namespaces. +You can +[add compliance frameworks to projects](../../group/compliance_frameworks.md#add-a-compliance-framework-to-a-project) +in a group that has a compliance framework. ## Configure project visibility, features, and permissions @@ -94,7 +84,6 @@ Use the toggles to enable or disable features in the project. | **Wiki** | **{check-circle}** Yes | Enables a separate system for [documentation](../wiki/index.md). | **Snippets** | **{check-circle}** Yes | Enables [sharing of code and text](../../snippets.md). | **Pages** | **{check-circle}** Yes | Allows you to [publish static websites](../pages/index.md). -| **Metrics Dashboard** | **{check-circle}** Yes | Control access to [metrics dashboard](../integrations/prometheus.md). | **Releases** | **{check-circle}** Yes | Control access to [Releases](../releases/index.md). | **Environments** | **{check-circle}** Yes | Control access to [Environments and Deployments](../../../ci/environments/index.md). | **Feature flags** | **{check-circle}** Yes | Control access to [Feature flags](../../../operations/feature_flags.md). @@ -125,6 +114,23 @@ When you disable a feature, the following additional features are also disabled: - Metrics dashboard access requires reading project environments and deployments. Users with access to the metrics dashboard can also access environments and deployments. +### Manage project access through LDAP groups + +You can [use LDAP to manage group membership](../../group/access_and_permissions.md#manage-group-memberships-via-ldap). + +You cannot use LDAP groups to manage project access, but you can use the following +workaround. + +Prerequisites: + +- You must [integrate LDAP with GitLab](../../../administration/auth/ldap/index.md). +- You must be an administrator. + +1. [Create a group](../../group/index.md#create-a-group) to track membership of your project. +1. [Set up LDAP synchronization](../../../administration/auth/ldap/ldap_synchronization.md) for that group. +1. To use LDAP groups to manage access to a project, [add the LDAP-synchronized group as a member](../members/index.md#add-groups-to-a-project) + to the project. + ## Disable CVE identifier request in issues **(FREE SAAS)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41203) in GitLab 13.4, only for public projects on GitLab.com. @@ -272,7 +278,7 @@ To transfer a project: You are redirected to the project's new page and GitLab applies a redirect. For more information about repository redirects, see [What happens when a repository path changes](../repository/index.md#what-happens-when-a-repository-path-changes). NOTE: -If you are an administrator, you can also use the [administration interface](../../admin_area/index.md#administering-projects) +If you are an administrator, you can also use the [administration interface](../../../administration/admin_area.md#administering-projects) to move any project to any namespace. ### Transferring a GitLab SaaS project to a different subscription tier @@ -302,7 +308,7 @@ To delete a project: 1. Select **Settings > General**. 1. Expand **Advanced**. 1. In the **Delete this project** section, select **Delete project**. -1. In the confirmation message text field, enter the name of the project as instructed, and select **Yes, delete project**. +1. On the confirmation dialog, enter the project name and select **Yes, delete project**. This action deletes the project and all associated resources (such as issues and merge requests). @@ -338,7 +344,7 @@ To immediately delete a project marked for deletion: 1. Select **Settings > General**. 1. Expand **Advanced**. 1. In the **Delete this project** section, select **Delete project**. -1. In the confirmation message text field, enter the name of the project as instructed, as select **Yes, delete project**. +1. On the confirmation dialog, enter the project name and select **Yes, delete project**. ## Restore a project **(PREMIUM)** diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index 7fd8fdf3a00..f4652d592f0 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -26,7 +26,7 @@ Use a project access token to authenticate: Project access tokens are similar to [group access tokens](../../group/settings/group_access_tokens.md) and [personal access tokens](../../profile/personal_access_tokens.md). -In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. +In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) as personal access tokens if the limit is set. WARNING: The ability to create project access tokens without expiry was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/369122) in GitLab 15.4 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/392855) in GitLab 16.0. In GitLab 16.0 and later, existing project access tokens without an expiry date are automatically given an expiry date of 365 days later than the current date. The automatic adding of an expiry date occurs on GitLab.com during the 16.0 milestone. The automatic adding of an expiry date occurs on self-managed instances when they are upgraded to GitLab 16.0. This change is a breaking change. @@ -39,7 +39,7 @@ You can use project access tokens: You cannot use project access tokens to create other group, project, or personal access tokens. -Project access tokens inherit the [default prefix setting](../../admin_area/settings/account_and_limit_settings.md#personal-access-token-prefix) +Project access tokens inherit the [default prefix setting](../../../administration/settings/account_and_limit_settings.md#personal-access-token-prefix) configured for personal access tokens. ## Create a project access token @@ -62,7 +62,7 @@ To create a project access token: - The token expires on that date at midnight UTC. - If you do not enter an expiry date, the expiry date is automatically set to 365 days later than the current date. - By default, this date can be a maximum of 365 days later than the current date. - - An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. + - An instance-wide [maximum lifetime](../../../administration/settings/account_and_limit_settings.md#limit-the-lifetime-of-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-project-access-token). 1. Select **Create project access token**. @@ -81,14 +81,15 @@ To revoke a project access token: The scope determines the actions you can perform when you authenticate with a project access token. -| Scope | Description | -|:-------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `api` | Grants complete read and write access to the scoped project API, including the [Package Registry](../../packages/package_registry/index.md). | -| `read_api` | Grants read access to the scoped project API, including the [Package Registry](../../packages/package_registry/index.md). | +| Scope | Description | +|:-------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `api` | Grants complete read and write access to the scoped project API, including the [Package Registry](../../packages/package_registry/index.md). | +| `read_api` | Grants read access to the scoped project API, including the [Package Registry](../../packages/package_registry/index.md). | | `read_registry` | Grants read access (pull) to the [Container Registry](../../packages/container_registry/index.md) images if a project is private and authorization is required. | | `write_registry` | Grants write access (push) to the [Container Registry](../../packages/container_registry/index.md). | -| `read_repository` | Grants read access (pull) to the repository. | -| `write_repository` | Grants read and write access (pull and push) to the repository. | +| `read_repository` | Grants read access (pull) to the repository. | +| `write_repository` | Grants read and write access (pull and push) to the repository. | +| `create_runner` | Grants permission to create runners in the project. | ## Enable or disable project access token creation diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 481eca5a890..45abcd867c0 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/115741) in GitLab 15.11. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../../administration/feature_flags.md) named `vscode_web_ide`. On GitLab.com, this feature is available. The Web IDE is an advanced editor with commit staging. You can use the Web IDE to make changes to multiple files directly from the GitLab UI. diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 2271c33b5b4..41fd7e81db5 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -14,7 +14,6 @@ to ensure all group members have the correct access permissions to contribute. Group wikis are similar to [project wikis](index.md), with a few limitations: - [Git LFS](../../../topics/git/lfs/index.md) is not supported. -- Group wikis are not included in [global search](../../search/advanced_search.md). - Changes to group wikis don't show up in the [group's activity feed](../../group/manage.md#group-activity-analytics). For updates, follow [the epic that tracks feature parity with project wikis](https://gitlab.com/groups/gitlab-org/-/epics/2782). diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md index 0d3782cfd09..4a53faeee73 100644 --- a/doc/user/project/wiki/index.md +++ b/doc/user/project/wiki/index.md @@ -37,7 +37,7 @@ To access a project wiki: - On any page in the project, use the <kbd>g</kbd> + <kbd>w</kbd> [wiki keyboard shortcut](../../shortcuts.md). -If **Wiki** is not listed in the left sidebar of your project, a project administrator +If **Plan > Wiki** is not listed in the left sidebar of your project, a project administrator has [disabled it](#enable-or-disable-a-project-wiki). ## Configure a default branch for your wiki @@ -310,11 +310,12 @@ Previously added wiki pages are preserved in case you want to re-enable the wiki. To re-enable it, repeat the process to disable the wiki but toggle it on (in blue). -## Content Editor +## Rich text editor > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5643) in GitLab 14.0. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345398) switching between editing experiences in GitLab 14.7 [with a flag](../../../administration/feature_flags.md) named `wiki_switch_between_content_editor_raw_markdown`. Enabled by default. > - Switching between editing experiences generally available in GitLab 14.10. [Feature flag `wiki_switch_between_content_editor_raw_markdown`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/83760) removed. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/398152) from content editor to rich text editor in GitLab 16.2. GitLab provides a WYSIWYG editing experience for GitLab Flavored Markdown in wikis. @@ -327,12 +328,12 @@ Support includes: - Previewing Mermaid, PlantUML, and Kroki diagrams ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/86701) in GitLab 15.2). - Creating and editing HTML comments ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/104084) in GitLab 15.7). -### Use the Content Editor +### Use the rich text editor 1. [Create](#create-a-new-wiki-page) a new wiki page, or [edit](#edit-a-wiki-page) an existing one. 1. Select **Markdown** as your format. 1. Above **Content**, select **Edit rich text**. -1. Customize your page's content using the various formatting options available in the content editor. +1. Customize your page's content using the various formatting options available in the rich text editor. 1. Select **Create page** for a new page, or **Save changes** for an existing page. The rich text editing mode remains the default until you switch back to @@ -340,12 +341,12 @@ The rich text editing mode remains the default until you switch back to ### Switch back to the old editor -1. *If you're editing the page in the content editor,* scroll to **Content**. +1. *If you're editing the page in the rich text editor,* scroll to **Content**. 1. Select **Edit source**. ### GitLab Flavored Markdown support -Supporting all GitLab Flavored Markdown content types in the Content Editor is a work in progress. +Supporting all GitLab Flavored Markdown content types in the rich text editor is a work in progress. For the status of the ongoing development for CommonMark and GitLab Flavored Markdown support, read: - [Basic Markdown formatting extensions](https://gitlab.com/groups/gitlab-org/-/epics/5404) epic. diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 5bd7c12ed31..472bbb81648 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -9,42 +9,62 @@ info: "To determine the technical writer assigned to the Stage/Group associated Most work in GitLab is done in a [project](../../user/project/index.md). Files and code are saved in projects, and most features are in the scope of projects. -## View projects +## View all projects for the instance -To view all your projects: +To view all projects for the GitLab instance: 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **View all your projects**. +1. Select **Explore**. + +On the left sidebar, **Projects** is selected. On the right, the list shows +all projects for the instance. + +If you are not authenticated, then the list shows public projects only. + +## View projects you are a member of -To browse all projects you can access: +To view projects you are a member of: 1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **Explore**. +1. Select **Your work**. -### Who can view the Projects page +On the left sidebar, **Projects** is selected. On the list, on the **Yours** tab, +all the projects you are a member of are displayed. -When you select a project, the project landing page shows the project contents. +## View personal projects -For public projects, and members of internal and private projects -with [permissions to view the project's code](../permissions.md#project-members-permissions), -the project landing page shows: +Personal projects are projects created under your personal namespace. -- A [`README` or index file](repository/index.md#readme-and-index-files). -- A list of directories in the project's repository. +For example, if you create an account with the username `alex`, and create a project +called `my-project` under your username, the project is created at `https://gitlab.example.com/alex/my-project`. -For users without permission to view the project's code, the landing page shows: +To view your personal projects: -- The wiki homepage. -- The list of issues in the project. +1. On the left sidebar, select your avatar and then your username. +1. On the left sidebar, select **Personal projects**. + +## View starred projects -### Access a project page with the project ID +To view projects you have [starred](#star-a-project): -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53671) in GitLab 11.8. +1. On the left sidebar, select your avatar and then your username. +1. On the left sidebar, select **Starred projects**. + +## Organizing projects with topics + +Topics are labels that you can assign to projects to help you organize and find them. +A topic is typically a short name that describes the content or purpose of a project. +You can assign a topic to several projects. + +For example, you can create and assign the topics `python` and `hackathon` to all projects that use Python and are intended for Hackathon contributions. -To access a project from the GitLab UI using the project ID, -visit the `/projects/:id` URL in your browser or other tool accessing the project. +Topics assigned to a project are listed in the **Project overview**, below the project name and activity information. -## Explore topics +Only users with access to the project can see the topics assigned to that project, +but everyone (including unauthenticated users) can see the topics available on the GitLab instance. +Do not include sensitive information in the name of a topic. + +### Explore topics To explore project topics: @@ -53,47 +73,67 @@ To explore project topics: 1. On the left sidebar, select **Topics**. 1. To view projects associated with a topic, select a topic. -The **Explore topics** page shows a list of topics, sorted by the number of associated projects. +The **Explore topics** page shows a list of projects with this topic. -You can assign topics to a project on the [Project Settings page](settings/index.md#assign-topics-to-a-project). +### Filter and sort topics -If you're an instance administrator, you can administer all project topics from the -[Admin Area's Topics page](../admin_area/index.md#administering-topics). +You can filter the list of projects that have a certain topic by: -## Star a project +- Name +- Language +- Owner +- Archive status +- Visibility -You can add a star to projects you use frequently to make them easier to find. +You can sort the projects by: -To add a star to a project: +- Date created +- Date updated +- Name +- Number of stars -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. In the upper-right corner of the page, select **Star**. +### Subscribe to a topic -## View starred projects +If you want to know when new projects are added to a topic, you can use its RSS feed. -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **View all your projects**. -1. Select the **Starred** tab. -1. GitLab displays information about your starred projects, including: +You can do this either from the **Explore topics** page or a project with topics. - - Project description, including name, description, and icon. - - Number of times this project has been starred. - - Number of times this project has been forked. - - Number of open merge requests. - - Number of open issues. +To subscribe to a topic: -## View personal projects +- From the **Explore topics** page: -Personal projects are projects created under your personal namespace. + 1. On the left sidebar, expand the top-most chevron ({**chevron-down**}). + 1. Select **Explore**. + 1. Select **Topics**. + 1. Select the topic you want to subscribe to. + 1. In the upper-right corner, select **Subscribe to the new projects feed** (**{rss}**). -For example, if you create an account with the username `alex`, and create a project -called `my-project` under your username, the project is created at `https://gitlab.example.com/alex/my-project`. +- From a project: -To view your personal projects: + 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. + 1. In the **Project overview** page, from the **Topics** list select the topic you want to subscribe to. + 1. In the upper-right corner, select **Subscribe to the new projects feed** (**{rss}**). -1. On the left sidebar, expand the top-most chevron (**{chevron-down}**). -1. Select **View all your projects**. -1. In the **Yours** tab, select **Personal**. +The results are displayed as an RSS feed in Atom format. +The URL of the result contains a feed token and the list of projects that have the topic. You can add this URL to your feed reader. + +### Assign a topic to a project + +You can assign topics to a project on the [Project Settings page](settings/index.md#assign-topics-to-a-project). + +### Administer topics + +Instance administrators can administer all project topics from the +[Admin Area's Topics page](../../administration/admin_area.md#administering-topics). + +## Star a project + +You can add a star to projects you use frequently to make them easier to find. + +To add a star to a project: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. In the upper-right corner of the page, select **Star**. ## Delete a project @@ -188,6 +228,34 @@ Prerequisite: 1. Use the toggle by each feature you want to turn on or off, or change access for. 1. Select **Save changes**. +## Access the Project overview page by using the project ID + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53671) in GitLab 11.8. + +To access a project by using the project ID instead of its name, +go to `https://gitlab.example.com/projects/:id`. + +The project ID is displayed in the **Project overview** page, under the project name. + +For example, if in your personal namespace `alex` you have a project `my-project` with the ID `123456`, you can access the project +either at `https://gitlab.example.com/alex/my-project` or `https://gitlab.example.com/projects/123456`. + +## Who can view the Project overview page + +When you select a project, the **Project overview** page shows the project contents. + +For public projects, and members of internal and private projects +with [permissions to view the project's code](../permissions.md#project-members-permissions), +the project landing page shows: + +- A [`README` or index file](repository/index.md#readme-and-index-files). +- A list of directories in the project's repository. + +For users without permission to view the project's code, the landing page shows: + +- The wiki homepage. +- The list of issues in the project. + ## Leave a project When you leave a project: diff --git a/doc/user/public_access.md b/doc/user/public_access.md index 002cb97dd93..57439a4595e 100644 --- a/doc/user/public_access.md +++ b/doc/user/public_access.md @@ -7,18 +7,24 @@ type: reference # Project and group visibility **(FREE)** -A project in GitLab can be private, internal, or public. +Projects and groups in GitLab can be private, internal, or public. + +The visibility level of the group or project has no influence on whether members within the group or project can see each other. +A group or project is an object to allow collaborative work. This is only possible if all members know about each other. + +Group or project members can see all members of the group or project they belong to. +Group or project owners can see the origin of membership (the original group or project) of all members. ## Private projects and groups -For private projects, only project members can: +For private projects, only members of the private project or group can: - Clone the project. - View the public access directory (`/public`). Users with the Guest role cannot clone the project. -Private groups can have private subgroups only. +Private groups can have only private subgroups. ## Internal projects and groups **(FREE SELF)** @@ -27,6 +33,8 @@ For internal projects, **any authenticated user**, including users with the Gues - Clone the project. - View the public access directory (`/public`). +Only internal members can view internal content. + [External users](admin_area/external_users.md) cannot clone the project. Internal groups can have internal or private subgroups. @@ -48,7 +56,7 @@ Public groups can have public, internal, or private subgroups. NOTE: If an administrator restricts the -[**Public** visibility level](admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels), +[**Public** visibility level](../administration/settings/visibility_and_access_controls.md#restrict-visibility-levels), then `/public` is visible only to authenticated users. ## Change project visibility @@ -91,7 +99,7 @@ Prerequisites: Administrators can restrict which visibility levels users can choose when they create a project or a snippet. This setting can help prevent users from publicly exposing their repositories by accident. -For more information, see [Restrict visibility levels](admin_area/settings/visibility_and_access_controls.md#restrict-visibility-levels). +For more information, see [Restrict visibility levels](../administration/settings/visibility_and_access_controls.md#restrict-visibility-levels). ## Related topics diff --git a/doc/user/report_abuse.md b/doc/user/report_abuse.md index c4b9af28220..23a662ccfeb 100644 --- a/doc/user/report_abuse.md +++ b/doc/user/report_abuse.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can report abuse from other GitLab users to GitLab administrators. -A GitLab administrator [can then choose](admin_area/review_abuse_reports.md) to: +A GitLab administrator [can then choose](../administration/review_abuse_reports.md) to: - Remove the user, which deletes them from the instance. - Block the user, which denies them access to the instance. @@ -66,4 +66,4 @@ A URL to the reported user's comment is pre-filled in the abuse report's ## Related topics -- [Abuse reports administration documentation](admin_area/review_abuse_reports.md) +- [Abuse reports administration documentation](../administration/review_abuse_reports.md) diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 6e8a7e7c1cf..bc9de9357f6 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -20,7 +20,7 @@ under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists ## Limitations on project and group names - Project or group names must start with a letter, digit, emoji, or "_". -- Project or group names can only contain letters, digits, emojis, "_", ".", "+", dashes, or spaces. +- Project or group names can only contain letters, digits, emoji, "_", ".", "+", dashes, or spaces. - Project or group slugs must start with a letter or digit. - Project or group slugs can only contain letters, digits, '_', '.', '+', or dashes. - Project or group slugs must not contain consecutive special characters. diff --git a/doc/user/rich_text_editor.md b/doc/user/rich_text_editor.md new file mode 100644 index 00000000000..ea85dfdbcb4 --- /dev/null +++ b/doc/user/rich_text_editor.md @@ -0,0 +1,134 @@ +--- +stage: Plan +group: Knowledge +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +type: index, reference +--- + +# Rich text editor **(FREE)** + +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5643) for [wikis](project/wiki/index.md#rich-text-editor) in GitLab 14.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/371931) for editing issue descriptions in GitLab 15.5 [with a flag](../administration/feature_flags.md) named `content_editor_on_issues`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/382636) for [discussions](discussions/index.md), and creating and editing issues and merge requests in GitLab 15.11 with the same flag. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407507) for epics in GitLab 16.1 with the same flag. +> - Feature flag `content_editor_on_issues` enabled by default in GitLab 16.2. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator +can [disable the feature flag](../administration/feature_flags.md) named `content_editor_on_issues`. +On GitLab.com, this feature is available. + +The rich text editor is a "what you see is what you get" (WYSIWYG) editor so you can use +[GitLab Flavored Markdown](markdown.md) in descriptions and comments, even if you can't remember all +of its syntax. + + + +Rich text editor is available in: + +- [Wikis](project/wiki/index.md) +- Issues +- Epics +- Merge requests +- [Designs](project/issues/design_management.md) + +Features of the editor include: + +- Format text, including as bold, italics, block quotes, headings, and inline code. +- Format ordered lists, unordered lists, and checklists. +- Insert links, attachments, images, video, and audio. +- Create and edit a table structure. +- Insert and format code blocks with syntax highlighting. +- Preview Mermaid, PlantUML, and Kroki diagrams in real time. + +To track work on adding the rich text editor to more places across GitLab, see +[epic 7098](https://gitlab.com/groups/gitlab-org/-/epics/7098). + +## Switch to the rich text editor + +Use the rich text editor to edit descriptions, wiki pages, add comments. + +To switch to the rich text editor: In a text box, in the lower-left corner, select +**Switch to rich text editing**. + +## Switch to the plain text editor + +If you want to enter Markdown source in the text box, return to using the plain text editor. + +To switch to the plain text editor: In a text box, in the lower-left corner, select +**Switch to plain text editing**. + +## Compatibility with GitLab Flavored Markdown + +The rich text editor is fully compatible with [GitLab Flavored Markdown](markdown.md). +It means that you can switch between plain text and rich text modes without losing any data. + +### Input rules + +Rich text editor also supports input rules that let you work with rich content as if you were +typing Markdown. + +Supported input rules: + +| Input rule syntax | Content inserted | +| --------------------------------------------------------- | -------------------- | +| `# Heading 1` <br>... <br> `###### Heading 6` | Headings 1 through 6 | +| `**bold**` or `__bold__` | Bold text | +| `_italics_` or `*italics*` | Italicized text | +| `~~strike~~` | Strikethrough | +| `[link](url)` | Hyperlink | +| `code` | Inline code | +| <code>```rb</code> + <kbd>Enter</kbd> <br> <code>```js</code> + <kbd>Enter</kbd> | Code block | +| `* List item`, or<br> `- List item`, or<br> `+ List item` | Unordered list | +| `1. List item` | Numbered list | +| `<details>` | Collapsible section | + +## Tables + +Unlike in raw Markdown, you can use the rich text editor to insert block content paragraphs, +list items, diagrams (or even another table!) in table cells. + +### Insert a table + +To insert a table: + +1. Select **Insert table** **{table}**. +1. From the dropdown list, select the dimensions of the new table. + + + +### Edit a table + +Inside a table cell, you can use a menu to insert or delete rows or columns. + +To open the menu: In the upper-right corner of a cell, select the chevron **{chevron-down}**. + + + +### Operations on multiple cells + +Select multiple cells and merge or split them. + +To merge selected cells into one: + +1. Select multiple cells - select one and drag your cursor. +1. In the upper-right corner of a cell, select the chevron **{chevron-down}** **> Merge N cells**. + +To split merged cells: In the upper-right corner of a cell, select the chevron **{chevron-down}** **> Split cell**. + +## Insert diagrams + +Insert [Mermaid](https://mermaidjs.github.io/) and [PlantUML](https://plantuml.com/) diagrams and +preview them live as you type the diagram code. + +To insert a diagram: + +1. On the top bar of a text box, select **{plus}** **More options** and then **Mermaid diagram** or **PlantUML diagram**. +1. Enter the code for your diagram. The diagram preview appears in the text box. + + + +## Related topics + +- [Keyboard shortcuts](shortcuts.md#rich-text-editor) for rich text editor +- [GitLab Flavored Markdown](markdown.md) diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md index 1444e5385f9..5a9f75f1d6c 100644 --- a/doc/user/search/advanced_search.md +++ b/doc/user/search/advanced_search.md @@ -27,7 +27,7 @@ You can use advanced search in: - Code - Comments - Commits -- Project wikis (not [group wikis](../project/wiki/group.md)) +- Project and group wikis ## Enable advanced search diff --git a/doc/user/search/command_palette.md b/doc/user/search/command_palette.md new file mode 100644 index 00000000000..ab284bd6a5f --- /dev/null +++ b/doc/user/search/command_palette.md @@ -0,0 +1,30 @@ +--- +stage: Manage +group: Foundations +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments" +type: reference +--- + +# Command palette **(FREE)** + +> Introduced in GitLab 16.2 [with a flag](../../administration/feature_flags.md) named `command_palette`. Enabled by default. + +You can use command palette to narrow down the scope of your search or to +find an object more quickly. + +FLAG: +On self-managed GitLab, by default this feature is available. +To hide the feature, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `command_palette`. +On GitLab.com, this feature is available. + +## Open the command palette + +To open the command palette: + +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) or use the <kbd>/</kbd> key to enable. +1. Type one of the special characters: + + - <kbd>></kbd> - Create a new object or find a menu item. + - <kbd>@</kbd> - Search for a user. + - <kbd>:</kbd> - Search for a project. + - <kbd>/</kbd> - Search for project files in the default repository branch. diff --git a/doc/user/search/img/code_search_git_blame_v15_1.png b/doc/user/search/img/code_search_git_blame_v15_1.png Binary files differdeleted file mode 100644 index 426f829b186..00000000000 --- a/doc/user/search/img/code_search_git_blame_v15_1.png +++ /dev/null diff --git a/doc/user/search/img/project_search_general_settings_v13_8.png b/doc/user/search/img/project_search_general_settings_v13_8.png Binary files differdeleted file mode 100644 index 08395e0d4f9..00000000000 --- a/doc/user/search/img/project_search_general_settings_v13_8.png +++ /dev/null diff --git a/doc/user/search/img/project_search_sha_redirect.png b/doc/user/search/img/project_search_sha_redirect.png Binary files differdeleted file mode 100644 index 718a859d4af..00000000000 --- a/doc/user/search/img/project_search_sha_redirect.png +++ /dev/null diff --git a/doc/user/search/img/search_navbar_v15_7.png b/doc/user/search/img/search_navbar_v15_7.png Binary files differdeleted file mode 100644 index 775c7f32b37..00000000000 --- a/doc/user/search/img/search_navbar_v15_7.png +++ /dev/null diff --git a/doc/user/search/img/search_scope_v15_7.png b/doc/user/search/img/search_scope_v15_7.png Binary files differdeleted file mode 100644 index 96f6e985523..00000000000 --- a/doc/user/search/img/search_scope_v15_7.png +++ /dev/null diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 2278da9a714..fe5ee3a5e0a 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Searching in GitLab **(FREE)** -GitLab has two types of searches available: _basic_ and _advanced_. +GitLab has two types of searches available: **basic** and **advanced**. Both types of search are the same, except when you are searching through code. @@ -27,16 +27,15 @@ can limit the search scope by disabling the following [`ops` feature flags](../. | Issues | `global_search_issues_tab` | When enabled, global search includes issues. | | Merge requests | `global_search_merge_requests_tab` | When enabled, global search includes merge requests. | | Users | `global_search_users_tab` | When enabled, global search includes users. | -| Wiki | `global_search_wiki_tab` | When enabled, global search includes project wikis (not [group wikis](../project/wiki/group.md)). | +| Wiki | `global_search_wiki_tab` | When enabled, global search includes project and [group wikis](../project/wiki/group.md). | -All global search scopes are enabled by default on GitLab.com -and self-managed instances. +All global search scopes are enabled by default on self-managed instances. ## Global search validation Global search ignores and logs as abusive any search with: -- Fewer than 2 characters +- Fewer than two characters - A term longer than 100 characters (URL search terms must not exceed 200 characters) - A stop word only (for example, `the`, `and`, or `if`) - An unknown `scope` @@ -48,29 +47,36 @@ Global search only flags with an error any search that includes more than: - 4096 characters - 64 terms -## Perform a search +## Search in all GitLab -To start a search, in the upper-right corner of the screen, in the search bar, type your search query. -You must type at least two characters. +To search in all GitLab: - +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**). +1. Type your search query. You must type at least two characters. +1. Press <kbd>Enter</kbd> to search, or select from the list. -After the results are displayed, you can modify the search, select a different type of data to -search, or choose a specific group or project. +The results are displayed. To filter the results, on the left sidebar, select a filter. - +## Search in a project -## Search in code +To search in a project: -To search through code or other documents in a project: +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. Select **Search GitLab** (**{search}**) again and type the string you want to search for. +1. Press <kbd>Enter</kbd> to search, or select from the list. + +The results are displayed. To filter the results, on the left sidebar, select a filter. + +## Search for code + +To search for code in a project: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and type the string you want to search for. -1. Press **Enter**. +1. Select **Search GitLab** (**{search}**) again and type the code you want to search for. +1. Press <kbd>Enter</kbd> to search, or select from the list. Code search shows only the first result in the file. - -To search across all of GitLab, ask your administrator to enable [advanced search](advanced_search.md). +To search for code in all GitLab, ask your administrator to enable [advanced search](advanced_search.md). ### View Git blame from code search @@ -82,8 +88,6 @@ where the results were found. 1. From the code search result, hover over the line number. 1. On the left, select **View blame**. - - ### Filter code search results by language > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342651) in GitLab 15.10. @@ -93,87 +97,89 @@ To filter code search results by one or more languages: 1. On the code search page, on the left sidebar, select one or more languages. 1. On the left sidebar, select **Apply**. -## Search for projects by full path +## Exclude search results -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114932) in GitLab 15.11. Feature flag `full_path_project_search` removed. +### From archived projects -You can search for a project by entering its full path (including the namespace it belongs to) in the search box. -As you type the project path, [autocomplete suggestions](#autocomplete-suggestions) are displayed. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121981) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `search_projects_hide_archived`. Disabled by default. -For example, the search query: +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `search_projects_hide_archived`. On GitLab.com, this feature is not available. -- `gitlab-org/gitlab` searches for the `gitlab` project in the `gitlab-org` namespace. -- `gitlab-org/` displays autocomplete suggestions for projects that belong to the `gitlab-org` namespace. +Archived projects are included in search results by default. To exclude archived projects, ensure the `search_projects_hide_archived` flag is enabled. -### Exclude archived projects from project search results +To include archived projects with `search_projects_hide_archived` enabled, you must add the parameter `include_archived=true` to the URL. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/121981) in GitLab 16.1 [with a flag](../../administration/feature_flags.md) named `search_projects_hide_archived`. Disabled by default. +### From issues in archived projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124846) in GitLab 16.2 [with a flag](../../administration/feature_flags.md) named `search_issues_hide_archived_projects`. Disabled by default. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `search_projects_hide_archived`. On GitLab.com, this feature is not available. +an administrator can [enable the feature flag](../../administration/feature_flags.md) named `search_issues_hide_archived_projects`. On GitLab.com, this feature is not available. -Archived projects are included in project search results by default. To exclude archived projects, ensure the `search_projects_hide_archived` flag is enabled. +Issues in archived projects are included in search results by default. To exclude issues in archived projects, ensure the `search_issues_hide_archived_projects` flag is enabled. -To include archived projects with `search_projects_hide_archived` enabled, you must add the parameter `include_archived=true` to the URL. +To include issues in archived projects with `search_issues_hide_archived_projects` enabled, you must add the parameter `include_archived=true` to the URL. -## Search for a SHA +## Search for a project by full path -You can search for a commit SHA. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/108906) in GitLab 15.9 [with a flag](../../administration/feature_flags.md) named `full_path_project_search`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/114932) in GitLab 15.11. Feature flag `full_path_project_search` removed. + +You can search for a project by entering its full path (including the namespace it belongs to) in the search box. +As you type the project path, [autocomplete suggestions](#autocomplete-suggestions) are displayed. + +For example: + +- `gitlab-org/gitlab` searches for the `gitlab` project in the `gitlab-org` namespace. +- `gitlab-org/` displays autocomplete suggestions for projects that belong to the `gitlab-org` namespace. + +## Search for a commit SHA + +To search for a commit SHA: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. -1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) and type the SHA. +1. Select **Search GitLab** (**{search}**) again and type the commit SHA you want to search for. +1. Press <kbd>Enter</kbd> to search, or select from the list. If a single result is returned, GitLab redirects to the commit result and gives you the option to return to the search results page. - +## Search for specific terms -## Searching for specific terms - -> - [Removed support for partial matches in issue searches](https://gitlab.com/gitlab-org/gitlab/-/issues/273784) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `issues_full_text_search`. Disabled by default. -> - Feature flag [`issues_full_text_search` enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/273784) in GitLab 14.10. -> - Feature flag [`issues_full_text_search` enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/273784) in GitLab 15.2. +> - [Support for partial matches in issue search](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71913) removed in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `issues_full_text_search`. Disabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/124703) in GitLab 16.2. Feature flag `issues_full_text_search` removed. You can filter issues and merge requests by specific terms included in titles or descriptions. - Syntax - Searches look for all the words in a query, in any order. For example: searching issues for `display bug` returns all issues matching both those words, in any order. - - To find the exact term, use double quotes: `"display bug"` + - To find the exact term, use double quotes: `"display bug"`. - Limitation - - For performance reasons, terms shorter than 3 chars are ignored. For example: searching + - For performance reasons, terms shorter than three characters are ignored. For example: searching issues for `included in titles` is same as `included titles` - Search is limited to 4096 characters and 64 terms per query. - When searching issues, partial matches are not allowed. For example: searching for `play` will not return issues that have the word `display`. But variations of words match, so searching for `displays` also returns issues that have the word `display`. -## Retrieve search results as feed - -> Feeds for merge requests were [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66336) in GitLab 14.3. - -GitLab provides RSS feeds of search results for your project. To subscribe to the -RSS feed of search results: +## Run a search from history -1. Go to your project's page. -1. On the left sidebar, select **Issues** or **Merge requests**. -1. Perform a search. -1. Select the feed symbol **{rss}** to display the results as an RSS feed in Atom format. - -The URL of the result contains both a feed token, and your search query. -You can add this URL to your feed reader. +You can run a search from history for issues and merge requests. Search history is stored locally +in your browser. To run a search from history: -## Search history +1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. +1. To view recent searches: -Search history is available for issues and merge requests, and is stored locally -in your browser. To run a search from history: + - For issues, on the left sidebar, select **Plan > Issues**. Above the list, to the left of the search box, select (**{history}**). + - For merge requests, on the left sidebar, select **Code > Merge requests**. Above the list, to the left of the search box, select **Recent searches**. -1. In the top menu, select **Issues** or **Merge requests**. -1. To the left of the search bar, select **Recent searches**, and select a search from the list. +1. From the dropdown list, select a search. -## Removing search filters +## Remove search filters Individual filters can be removed by selecting the filter's (x) button or backspacing. The entire search filter can be cleared by selecting the search box's (x) button or via <kbd>⌘</kbd> (Mac) + <kbd>⌫</kbd>. @@ -181,9 +187,9 @@ To delete filter tokens one at a time, the <kbd>⌥</kbd> (Mac) / <kbd>Control</ ## Autocomplete suggestions -In the search bar, you can view autocomplete suggestions for: +In the search box, you can view autocomplete suggestions for: -- [Projects](#search-for-projects-by-full-path) and groups +- [Projects](#search-for-a-project-by-full-path) and groups - Users - Various help pages (try and type **API help**) - Project feature pages (try and type **milestones**) @@ -195,13 +201,6 @@ In the search bar, you can view autocomplete suggestions for: ## Search settings -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292941) in GitLab 13.8 [with a flag](../../administration/feature_flags.md) named `search_settings_in_page`. Disabled by default. -> - [Added](https://gitlab.com/groups/gitlab-org/-/epics/4842) to Group, Administrator, and User settings in GitLab 13.9. -> - [Feature flag `search_settings_in_page` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294025) in GitLab 13.11. -> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/294025) in GitLab 13.11. - You can search inside a Project, Group, Administrator, or User's settings by entering a search term in the search box located at the top of the page. The search results appear highlighted in the sections that match the search term. - - diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index e195be5586a..ac3c6bace09 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -68,24 +68,24 @@ relatively quickly to work, and they take you to another page in the project. | Keyboard shortcut | Description | |-----------------------------|-------------| -| <kbd>g</kbd> + <kbd>o</kbd> | Go to the project overview page (**Project > Details**). | -| <kbd>g</kbd> + <kbd>v</kbd> | Go to the project activity feed (**Project > Activity**). | -| <kbd>g</kbd> + <kbd>r</kbd> | Go to the project releases list (**Project > Releases**). | -| <kbd>g</kbd> + <kbd>f</kbd> | Go to the [project files](#project-files) list (**Repository > Files**). | -| <kbd>t</kbd> | Go to the project file search page. (**Repository > Files**, select **Find Files**). | -| <kbd>g</kbd> + <kbd>c</kbd> | Go to the project commits list (**Repository > Commits**). | -| <kbd>g</kbd> + <kbd>n</kbd> | Go to the [repository graph](#repository-graph) page (**Repository > Graph**). | -| <kbd>g</kbd> + <kbd>d</kbd> | Go to repository charts (**Analytics > Repository Analytics**). | -| <kbd>g</kbd> + <kbd>i</kbd> | Go to the project issues list (**Issues > List**). | -| <kbd>i</kbd> | Go to the New Issue page (**Issues**, select **New Issue** ). | -| <kbd>g</kbd> + <kbd>b</kbd> | Go to the project issue boards list (**Issues > Boards**). | -| <kbd>g</kbd> + <kbd>m</kbd> | Go to the project [merge requests](project/merge_requests/index.md) list (**Merge Requests**). | -| <kbd>g</kbd> + <kbd>p</kbd> | Go to the CI/CD pipelines list (**CI/CD > Pipelines**). | -| <kbd>g</kbd> + <kbd>j</kbd> | Go to the CI/CD jobs list (**CI/CD > Jobs**). | -| <kbd>g</kbd> + <kbd>e</kbd> | Go to the project environments (**Deployments > Environments**). | -| <kbd>g</kbd> + <kbd>k</kbd> | Go to the project Kubernetes cluster integration page (**Infrastructure > Kubernetes clusters**). You must have at least [`maintainer` permissions](permissions.md) to access this page. | -| <kbd>g</kbd> + <kbd>s</kbd> | Go to the project snippets list (**Snippets**). | -| <kbd>g</kbd> + <kbd>w</kbd> | Go to the [project wiki](project/wiki/index.md) (**Wiki**), if enabled. | +| <kbd>g</kbd> + <kbd>o</kbd> | Go to the project overview page (**Project overview**). | +| <kbd>g</kbd> + <kbd>v</kbd> | Go to the project activity feed (**Manage > Activity**). | +| <kbd>g</kbd> + <kbd>r</kbd> | Go to the project releases list (**Deploy > Releases**). | +| <kbd>g</kbd> + <kbd>f</kbd> | Go to the [project files](#project-files) list (**Code > Repository**). | +| <kbd>t</kbd> | Go to the project file search page. (**Code > Repository**, select **Find Files**). | +| <kbd>g</kbd> + <kbd>c</kbd> | Go to the project commits list (**Code > Commits**). | +| <kbd>g</kbd> + <kbd>n</kbd> | Go to the [repository graph](#repository-graph) page (**Code > Repository graph**). | +| <kbd>g</kbd> + <kbd>d</kbd> | Go to repository charts (**Analyze > Repository analytics**). | +| <kbd>g</kbd> + <kbd>i</kbd> | Go to the project issues list (**Plan > Issues**). | +| <kbd>i</kbd> | Go to the New Issue page (**Pan > Issues**, select **New issue** ). | +| <kbd>g</kbd> + <kbd>b</kbd> | Go to the project issue boards list (**Plan > Issue boards**). | +| <kbd>g</kbd> + <kbd>m</kbd> | Go to the project [merge requests](project/merge_requests/index.md) list (**Code > Merge requests**). | +| <kbd>g</kbd> + <kbd>p</kbd> | Go to the CI/CD pipelines list (**Build > Pipelines**). | +| <kbd>g</kbd> + <kbd>j</kbd> | Go to the CI/CD jobs list (**Build > Jobs**). | +| <kbd>g</kbd> + <kbd>e</kbd> | Go to the project environments (**Operate > Environments**). | +| <kbd>g</kbd> + <kbd>k</kbd> | Go to the project Kubernetes cluster integration page (**Operate > Kubernetes clusters**). You must have at least [`maintainer` permissions](permissions.md) to access this page. | +| <kbd>g</kbd> + <kbd>s</kbd> | Go to the project snippets list (**Code > Snippets**). | +| <kbd>g</kbd> + <kbd>w</kbd> | Go to the [project wiki](project/wiki/index.md) (**Plan > Wiki**), if enabled. | | <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | ### Issues @@ -123,14 +123,14 @@ These shortcuts are available when viewing [merge requests](project/merge_reques ### Project files These shortcuts are available when browsing the files in a project (go to -**Repository > Files**): +**Code > Repository**): | Keyboard shortcut | Description | |-------------------|-------------| | <kbd>↑</kbd> | Move selection up. | | <kbd>↓</kbd> | Move selection down. | | <kbd>Enter</kbd> | Open selection. | -| <kbd>Escape</kbd> | Go back to file list screen (only while searching for files, **Repository > Files**, then select **Find File**). | +| <kbd>Escape</kbd> | Go back to file list screen (only while searching for files, **Code > Repository**, then select **Find File**). | | <kbd>y</kbd> | Go to file permalink (only while viewing a file). | | <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | @@ -219,7 +219,7 @@ These shortcuts are available when editing a file with the [Web IDE](project/web ### Repository graph These shortcuts are available when viewing the project [repository graph](project/repository/index.md#repository-history-graph) -page (go to **Repository > Graph**): +page (go to **Code > Repository graph**): | Keyboard shortcut | Description | |--------------------------------------------------------------------|-------------| @@ -238,10 +238,10 @@ This shortcut is available when viewing a [wiki page](project/wiki/index.md): |-------------------|-----------------| | <kbd>e</kbd> | Edit wiki page. | -### Content editor +### Rich text editor These shortcuts are available when editing a file with the -[Content Editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/): +[rich text editor](https://about.gitlab.com/direction/plan/knowledge/content_editor/): | macOS shortcut | Windows shortcut | Description | |----------------|------------------|-------------| @@ -261,6 +261,7 @@ These shortcuts are available when editing a file with the | <kbd>Command</kbd> + <kbd>i</kbd> | <kbd>Control</kbd> + <kbd>i</kbd> | Italic | | <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | Strikethrough | | <kbd>Command</kbd> + <kbd>e</kbd> | <kbd>Control</kbd> + <kbd>e</kbd> | Code | +| <kbd>Command</kbd> + <kbd>k</kbd> | <kbd>Control</kbd> + <kbd>k</kbd> | Insert a link | | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | Apply normal text style | | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | Apply heading style 1 | | <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | Apply heading style 2 | diff --git a/doc/user/snippets.md b/doc/user/snippets.md index 3ad003e4f4c..aace26b4bb0 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -38,19 +38,15 @@ visibility setting keep this setting. You can read more about the change in the You can create snippets in multiple ways, depending on whether you want to create a personal or project snippet: 1. Select the kind of snippet you want to create: - - **To create a personal snippet**: On the - [Snippets dashboard](https://gitlab.com/dashboard/snippets), select - **New snippet**, or: - - *If you're on a project's page,* select the plus icon (**{plus-square-o}**) - in the top navigation bar, and then select **New snippet** from the - **GitLab** section of the same dropdown list. - - *For all other pages,* select the plus icon (**{plus-square-o}**) - in the top navigation bar, then select **New snippet** from the dropdown list. + - **To create a personal snippet**, do one of the following: + - On the [Snippets dashboard](https://gitlab.com/dashboard/snippets), select + **New snippet**. + - From a project: On the left sidebar, select **Create new** (**{plus}**). Below **In GitLab**, select **New snippet**. + - From any other page: On the left sidebar, select **Create new** (**{plus}**) and then **New snippet**. - If you installed the [GitLab Workflow VS Code extension](project/repository/vscode.md), use the [`Gitlab: Create snippet` command](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#create-snippet). - - **To create a project snippet**: Go to your project's page. Select the - plus icon (**{plus-square-o}**), and then select **New snippet** from the - **This project** section of the dropdown list. + - **To create a project snippet**: Go to your project's page. Select + **Create new** (**{plus}**). Below **In this project**, select **New snippet**. 1. Add a **Title** and **Description**. 1. Name your **File** with an appropriate extension, such as `example.rb` or `index.html`. Filenames with appropriate extensions display [syntax highlighting](#filenames). @@ -257,7 +253,7 @@ GitLab forwards the spam to Akismet. ### Reduce snippets repository size -Because versioned snippets are considered as part of the [namespace storage size](../user/admin_area/settings/account_and_limit_settings.md), +Because versioned snippets are considered as part of the [namespace storage size](../administration/settings/account_and_limit_settings.md), it's recommended to keep snippets' repositories as compact as possible. For more information about tools to compact repositories, diff --git a/doc/user/ssh.md b/doc/user/ssh.md index a11f3c4dbd6..d92e3944521 100644 --- a/doc/user/ssh.md +++ b/doc/user/ssh.md @@ -76,7 +76,7 @@ must have [OpenSSH 8.2](https://www.openssh.com/releasenotes.html#8.2) or later Available documentation suggests ED25519 is more secure than RSA. -If you use an RSA key, the US National Institute of Science and Technology in +If you use an RSA key, the US National Institute of Standards and Technology in [Publication 800-57 Part 3 (PDF)](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-57Pt3r1.pdf) recommends a key size of at least 2048 bits. The default key size depends on your version of `ssh-keygen`. Review the `man` page for your installed `ssh-keygen` command for details. @@ -336,7 +336,7 @@ To use SSH with GitLab, copy your public key to your GitLab account: In: - GitLab 13.12 and earlier, the expiration date is informational only. It doesn't prevent you from using the key. Administrators can view expiration dates and use them for - guidance when [deleting keys](admin_area/credentials_inventory.md#delete-a-users-ssh-key). + guidance when [deleting keys](../administration/credentials_inventory.md#delete-a-users-ssh-key). - GitLab checks all SSH keys at 02:00 AM UTC every day. It emails an expiration notice for all SSH keys that expire on the current date. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) - GitLab checks all SSH keys at 01:00 AM UTC every day. It emails an expiration notice for all SSH keys that are scheduled to expire seven days from now. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.11.) 1. Select **Add key**. diff --git a/doc/user/tasks.md b/doc/user/tasks.md index b14f4acca36..ecd5ef0c42f 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -18,7 +18,7 @@ For the latest updates, check the [Tasks Roadmap](https://gitlab.com/groups/gitl FLAG: On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flags](../administration/feature_flags.md) named `work_items`. +an administrator can [disable the feature flags](../administration/feature_flags.md) named `work_items`. On GitLab.com, this feature is available. Use tasks to track steps needed for the [issue](project/issues/index.md) to be closed. @@ -294,7 +294,7 @@ To set issue weight of a task: > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/367456) in GitLab 15.7. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `work_items_mvc`. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../administration/feature_flags.md) named `work_items_mvc`. On GitLab.com, this feature is available. You can add a task to an [iteration](group/iterations/index.md). @@ -359,3 +359,18 @@ To copy the task's email address: 1. On the left sidebar, at the top, select **Search GitLab** (**{search}**) to find your project. 1. Select **Plan > Issues**, then select your issue to view it. 1. In the top right corner, select the vertical ellipsis (**{ellipsis_v}**), then select **Copy task email address**. + +## Two-column layout + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/415077) in GitLab 16.2 [with a flag](../administration/feature_flags.md) named `work_items_mvc_2`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, an administrator can [enable the feature flag](../administration/feature_flags.md) named `work_items_mvc_2`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +When enabled, tasks use a two-column layout, similar to issues. +The description and threads are on the left, and attributes, such as labels +or assignees, on the right. + + diff --git a/doc/user/todos.md b/doc/user/todos.md index 95bc8a553c1..97f67c67671 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -69,7 +69,7 @@ To-do items aren't affected by [GitLab notification email settings](profile/noti FLAG: On self-managed GitLab, by default this feature is not available. To make it available per user, -ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `multiple_todos`. +an administrator can [enable the feature flag](../administration/feature_flags.md) named `multiple_todos`. On GitLab.com, this feature is not available. The feature is not ready for production use. @@ -81,7 +81,7 @@ When you enable this feature: ## Create a to-do item -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390549) in objectives, key results and, tasks in GitLab 16.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/390549) in objectives, key results, and tasks in GitLab 16.0. You can manually add an item to your To-Do List. @@ -97,10 +97,6 @@ You can manually add an item to your To-Do List. 1. In the upper-right corner, select **Add a to do** (**{todo-add}**). -  - -  - ## Create a to-do item by mentioning someone You can create a to-do item by mentioning someone anywhere except for a code block. Mentioning a user many times in one message only creates one to-do item. @@ -158,10 +154,6 @@ There are two ways to do this: - In the To-Do List, to the right of the to-do item, select **Mark as done** (**{check}**). - In the upper-right corner of the resource (for example, issue or merge request), select **Mark as done** (**{todo-done}**). -  - -  - ## Mark all to-do items as done You can mark all your to-do items as done at the same time. diff --git a/doc/user/upgrade_email_bypass.md b/doc/user/upgrade_email_bypass.md index afbdcbcdf09..ae47a9156c7 100644 --- a/doc/user/upgrade_email_bypass.md +++ b/doc/user/upgrade_email_bypass.md @@ -73,7 +73,7 @@ Your account has been blocked. Fatal: Could not read from remote repository Your primary email address is not confirmed. ``` -You can assure your users that they have not been [Blocked](admin_area/moderate_users.md#block-and-unblock-users) by an administrator. +You can assure your users that they have not been [Blocked](../administration/moderate_users.md#block-and-unblock-users) by an administrator. When affected users see this message, they must confirm their email address before they can commit code. ## What do you need to know as an administrator of a GitLab self-managed Instance? @@ -83,7 +83,7 @@ You have the following options to help your users: - They can confirm their address through the email that they received. - They can confirm the subject email address themselves by navigating to `https://gitlab.example.com/users/confirmation/new`. -As an administrator, you may also confirm a user in the [Admin Area](admin_area/index.md#administering-users). +As an administrator, you may also confirm a user in the [Admin Area](../administration/admin_area.md#administering-users). ## What do you do if you are an administrator and you're locked out? diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 5c6c64a3485..d119044930a 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -181,7 +181,7 @@ Storage types that add to the total namespace storage are: - Git repository - Git LFS -- Artifacts +- Job artifacts - Container registry - Package registry - Dependency proxy diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md index eb3ce1c5aff..a101e5f5f8c 100644 --- a/doc/user/workspace/index.md +++ b/doc/user/workspace/index.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/391543) in GitLab 16.0. FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `remote_development_feature_flag`. On GitLab.com, this feature is available. The feature is not ready for production use. +On self-managed GitLab, by default this feature is available. To hide the feature, an administrator can [disable the feature flag](../../administration/feature_flags.md) named `remote_development_feature_flag`. On GitLab.com, this feature is available. The feature is not ready for production use. WARNING: This feature is in [Beta](../../policy/experiment-beta-support.md#beta) and subject to change without notice. To leave feedback, see the [feedback issue](https://gitlab.com/gitlab-org/gitlab/-/issues/410031). @@ -19,6 +19,8 @@ A workspace is a virtual sandbox environment for your code in GitLab. You can us Each workspace includes its own set of dependencies, libraries, and tools, which you can customize to meet the specific needs of each project. Workspaces use the AMD64 architecture. +For a demo of this feature, see [GitLab Workspaces Demo](https://go.gitlab.com/qtu66q). + ## Set up a workspace ### Prerequisites @@ -60,6 +62,17 @@ To create a workspace: The workspace might take a few minutes to start. To access the workspace, under **Preview**, select the workspace link. You also have access to the terminal and can install any necessary dependencies. +## Deleting data associated with a workspace + +When you delete a project, agent, user, or token associated with a workspace: + +- The workspace is deleted from the user interface. +- In the Kubernetes cluster, the running workspace resources become orphaned. + +To clean up orphaned resources, an administrator must manually delete the workspace in Kubernetes. + +For more information about our plans to change the current behavior, see [issue 414384](https://gitlab.com/gitlab-org/gitlab/-/issues/414384). + ## Devfile A devfile is a file that defines a development environment by specifying the necessary tools, languages, runtimes, and other components for a GitLab project. @@ -159,3 +172,17 @@ You can provide your own container image, which can run as any Linux user ID. It GitLab uses the Linux root group ID permission to create, update, or delete files in a container. The container runtime used by the Kubernetes cluster must ensure all containers have a default Linux group ID of `0`. If you have a container image that does not support arbitrary user IDs, you cannot create, update, or delete files in a workspace. To create a container image that supports arbitrary user IDs, see the [OpenShift documentation](https://docs.openshift.com/container-platform/4.12/openshift_images/create-images.html#use-uid_create-images). + +## Troubleshooting + +When working with workspaces, you might encounter the following issues. + +### `Failed to renew lease` when creating a workspace + +You might not be able to create a workspace due to a known issue in the GitLab agent for Kubernetes. The following error message might appear in the agent's log: + +```plaintext +{"level":"info","time":"2023-01-01T00:00:00.000Z","msg":"failed to renew lease gitlab-agent-remote-dev-dev/agent-123XX-lock: timed out waiting for the condition\n","agent_id":XXXX} +``` + +This issue occurs when an agent instance cannot renew its leadership lease, which results in the shutdown of leader-only modules including the `remote_development` module. To resolve this issue, restart the agent instance. |
