Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-05-19 21:09:04 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2022-05-19 21:09:04 +0300
commitabe6e7a2e7a456fa05f4ebeb28f8747a92cf59b1 (patch)
treed028bb982b43b4079e4b47477b3fe51d4b913ba5 /doc
parentd0bb0e04f40b962576353ce56e270aa7bd25a5c0 (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r--doc/.vale/gitlab/spelling-exceptions.txt2
-rw-r--r--doc/administration/packages/index.md2
-rw-r--r--doc/architecture/blueprints/database_testing/index.md2
-rw-r--r--doc/development/backend/ruby_style_guide.md2
-rw-r--r--doc/development/code_review.md2
-rw-r--r--doc/development/contributing/merge_request_workflow.md2
-rw-r--r--doc/development/migration_style_guide.md2
-rw-r--r--doc/development/packages.md305
-rw-r--r--doc/development/packages/index.md25
-rw-r--r--doc/development/packages/new_format_development.md302
-rw-r--r--doc/development/packages/settings.md82
-rw-r--r--doc/development/packages/structure.md79
-rw-r--r--doc/development/secure_coding_guidelines.md2
-rw-r--r--doc/development/sidekiq/logging.md2
-rw-r--r--doc/install/docker.md20
-rw-r--r--doc/user/packages/package_registry/index.md6
-rw-r--r--doc/user/project/labels.md6
-rw-r--r--doc/user/project/merge_requests/code_quality.md10
18 files changed, 524 insertions, 329 deletions
diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt
index 92e10a91753..adc41bb72eb 100644
--- a/doc/.vale/gitlab/spelling-exceptions.txt
+++ b/doc/.vale/gitlab/spelling-exceptions.txt
@@ -571,7 +571,7 @@ rsyncing
rsyncs
Rubinius
Rubix
-Rubocop
+RuboCop
Rubular
ruleset
rulesets
diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md
index ef00127a70e..56786f334b0 100644
--- a/doc/administration/packages/index.md
+++ b/doc/administration/packages/index.md
@@ -26,7 +26,7 @@ The Packages feature allows GitLab to act as a repository and supports the follo
## Accepting contributions
-The below table lists formats that are not supported, but are accepting Community contributions for. Consider contributing to GitLab. This [development documentation](../../development/packages.md)
+The below table lists formats that are not supported, but are accepting Community contributions for. Consider contributing to GitLab. This [development documentation](../../development/packages/index.md)
guides you through the process.
<!-- vale gitlab.Spelling = NO -->
diff --git a/doc/architecture/blueprints/database_testing/index.md b/doc/architecture/blueprints/database_testing/index.md
index 22857abf176..8a3970f362d 100644
--- a/doc/architecture/blueprints/database_testing/index.md
+++ b/doc/architecture/blueprints/database_testing/index.md
@@ -20,7 +20,7 @@ Developers are expected to test database migrations prior to deploying to any en
The [code review phase](../../../development/database_review.md) involves Database Reviewers and Maintainers to manually check the migrations committed. This often involves knowing and spotting problematic patterns and their particular behavior on GitLab.com from experience. There is no large-scale environment available that allows us to test database migrations before they are being merged.
-Testing in CI is done on a very small database. We mainly check forward/backward migration consistency, evaluate Rubocop rules to detect well-known problematic behaviors (static code checking) and have a few other, rather technical checks in place (adding the right files etc). That is, we typically find code or other rather simple errors, but cannot surface any data related errors - which are also typically not covered by unit tests either.
+Testing in CI is done on a very small database. We mainly check forward/backward migration consistency, evaluate RuboCop rules to detect well-known problematic behaviors (static code checking) and have a few other, rather technical checks in place (adding the right files etc). That is, we typically find code or other rather simple errors, but cannot surface any data related errors - which are also typically not covered by unit tests either.
Once merged, migrations are being deployed to the staging environment. Its database size is less than 5% of the production database size as of January 2021 and its recent data distribution does not resemble the production site. Oftentimes, we see migrations succeed in staging but then fail in production due to query timeouts or other unexpected problems. Even if we caught problems in staging, this is still expensive to reconcile and ideally we want to catch those problems as early as possible in the development cycle.
diff --git a/doc/development/backend/ruby_style_guide.md b/doc/development/backend/ruby_style_guide.md
index 6c8125a6157..eff6ae7f217 100644
--- a/doc/development/backend/ruby_style_guide.md
+++ b/doc/development/backend/ruby_style_guide.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
This is a GitLab-specific style guide for Ruby code.
-Generally, if a style is not covered by [existing Rubocop rules or style guides](../contributing/style_guides.md#ruby-rails-rspec), it shouldn't be a blocker.
+Generally, if a style is not covered by [existing RuboCop rules or style guides](../contributing/style_guides.md#ruby-rails-rspec), it shouldn't be a blocker.
Before adding a new cop to enforce a given style, make sure to discuss it with your team.
When the style is approved by a backend EM or by a BE staff eng, add a new section to this page to
document the new rule. For every new guideline, add it in a new section and link the discussion from the section's
diff --git a/doc/development/code_review.md b/doc/development/code_review.md
index 252bd1daf55..8ab17586b0a 100644
--- a/doc/development/code_review.md
+++ b/doc/development/code_review.md
@@ -246,7 +246,7 @@ request diff alerting the reviewer to anything important as well as for anything
that demands further explanation or attention. Examples of content that may
warrant a comment could be:
-- The addition of a linting rule (Rubocop, JS etc).
+- The addition of a linting rule (RuboCop, JS etc).
- The addition of a library (Ruby gem, JS lib etc).
- Where not obvious, a link to the parent class or method.
- Any benchmarking performed to complement the change.
diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md
index ee1ed744cd4..399ce9141d8 100644
--- a/doc/development/contributing/merge_request_workflow.md
+++ b/doc/development/contributing/merge_request_workflow.md
@@ -322,7 +322,7 @@ issue) that are incremental improvements, such as:
1. Unprioritized bug fixes (for example, [Banner alerting of project move is
showing up everywhere](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/18985))
1. Documentation improvements
-1. Rubocop or Code Quality improvements
+1. RuboCop or Code Quality improvements
Tag a merge request with ~"Stuff that should Just Work" to track work in
this area.
diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md
index aebecd90574..8221dc2239f 100644
--- a/doc/development/migration_style_guide.md
+++ b/doc/development/migration_style_guide.md
@@ -495,7 +495,7 @@ def up
end
```
-The RuboCop rule generally allows standard Rails migration methods, listed below. This example causes a Rubocop offense:
+The RuboCop rule generally allows standard Rails migration methods, listed below. This example causes a RuboCop offense:
```ruby
disable_ddl_transaction!
diff --git a/doc/development/packages.md b/doc/development/packages.md
index 6526bdd45a1..a79f5f09677 100644
--- a/doc/development/packages.md
+++ b/doc/development/packages.md
@@ -1,302 +1,11 @@
---
-stage: Package
-group: Package
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+redirect_to: 'packages/index.md'
+remove_date: '2022-08-19'
---
-# Packages
+This document was moved to [another location](packages/index.md).
-This document guides you through adding support to GitLab for a new a [package management system](../administration/packages/index.md).
-
-See the already supported formats in the [Packages & Registries documentation](../user/packages/index.md)
-
-It is possible to add a new format with only backend changes.
-This guide is superficial and does not cover the way the code should be written.
-However, you can find a good example by looking at the following merge requests:
-
-- [npm registry support](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8673)
-- [Maven repository](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6607)
-- [Instance-level API for Maven repository](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8757)
-- [NuGet group-level API](https://gitlab.com/gitlab-org/gitlab/-/issues/36423)
-
-## General information
-
-The existing database model requires the following:
-
-- Every package belongs to a project.
-- Every package file belongs to a package.
-- A package can have one or more package files.
-- The package model is based on storing information about the package and its version.
-
-### API endpoints
-
-Package systems work with GitLab via API. For example `lib/api/npm_project_packages.rb`
-implements API endpoints to work with npm clients. So, the first thing to do is to
-add a new `lib/api/your_name_project_packages.rb` file with API endpoints that are
-necessary to make the package system client to work. Usually that means having
-endpoints like:
-
-- GET package information.
-- GET package file content.
-- PUT upload package.
-
-Since the packages belong to a project, it's expected to have project-level endpoint (remote)
-for uploading and downloading them. For example:
-
-```plaintext
-GET https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/
-PUT https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/
-```
-
-Group-level and instance-level endpoints should only be considered after the project-level endpoint is available in production.
-
-#### Remote hierarchy
-
-Packages are scoped within various levels of access, which is generally configured by setting your remote. A
-remote endpoint may be set at the project level, meaning when installing packages, only packages belonging to that
-project are visible. Alternatively, a group-level endpoint may be used to allow visibility to all packages
-within a given group. Lastly, an instance-level endpoint can be used to allow visibility to all packages within an
-entire GitLab instance.
-
-As an MVC, we recommend beginning with a project-level endpoint. A typical iteration plan for remote hierarchies is to go from:
-
-- Publish and install in a project
-- Install from a group
-- Publish and install in an Instance (this is for Self-Managed customers)
-
-Using instance-level endpoints requires [stricter naming conventions](#naming-conventions).
-
-NOTE:
-Composer package naming scope is Instance Level.
-
-### Naming conventions
-
-To avoid name conflict for instance-level endpoints you must define a package naming convention
-that gives a way to identify the project that the package belongs to. This generally involves using the project
-ID or full project path in the package name. See
-[Conan's naming convention](../user/packages/conan_repository/index.md#package-recipe-naming-convention-for-instance-remotes) as an example.
-
-For group and project-level endpoints, naming can be less constrained and it is up to the group and project
-members to be certain that there is no conflict between two package names. However, the system should prevent
-a user from reusing an existing name within a given scope.
-
-Otherwise, naming should follow the package manager's naming conventions and include a validation in the `package.md`
-model for that package type.
-
-### Services and finders
-
-Logic for performing tasks such as creating package or package file records or finding packages should not live
-within the API file, but should live in services and finders. Existing services and finders should be used or
-extended when possible to keep the common package logic grouped as much as possible.
-
-### Configuration
-
-GitLab has a `packages` section in its configuration file (`gitlab.rb`).
-It applies to all package systems supported by GitLab. Usually you don't need
-to add anything there.
-
-Packages can be configured to use object storage, therefore your code must support it.
-
-## MVC Approach
-
-The way new package systems are integrated in GitLab is using an [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc). Therefore, the first iteration should support the bare minimum user actions:
-
-- Authentication with a GitLab job, personal access, project access, or deploy token
-- Uploading a package and displaying basic metadata in the user interface
-- Pulling a package
-- Required actions
-
-Required actions are all the additional requests that GitLab needs to handle so the corresponding package manager CLI can work properly. It could be a search feature or an endpoint providing meta information about a package. For example:
-
-- For NuGet, the search request was implemented during the first MVC iteration, to support Visual Studio.
-- For npm, there is a metadata endpoint used by `npm` to get the tarball URL.
-
-For the first MVC iteration, it's recommended to stay at the project level of the [remote hierarchy](#remote-hierarchy). Other levels can be tackled with [future Merge Requests](#future-work).
-
-There are usually 2 phases for the MVC:
-
-- [Analysis](#analysis)
-- [Implementation](#implementation)
-
-### Keep iterations small
-
-When implementing a new package manager, it is tempting to create one large merge request containing all of the
-necessary endpoints and services necessary to support basic usage. Instead, put the
-API endpoints behind a [feature flag](feature_flags/index.md) and
-submit each endpoint or behavior (download, upload, etc) in a different merge request to shorten the review
-process.
-
-### Analysis
-
-During this phase, the idea is to collect as much information as possible about the API used by the package system. Here some aspects that can be useful to include:
-
-- **Authentication**: What authentication mechanisms are available (OAuth, Basic
- Authorization, other). Keep in mind that GitLab users often want to use their
- [Personal Access Tokens](../user/profile/personal_access_tokens.md).
- Although not needed for the MVC first iteration, the [CI/CD job tokens](../ci/jobs/ci_job_token.md)
- have to be supported at some point in the future.
-- **Requests**: Which requests are needed to have a working MVC. Ideally, produce
- a list of all the requests needed for the MVC (including required actions). Further
- investigation could provide an example for each request with the request and the response bodies.
-- **Upload**: Carefully analyze how the upload process works. This is likely the most
- complex request to implement. A detailed analysis is desired here as uploads can be
- encoded in different ways (body or multipart) and can even be in a totally different
- format (for example, a JSON structure where the package file is a Base64 value of
- a particular field). These different encodings lead to slightly different implementations
- on GitLab and GitLab Workhorse. For more detailed information, review [file uploads](#file-uploads).
-- **Endpoints**: Suggest a list of endpoint URLs to implement in GitLab.
-- **Split work**: Suggest a list of changes to do to incrementally build the MVC.
- This gives a good idea of how much work there is to be done. Here is an example
- list that would need to be adapted on a case by case basis:
- 1. Empty file structure (API file, base service for this package)
- 1. Authentication system for "logging in" to the package manager
- 1. Identify metadata and create applicable tables
- 1. Workhorse route for [object storage direct upload](uploads/index.md#direct-upload)
- 1. Endpoints required for upload/publish
- 1. Endpoints required for install/download
- 1. Endpoints required for required actions
-
-The analysis usually takes a full milestone to complete, though it's not impossible to start the implementation in the same milestone.
-
-In particular, the upload request can have some [requirements in the GitLab Workhorse project](#file-uploads). This project has a different release cycle than the rails backend. It's **strongly** recommended that you open an issue there as soon as the upload request analysis is done. This way GitLab Workhorse is already ready when the upload request is implemented on the rails backend.
-
-### Implementation
-
-The implementation of the different Merge Requests varies between different package system integrations. Contributors should take into account some important aspects of the implementation phase.
-
-#### Authentication
-
-The MVC must support [Personal Access Tokens](../user/profile/personal_access_tokens.md) right from the start. We currently support two options for these tokens: OAuth and Basic Access.
-
-OAuth authentication is already supported. You can see an example in the [npm API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/npm_project_packages.rb).
-
-[Basic Access authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication)
-support is done by overriding a specific function in the API helpers, like
-[this example in the Conan API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/conan_packages.rb).
-For this authentication mechanism, keep in mind that some clients can send an unauthenticated
-request first, wait for the 401 Unauthorized response with the [`WWW-Authenticate`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/WWW-Authenticate)
-field, then send an updated (authenticated) request. This case is more involved as
-GitLab needs to handle the 401 Unauthorized response. The [NuGet API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/nuget_packages.rb)
-supports this case.
-
-#### Authorization
-
-There are project and group level permissions for `read_package`, `create_package`, and `destroy_package`. Each
-endpoint should
-[authorize the requesting user](https://gitlab.com/gitlab-org/gitlab/-/blob/398fef1ca26ae2b2c3dc89750f6b20455a1e5507/ee/lib/api/conan_packages.rb)
-against the project or group before continuing.
-
-#### Database and handling metadata
-
-The current database model allows you to store a name and a version for each package.
-Every time you upload a new package, you can either create a new record of `Package`
-or add files to existing record. `PackageFile` should be able to store all file-related
-information like the file `name`, `side`, `sha1`, and so on.
-
-If there is specific data necessary to be stored for only one package system support,
-consider creating a separate metadata model. See `packages_maven_metadata` table
-and `Packages::Maven::Metadatum` model as an example for package specific data, and `packages_conan_file_metadata` table
-and `Packages::Conan::FileMetadatum` model as an example for package file specific data.
-
-If there is package specific behavior for a given package manager, add those methods to the metadata models and
-delegate from the package model.
-
-Note that the existing package UI only displays information within the `packages_packages` and `packages_package_files`
-tables. If the data stored in the metadata tables need to be displayed, a ~frontend change is required.
-
-#### File uploads
-
-File uploads should be handled by GitLab Workhorse using object accelerated uploads. What this means is that
-the workhorse proxy that checks all incoming requests to GitLab intercept the upload request,
-upload the file, and forward a request to the main GitLab codebase only containing the metadata
-and file location rather than the file itself. An overview of this process can be found in the
-[development documentation](uploads/index.md#direct-upload).
-
-In terms of code, this means a route must be added to the
-[GitLab Workhorse project](https://gitlab.com/gitlab-org/gitlab-workhorse) for each upload endpoint being added
-(instance, group, project). [This merge request](https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/412/diffs)
-demonstrates adding an instance-level endpoint for Conan to workhorse. You can also see the Maven project level endpoint
-implemented in the same file.
-
-Once the route has been added, you must add an additional `/authorize` version of the upload endpoint to your API file.
-[This example](https://gitlab.com/gitlab-org/gitlab/-/blob/398fef1ca26ae2b2c3dc89750f6b20455a1e5507/ee/lib/api/maven_packages.rb#L164)
-shows the additional endpoint added for Maven. The `/authorize` endpoint verifies and authorizes the request from workhorse,
-then the normal upload endpoint is implemented below, consuming the metadata that workhorse provides in order to
-create the package record. Workhorse provides a variety of file metadata such as type, size, and different checksum formats.
-
-For testing purposes, you may want to [enable object storage](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/object_storage.md)
-in your local development environment.
-
-#### File size limits
-
-Files uploaded to the GitLab Package Registry are [limited by format](../administration/instance_limits.md#package-registry-limits).
-On GitLab.com, these are typically set to 5GB to help prevent timeout issues and abuse.
-
-When a new package type is added to the `Packages::Package` model, a size limit must be added
-similar to [this example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52639/diffs#382f879fb09b0212e3cedd99e6c46e2083867216),
-or the [related test](https://gitlab.com/gitlab-org/gitlab/-/blob/fe4ba43766781371cebfacd78364a1de762917cd/spec/models/packages/package_spec.rb#L761)
-must be updated if file size limits do not apply. The only reason a size limit does not apply is if
-the package format does not upload and store package files.
-
-#### Rate Limits on GitLab.com
-
-Package manager clients can make rapid requests that exceed the
-[GitLab.com standard API rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
-This results in a `429 Too Many Requests` error.
-
-We have opened a set of paths to allow higher rate limits. Unless it is not possible,
-new package managers should follow these conventions so they can take advantage of the
-expanded package rate limit.
-
-These route prefixes guarantee a higher rate limit:
-
-```plaintext
-/api/v4/packages/
-/api/v4/projects/:project_id/packages/
-/api/v4/groups/:group_id/-/packages/
-```
-
-### MVC Checklist
-
-When adding support to GitLab for a new package manager, the first iteration must contain the
-following features. You can add the features through many merge requests as needed, but all the
-features must be implemented when the feature flag is removed.
-
-- Project-level API
-- Push event tracking
-- Pull event tracking
-- Authentication with Personal Access Tokens
-- Authentication with Job Tokens
-- Authentication with Deploy Tokens (group and project)
-- File size [limit](#file-size-limits)
-- File format guards (only accept valid file formats for the package type)
-- Name regex with validation
-- Version regex with validation
-- Workhorse route for [accelerated](uploads/working_with_uploads.md) uploads
-- Background workers for extracting package metadata (if applicable)
-- Documentation (how to use the feature)
-- API Documentation (individual endpoints with curl examples)
-- Seeding in [`db/fixtures/development/26_packages.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/26_packages.rb)
-- Update the [runbook](https://gitlab.com/gitlab-com/runbooks/-/blob/31fb4959e89db25fddf865bc81734c222daf32dd/dashboards/stage-groups/package.dashboard.jsonnet#L74) for the Grafana charts
-- End-to-end feature tests for (at the minimum) publishing and installing a package
-
-### Future Work
-
-While working on the MVC, contributors might find features that are not mandatory for the MVC but can provide a better user experience. It's generally a good idea to keep an eye on those and open issues.
-
-Here are some examples
-
-1. Endpoints required for search
-1. Front end updates to display additional package information and metadata
-1. Limits on file sizes
-1. Tracking for metrics
-1. Read more metadata fields from the package to make it available to the front end. For example, it's usual to be able to tag a package. Those tags can be read and saved by backend and then displayed on the packages UI.
-1. Endpoints for the upper levels of the [remote hierarchy](#remote-hierarchy). This step might need to create a [naming convention](#naming-conventions)
-
-## Exceptions
-
-This documentation is just guidelines on how to implement a package manager to match the existing structure and logic
-already present within GitLab. While the structure is intended to be extendable and flexible enough to allow for
-any given package manager, if there is good reason to stray due to the constraints or needs of a given package
-manager, then it should be raised and discussed within the implementation issue or merge request to work towards
-the most efficient outcome.
+<!-- This redirect file can be deleted after <2022-08-19>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/development/packages/index.md b/doc/development/packages/index.md
new file mode 100644
index 00000000000..55deaa229ba
--- /dev/null
+++ b/doc/development/packages/index.md
@@ -0,0 +1,25 @@
+---
+stage: Package
+group: Package
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Package Registry Development
+
+Development and Architectural documentation for the package registry
+
+- [Developing a new format](new_format_development.md)
+- [Settings](settings.md)
+- [Structure / Schema](structure.md)
+- API documentation
+ - [Composer](../../api/packages/composer.md)
+ - [Conan](../../api/packages/conan.md)
+ - [Debian](../../api/packages/debian.md)
+ - [Generic](../../user/packages/generic_packages/index.md)
+ - [Go Proxy](../../api/packages/go_proxy.md)
+ - [Helm](../../api/packages/helm.md)
+ - [Maven](../../api/packages/maven.md)
+ - [npm](../../api/packages/npm.md)
+ - [NuGet](../../api/packages/nuget.md)
+ - [PyPI](../../api/packages/pypi.md)
+ - [Ruby Gems](../../api/packages/rubygems.md)
diff --git a/doc/development/packages/new_format_development.md b/doc/development/packages/new_format_development.md
new file mode 100644
index 00000000000..f7d02f9160b
--- /dev/null
+++ b/doc/development/packages/new_format_development.md
@@ -0,0 +1,302 @@
+---
+stage: Package
+group: Package
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Developing support for a new package format
+
+This document guides you through adding support to GitLab for a new a [package management system](../../administration/packages/index.md).
+
+See the already supported formats in the [Packages & Registries documentation](../../user/packages/index.md)
+
+It is possible to add a new format with only backend changes.
+This guide is superficial and does not cover the way the code should be written.
+However, you can find a good example by looking at the following merge requests:
+
+- [npm registry support](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8673)
+- [Maven repository](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/6607)
+- [Instance-level API for Maven repository](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/8757)
+- [NuGet group-level API](https://gitlab.com/gitlab-org/gitlab/-/issues/36423)
+
+## General information
+
+The existing database model requires the following:
+
+- Every package belongs to a project.
+- Every package file belongs to a package.
+- A package can have one or more package files.
+- The package model is based on storing information about the package and its version.
+
+### API endpoints
+
+Package systems work with GitLab via API. For example `lib/api/npm_project_packages.rb`
+implements API endpoints to work with npm clients. So, the first thing to do is to
+add a new `lib/api/your_name_project_packages.rb` file with API endpoints that are
+necessary to make the package system client work. Usually that means having
+endpoints like:
+
+- GET package information.
+- GET package file content.
+- PUT upload package.
+
+Because the packages belong to a project, it's expected to have project-level endpoint (remote)
+for uploading and downloading them. For example:
+
+```plaintext
+GET https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/
+PUT https://gitlab.com/api/v4/projects/<your_project_id>/packages/npm/
+```
+
+Group-level and instance-level endpoints should only be considered after the project-level endpoint is available in production.
+
+#### Remote hierarchy
+
+Packages are scoped within various levels of access, which is generally configured by setting your remote. A
+remote endpoint may be set at the project level, meaning when installing packages, only packages belonging to that
+project are visible. Alternatively, a group-level endpoint may be used to allow visibility to all packages
+in a given group. Lastly, an instance-level endpoint can be used to allow visibility to all packages in an
+entire GitLab instance.
+
+As an MVC, we recommend beginning with a project-level endpoint. A typical iteration plan for remote hierarchies is to go from:
+
+- Publish and install in a project
+- Install from a group
+- Publish and install in an Instance (this is for Self-Managed customers)
+
+Using instance-level endpoints requires [stricter naming conventions](#naming-conventions).
+
+NOTE:
+Composer package naming scope is Instance Level.
+
+### Naming conventions
+
+To avoid name conflict for instance-level endpoints you must define a package naming convention
+that gives a way to identify the project that the package belongs to. This generally involves using the project
+ID or full project path in the package name. See
+[Conan's naming convention](../../user/packages/conan_repository/index.md#package-recipe-naming-convention-for-instance-remotes) as an example.
+
+For group and project-level endpoints, naming can be less constrained and it is up to the group and project
+members to be certain that there is no conflict between two package names. However, the system should prevent
+a user from reusing an existing name within a given scope.
+
+Otherwise, naming should follow the package manager's naming conventions and include a validation in the `package.md`
+model for that package type.
+
+### Services and finders
+
+Logic for performing tasks such as creating package or package file records or finding packages should not live
+in the API file, but should live in services and finders. Existing services and finders should be used or
+extended when possible to keep the common package logic grouped as much as possible.
+
+### Configuration
+
+GitLab has a `packages` section in its configuration file (`gitlab.rb`).
+It applies to all package systems supported by GitLab. Usually you don't need
+to add anything there.
+
+Packages can be configured to use object storage, therefore your code must support it.
+
+## MVC Approach
+
+The way new package systems are integrated in GitLab is using an [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc). Therefore, the first iteration should support the bare minimum user actions:
+
+- Authentication with a GitLab job, personal access, project access, or deploy token
+- Uploading a package and displaying basic metadata in the user interface
+- Pulling a package
+- Required actions
+
+Required actions are all the additional requests that GitLab must handle so the corresponding package manager CLI can work properly. It could be a search feature or an endpoint providing meta information about a package. For example:
+
+- For NuGet, the search request was implemented during the first MVC iteration, to support Visual Studio.
+- For npm, there is a metadata endpoint used by `npm` to get the tarball URL.
+
+For the first MVC iteration, it's recommended to stay at the project level of the [remote hierarchy](#remote-hierarchy). Other levels can be tackled with [future Merge Requests](#future-work).
+
+The MVC usually has two phases:
+
+- [Analysis](#analysis)
+- [Implementation](#implementation)
+
+### Keep iterations small
+
+When implementing a new package manager, it is tempting to create one large merge request containing all of the
+necessary endpoints and services necessary to support basic usage. Instead:
+
+1. Put the API endpoints behind a [feature flag](../feature_flags/index.md).
+1. Submit each endpoint or behavior (download, upload, etc) in a different merge request to shorten the review process.
+
+### Analysis
+
+During this phase, the idea is to collect as much information as possible about the API used by the package system. Here some aspects that can be useful to include:
+
+- **Authentication**: What authentication mechanisms are available (OAuth, Basic
+ Authorization, other). Keep in mind that GitLab users often want to use their
+ [Personal Access Tokens](../../user/profile/personal_access_tokens.md).
+ Although not needed for the MVC first iteration, the [CI/CD job tokens](../../ci/jobs/ci_job_token.md)
+ have to be supported at some point in the future.
+- **Requests**: Which requests are needed to have a working MVC. Ideally, produce
+ a list of all the requests needed for the MVC (including required actions). Further
+ investigation could provide an example for each request with the request and the response bodies.
+- **Upload**: Carefully analyze how the upload process works. This request is likely the most
+ complex to implement. A detailed analysis is desired here as uploads can be
+ encoded in different ways (body or multipart) and can even be in a totally different
+ format (for example, a JSON structure where the package file is a Base64 value of
+ a particular field). These different encodings lead to slightly different implementations
+ on GitLab and GitLab Workhorse. For more detailed information, review [file uploads](#file-uploads).
+- **Endpoints**: Suggest a list of endpoint URLs to implement in GitLab.
+- **Split work**: Suggest a list of changes to do to incrementally build the MVC.
+ This gives a good idea of how much work there is to be done. Here is an example
+ list that must be adapted on a case by case basis:
+ 1. Empty file structure (API file, base service for this package)
+ 1. Authentication system for "logging in" to the package manager
+ 1. Identify metadata and create applicable tables
+ 1. Workhorse route for [object storage direct upload](../uploads/index.md#direct-upload)
+ 1. Endpoints required for upload/publish
+ 1. Endpoints required for install/download
+ 1. Endpoints required for required actions
+
+The analysis usually takes a full milestone to complete, though it's not impossible to start the implementation in the same milestone.
+
+In particular, the upload request can have some [requirements in the GitLab Workhorse project](#file-uploads). This project has a different release cycle than the rails backend. It's **strongly** recommended that you open an issue there as soon as the upload request analysis is done. This way GitLab Workhorse is already ready when the upload request is implemented on the rails backend.
+
+### Implementation
+
+The implementation of the different Merge Requests varies between different package system integrations. Contributors should take into account some important aspects of the implementation phase.
+
+#### Authentication
+
+The MVC must support [Personal Access Tokens](../../user/profile/personal_access_tokens.md) right from the start. We support two options for these tokens: OAuth and Basic Access.
+
+OAuth authentication is already supported. You can see an example in the [npm API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/npm_project_packages.rb).
+
+[Basic Access authentication](https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication)
+support is done by overriding a specific function in the API helpers, like
+[this example in the Conan API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/conan_packages.rb).
+For this authentication mechanism, keep in mind that some clients can send an unauthenticated
+request first, wait for the 401 Unauthorized response with the [`WWW-Authenticate`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/WWW-Authenticate)
+field, then send an updated (authenticated) request. This case is more involved as
+GitLab must handle the `401 Unauthorized` response. The [NuGet API](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/nuget_packages.rb)
+supports this case.
+
+#### Authorization
+
+Project and group level permissions exist for `read_package`, `create_package`, and `destroy_package`. Each
+endpoint should
+[authorize the requesting user](https://gitlab.com/gitlab-org/gitlab/-/blob/398fef1ca26ae2b2c3dc89750f6b20455a1e5507/ee/lib/api/conan_packages.rb)
+against the project or group before continuing.
+
+#### Database and handling metadata
+
+The current database model allows you to store a name and a version for each package.
+Every time you upload a new package, you can either create a new record of `Package`
+or add files to existing record. `PackageFile` should be able to store all file-related
+information like the file `name`, `side`, `sha1`, and so on.
+
+If there is specific data necessary to be stored for only one package system support,
+consider creating a separate metadata model. See `packages_maven_metadata` table
+and `Packages::Maven::Metadatum` model as an example for package specific data, and `packages_conan_file_metadata` table
+and `Packages::Conan::FileMetadatum` model as an example for package file specific data.
+
+If there is package specific behavior for a given package manager, add those methods to the metadata models and
+delegate from the package model.
+
+The existing package UI only displays information in the `packages_packages` and `packages_package_files`
+tables. If the data stored in the metadata tables must be displayed, a `~frontend` change is required.
+
+#### File uploads
+
+File uploads should be handled by GitLab Workhorse using object accelerated uploads. What this means is that
+the workhorse proxy that checks all incoming requests to GitLab intercept the upload request,
+upload the file, and forward a request to the main GitLab codebase only containing the metadata
+and file location rather than the file itself. An overview of this process can be found in the
+[development documentation](../uploads/index.md#direct-upload).
+
+In terms of code, this means a route must be added to the
+[GitLab Workhorse project](https://gitlab.com/gitlab-org/gitlab-workhorse) for each upload endpoint being added
+(instance, group, project). [This merge request](https://gitlab.com/gitlab-org/gitlab-workhorse/-/merge_requests/412/diffs)
+demonstrates adding an instance-level endpoint for Conan to workhorse. You can also see the Maven project level endpoint
+implemented in the same file.
+
+After the route has been added, you must add an additional `/authorize` version of the upload endpoint to your API file.
+[This example](https://gitlab.com/gitlab-org/gitlab/-/blob/398fef1ca26ae2b2c3dc89750f6b20455a1e5507/ee/lib/api/maven_packages.rb#L164)
+shows the additional endpoint added for Maven. The `/authorize` endpoint verifies and authorizes the request from workhorse,
+then the normal upload endpoint is implemented below, consuming the metadata that Workhorse provides to
+create the package record. Workhorse provides a variety of file metadata such as type, size, and different checksum formats.
+
+For testing purposes, you may want to [enable object storage](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/object_storage.md)
+in your local development environment.
+
+#### File size limits
+
+Files uploaded to the GitLab Package Registry are [limited by format](../../administration/instance_limits.md#package-registry-limits).
+On GitLab.com, these are typically set to 5GB to help prevent timeout issues and abuse.
+
+When a new package type is added to the `Packages::Package` model, a size limit must be added
+similar to [this example](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52639/diffs#382f879fb09b0212e3cedd99e6c46e2083867216),
+or the [related test](https://gitlab.com/gitlab-org/gitlab/-/blob/fe4ba43766781371cebfacd78364a1de762917cd/spec/models/packages/package_spec.rb#L761)
+must be updated if file size limits do not apply. The only reason a size limit does not apply is if
+the package format does not upload and store package files.
+
+#### Rate Limits on GitLab.com
+
+Package manager clients can make rapid requests that exceed the
+[GitLab.com standard API rate limits](../../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
+This results in a `429 Too Many Requests` error.
+
+We have opened a set of paths to allow higher rate limits. Unless it is not possible,
+new package managers should follow these conventions so they can take advantage of the
+expanded package rate limit.
+
+These route prefixes guarantee a higher rate limit:
+
+```plaintext
+/api/v4/packages/
+/api/v4/projects/:project_id/packages/
+/api/v4/groups/:group_id/-/packages/
+```
+
+### MVC Checklist
+
+When adding support to GitLab for a new package manager, the first iteration must contain the
+following features. You can add the features through many merge requests as needed, but all the
+features must be implemented when the feature flag is removed.
+
+- Project-level API
+- Push event tracking
+- Pull event tracking
+- Authentication with Personal Access Tokens
+- Authentication with Job Tokens
+- Authentication with Deploy Tokens (group and project)
+- File size [limit](#file-size-limits)
+- File format guards (only accept valid file formats for the package type)
+- Name regex with validation
+- Version regex with validation
+- Workhorse route for [accelerated](../uploads/working_with_uploads.md) uploads
+- Background workers for extracting package metadata (if applicable)
+- Documentation (how to use the feature)
+- API Documentation (individual endpoints with curl examples)
+- Seeding in [`db/fixtures/development/26_packages.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/db/fixtures/development/26_packages.rb)
+- Update the [runbook](https://gitlab.com/gitlab-com/runbooks/-/blob/31fb4959e89db25fddf865bc81734c222daf32dd/dashboards/stage-groups/package.dashboard.jsonnet#L74) for the Grafana charts
+- End-to-end feature tests for (at the minimum) publishing and installing a package
+
+### Future Work
+
+While working on the MVC, contributors might find features that are not mandatory for the MVC but can provide a better user experience. It's generally a good idea to keep an eye on those and open issues.
+
+Here are some examples
+
+1. Endpoints required for search
+1. Front end updates to display additional package information and metadata
+1. Limits on file sizes
+1. Tracking for metrics
+1. Read more metadata fields from the package to make it available to the front end. For example, it's usual to be able to tag a package. Those tags can be read and saved by backend and then displayed on the packages UI.
+1. Endpoints for the upper levels of the [remote hierarchy](#remote-hierarchy). This step might require you to create a [naming convention](#naming-conventions)
+
+## Exceptions
+
+This documentation is just guidelines on how to implement a package manager to match the existing structure and logic
+already present in GitLab. While the structure is intended to be extendable and flexible enough to allow for
+any given package manager, if there is good reason to stray due to the constraints or needs of a given package
+manager, then it should be raised and discussed in the implementation issue or merge request to work towards
+the most efficient outcome.
diff --git a/doc/development/packages/settings.md b/doc/development/packages/settings.md
new file mode 100644
index 00000000000..37961c0504c
--- /dev/null
+++ b/doc/development/packages/settings.md
@@ -0,0 +1,82 @@
+---
+stage: Package
+group: Package
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Package Settings
+
+This page includes an exhaustive list of settings related to and maintained by the package stage.
+
+## Instance Settings
+
+### Package Registry
+
+Setting | Table | Description
+------- | ----- | -----------
+`npm_package_requests_forwarding` | `application_settings` | Enables or disables npm package forwarding at the instance level.
+`pypi_package_requests_forwarding` | `application_settings` | Enables or disables PyPI package forwarding at the instance level.
+`packages_cleanup_package_file_worker_capacity` | `application_settings` | Number of concurrent workers allowed for package file cleanup.
+`throttle_unauthenticated_packages_api_requests_per_period` | `application_settings` | Request limit for unauthenticated package API requests in the period defined by `throttle_unauthenticated_packages_api_period_in_seconds`.
+`throttle_unauthenticated_packages_api_period_in_seconds` | `application_settings` | Period in seconds to measure unauthenticated package API requests.
+`throttle_authenticated_packages_api_requests_per_period` | `application_settings` | Request limit for authenticated package API requests in the period defined by `throttle_authenticated_packages_api_period_in_seconds`.
+`throttle_authenticated_packages_api_period_in_seconds` | `application_settings` | Period in seconds to measure authenticated package API requests.
+`throttle_unauthenticated_packages_api_enabled` | `application_settings`
+`throttle_authenticated_packages_api_enabled` | `application_settings` | Enables or disables request limits/throttling for the package API.
+`conan_max_file_size` | `plan_limits` | Maximum file size for a Conan package file.
+`maven_max_file_size` | `plan_limits` | Maximum file size for a Maven package file.
+`npm_max_file_size` | `plan_limits` | Maximum file size for an npm package file.
+`nuget_max_file_size` | `plan_limits` | Maximum file size for a NuGet package file.
+`pypi_max_file_size` | `plan_limits` | Maximum file size for a PyPI package file.
+`generic_packages_max_file_size` | `plan_limits` | Maximum file size for a generic package file.
+`golang_max_file_size` | `plan_limits` | Maximum file size for a GoProxy package file.
+`debian_max_file_size` | `plan_limits` | Maximum file size for a Debian package file.
+`rubygems_max_file_size` | `plan_limits` | Maximum file size for a RubyGems package file.
+`terraform_module_max_file_size` | `plan_limits` | Maximum file size for a Terraform package file.
+`helm_max_file_size` | `plan_limits` | Maximum file size for a Helm package file.
+
+### Container Registry
+
+Setting | Table | Description
+------- | ----- | -----------
+`container_registry_token_expire_delay` | `application_settings` | The time in minutes before the container registry auth token (JWT) expires.
+`container_expiration_policies_enable_historic_entries` | `application_settings` | Allow or prevent projects older than 12.8 to use container cleanup policies.
+`container_registry_vendor` | `application_settings` | The vendor of the container registry. `gitlab` for the GitLab container registry, other values for external registries.
+`container_registry_version` | `application_settings` | The current version of the container registry.
+`container_registry_features` | `application_settings` | Features supported by the connected container registry. For example, tag deletion.
+`container_registry_delete_tags_service_timeout` | `application_settings` | The maximum time (in seconds) that the cleanup process can take to delete a batch of tags.
+`container_registry_expiration_policies_worker_capacity` | `application_settings` | Number of concurrent container image cleanup policy workers allowed.
+`container_registry_cleanup_tags_service_max_list_size` | `application_settings` | The maximum number of tags that can be deleted in a cleanup policy single execution. Additional tags must be deleted in another execution.
+`container_registry_expiration_policies_caching` | `application_settings` | Enable or disable tag creation timestamp caching during execution of cleanup policies.
+`container_registry_import_max_tags_count` | `application_settings` | Defines what is a the maximum amount of tags that we accept to migrate.
+`container_registry_import_max_retries` | `application_settings` | The maximum amount of retries done on a migration that is aborted.
+`container_registry_import_start_max_retries` | `application_settings` | The maximum amount of requests to start an import step that is sent to the Container Registry API.
+`container_registry_import_max_step_duration` | `application_settings` | The maximum amount of seconds before an ongoing migration is considered as stale.
+`container_registry_import_target_plan` | `application_settings` | The target subscription plan on which we're intend to pick container repositories.
+`container_registry_import_created_before` | `application_settings` | Only image repositories created before this timestamp are eligible for the migration.
+`container_registry_pre_import_timeout` | `application_settings` | The timeout for long running `pre_imports` before they are canceled by the `GuardWorker`.
+`container_registry_import_timeout` | `application_settings` | The timeout for long running imports before they are canceled by the `GuardWorker`.
+`dependency_proxy_ttl_group_policy_worker_capacity` | `application_settings` | Number of concurrent dependency proxy cleanup policy workers allowed.
+
+## Namespace/Group Settings
+
+Setting | Table | Description
+------- | ----- | -----------
+`maven_duplicates_allowed` | `namespace_package_settings` | Allow or prevent duplicate Maven packages.
+`maven_duplicate_exception_regex` | `namespace_package_settings` | Regex defining Maven packages that are allowed to be duplicate when duplicates are not allowed. This matches the name and version of the package.
+`generic_duplicates_allowed` | `namespace_package_settings` | Allow or prevent duplicate generic packages.
+`generic_duplicate_exception_regex` | `namespace_package_settings` | Regex defining generic packages that are allowed to be duplicate when duplicates are not allowed.
+Dependency Proxy Cleanup Policies - `ttl` | `dependency_proxy_image_ttl_group_policies` | Number of days to retain an unused Dependency Proxy file before it is removed.
+Dependency Proxy - `enabled` | `dependency_proxy_image_ttl_group_policies` | Enable or disable the Dependency Proxy cleanup policy.
+
+## Project Settings
+
+Setting | Table | Description
+------- | ----- | -----------
+Container Cleanup Policies - `next_run_at` | `container_expiration_policies` | When the project qualifies for the next container cleanup policy cron worker.
+Container Cleanup Policies - `name_regex` | `container_expiration_policies` | Regex defining image names to remove with the container cleanup policy.
+Container Cleanup Policies - `cadence` | `container_expiration_policies` | How often the container cleanup policy should run.
+Container Cleanup Policies - `older_than` | `container_expiration_policies` | Age of images to remove with the container cleanup policy.
+Container Cleanup Policies - `keep_n` | `container_expiration_policies` | Number of images to retain in a container cleanup policy.
+Container Cleanup Policies - `enabled` | `container_expiration_policies` | Enable or disable a container cleanup policy.
+Container Cleanup Policies - `name_regex_keep` | `container_expiration_policies` | Regex defining image names to always keep regardless of other rules with the container cleanup policy.
diff --git a/doc/development/packages/structure.md b/doc/development/packages/structure.md
new file mode 100644
index 00000000000..a2716232b11
--- /dev/null
+++ b/doc/development/packages/structure.md
@@ -0,0 +1,79 @@
+---
+stage: Package
+group: Package
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Package Structure
+
+## Package Registry
+
+```mermaid
+erDiagram
+ projects }|--|| namespaces : ""
+ packages_package_files }o--|| packages_packages : ""
+ packages_package_file_build_infos }o--|| packages_package_files : ""
+ packages_build_infos }o--|| packages_packages : ""
+ packages_tags }o--|| packages_packages : ""
+ packages_packages }|--|| projects : ""
+ packages_maven_metadata |o--|| packages_packages : ""
+ packages_nuget_metadata |o--|| packages_packages : ""
+ packages_composer_metadata |o--|| packages_packages : ""
+ packages_conan_metadata |o--|| packages_packages : ""
+ packages_pypi_metadata |o--|| packages_packages : ""
+ packages_npm_metadata |o--|| packages_packages : ""
+ package_conan_file_metadatum |o--|| packages_package_files : ""
+ package_helm_file_metadatum |o--|| packages_package_files : ""
+ packages_nuget_dependency_link_metadata |o--|| packages_dependency_links: ""
+ packages_dependencies ||--o| packages_dependency_links: ""
+ packages_packages ||--o{ packages_dependency_links: ""
+ namespace_package_settings |o--|| namespaces: ""
+```
+
+### Debian packages
+
+Debian contains a higher number of dedicated tables, so it is displayed here separately:
+
+```mermaid
+erDiagram
+ projects }|--|| namespaces : ""
+ packages_packages }|--|| projects : ""
+ packages_package_files }o--|| packages_packages : ""
+ package_debian_file_metadatum |o--|| packages_package_files : ""
+ packages_debian_group_architectures }|--|| packages_debian_group_distributions : ""
+ packages_debian_group_component_files }|--|| packages_debian_group_components : ""
+ packages_debian_group_component_files }|--|| packages_debian_group_architectures : ""
+ packages_debian_group_components }|--|| packages_debian_group_distributions : ""
+ packages_debian_group_distribution_keys }|--|| packages_debian_group_distributions : ""
+ packages_debian_group_distributions }o--|| namespaces : ""
+ packages_debian_project_architectures }|--|| packages_debian_project_distributions : ""
+ packages_debian_project_component_files }|--|| packages_debian_project_components : ""
+ packages_debian_project_component_files }|--|| packages_debian_project_architectures : ""
+ packages_debian_project_components }|--|| packages_debian_project_distributions : ""
+ packages_debian_project_distribution_keys }|--|| packages_debian_project_distributions : ""
+ packages_debian_project_distributions }o--|| projects : ""
+ packages_debian_publications }|--|| packages_debian_project_distributions : ""
+ packages_debian_publications |o--|| packages_packages : ""
+ packages_debian_project_distributions |o--|| packages_packages : ""
+ packages_debian_group_distributions |o--|| namespaces : ""
+ packages_debian_file_metadata |o--|| packages_package_files : ""
+```
+
+## Container Registry
+
+```mermaid
+erDiagram
+ projects }|--|| namespaces : ""
+ container_repositories }|--|| projects : ""
+ container_expiration_policy |o--|| projects : ""
+```
+
+## Dependency Proxy
+
+```mermaid
+erDiagram
+ dependency_proxy_blobs }o--|| namespaces : ""
+ dependency_proxy_manifests }o--|| namespaces : ""
+ dependency_proxy_image_ttl_group_policies |o--|| namespaces : ""
+ dependency_proxy_group_settings |o--|| namespaces : ""
+```
diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md
index 3e46891d20e..065e6eaf9c6 100644
--- a/doc/development/secure_coding_guidelines.md
+++ b/doc/development/secure_coding_guidelines.md
@@ -59,7 +59,7 @@ Some example of well implemented access controls and tests:
1. [example2](https://dev.gitlab.org/gitlab/gitlabhq/-/merge_requests/2511/diffs#ed3aaab1510f43b032ce345909a887e5b167e196_142_155)
1. [example3](https://dev.gitlab.org/gitlab/gitlabhq/-/merge_requests/3170/diffs?diff_id=17494)
-**NB:** any input from development team is welcome, for example, about Rubocop rules.
+**NB:** any input from development team is welcome, for example, about RuboCop rules.
## Regular Expressions guidelines
diff --git a/doc/development/sidekiq/logging.md b/doc/development/sidekiq/logging.md
index 015376b0fc6..474ea5de951 100644
--- a/doc/development/sidekiq/logging.md
+++ b/doc/development/sidekiq/logging.md
@@ -25,7 +25,7 @@ need to do anything.
There are however some instances when there would be no context
present when the job is scheduled, or the context that is present is
-likely to be incorrect. For these instances, we've added Rubocop rules
+likely to be incorrect. For these instances, we've added RuboCop rules
to draw attention and avoid incorrect metadata in our logs.
As with most our cops, there are perfectly valid reasons for disabling
diff --git a/doc/install/docker.md b/doc/install/docker.md
index d5ca2c2e29f..dcb44882ca8 100644
--- a/doc/install/docker.md
+++ b/doc/install/docker.md
@@ -449,14 +449,14 @@ web browser under `<hostIP>:8929` and push using SSH under the port `2289`.
A `docker-compose.yml` example that uses different ports can be found in the
[Docker compose](#install-gitlab-using-docker-compose) section.
-## Update
+## Upgrade
-In most cases, updating GitLab is as easy as downloading the newest Docker
+In most cases, upgrading GitLab is as easy as downloading the newest Docker
[image tag](#use-tagged-versions-of-gitlab).
-### Update GitLab using Docker Engine
+### Upgrade GitLab using Docker Engine
-To update GitLab that was [installed using Docker Engine](#install-gitlab-using-docker-engine):
+To upgrade GitLab that was [installed using Docker Engine](#install-gitlab-using-docker-engine):
1. Take a [backup](#back-up-gitlab).
1. Stop the running container:
@@ -493,17 +493,17 @@ To update GitLab that was [installed using Docker Engine](#install-gitlab-using-
gitlab/gitlab-ee:latest
```
-On the first run, GitLab will reconfigure and update itself.
+On the first run, GitLab will reconfigure and upgrade itself.
-Refer to the GitLab [Update recommendations](../policy/maintenance.md#upgrade-recommendations)
+Refer to the GitLab [Upgrade recommendations](../policy/maintenance.md#upgrade-recommendations)
when upgrading between major versions.
-### Update GitLab using Docker compose
+### Upgrade GitLab using Docker compose
-To update GitLab that was [installed using Docker Compose](#install-gitlab-using-docker-compose):
+To upgrade GitLab that was [installed using Docker Compose](#install-gitlab-using-docker-compose):
1. Take a [backup](#back-up-gitlab).
-1. Download the newest release and update your GitLab instance:
+1. Download the newest release and upgrade your GitLab instance:
```shell
docker-compose pull
@@ -517,7 +517,7 @@ To update GitLab that was [installed using Docker Compose](#install-gitlab-using
You can convert an existing Docker-based GitLab Community Edition (CE) container
to a GitLab [Enterprise Edition](https://about.gitlab.com/pricing/) (EE) container
-using the same approach as [updating the version](#update).
+using the same approach as [upgrading the version](#upgrade).
We recommend you convert from the same version of CE to EE (for example, CE 14.1 to EE 14.1).
This is not explicitly necessary, and any standard upgrade (for example, CE 14.0 to EE 14.1) should work.
diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md
index 1f278bd1476..86f6ee225c9 100644
--- a/doc/user/packages/package_registry/index.md
+++ b/doc/user/packages/package_registry/index.md
@@ -65,7 +65,7 @@ For most package types, the following credential types are valid:
[GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444),
which prevents you from using a job token with internal projects. This bug only impacts self-managed
GitLab instances.
-
+
## Use GitLab CI/CD to build packages
You can use [GitLab CI/CD](../../../ci/index.md) to build packages.
@@ -125,7 +125,7 @@ to **Only Project Members**, the Package Registry is then public. However, disab
Registry disables all Package Registry operations.
[GitLab-#329253](https://gitlab.com/gitlab-org/gitlab/-/issues/329253)
-proposes adding the ability to control Package Registry visibility from the UI.
+proposes adding the ability to control Package Registry visibility from the UI.
| | | Anonymous<br/>(everyone on internet) | Guest | Reporter, Developer, Maintainer, Owner |
| -------------------- | --------------------- | --------- | ----- | ------------------------------------------ |
@@ -167,7 +167,7 @@ You can also use the [API](../../../api/packages.md) to administer the Package R
## Accepting contributions
This table lists unsupported package manager formats that we are accepting contributions for.
-Consider contributing to GitLab. This [development documentation](../../../development/packages.md)
+Consider contributing to GitLab. This [development documentation](../../../development/packages/index.md)
guides you through the process.
<!-- vale gitlab.Spelling = NO -->
diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md
index 2cc23b14857..319e47acf32 100644
--- a/doc/user/project/labels.md
+++ b/doc/user/project/labels.md
@@ -125,8 +125,7 @@ To do so:
1. Select **Create project label**.
1. Fill in the name field. You can't specify a description if creating a label this way.
You can add a description later by [editing the label](#edit-a-label).
-1. Optional. Select a color by selecting from the available colors, or enter a hex color value for
- a specific color.
+1. Select a color by selecting from the available colors, or enter a hex color value for a specific color.
1. Select **Create**.
### Create a group label
@@ -160,8 +159,7 @@ To do so:
1. Select **Create group label**.
1. Fill in the name field. You can't specify a description if creating a label this way.
You can add a description later by [editing the label](#edit-a-label).
-1. Optional. Select a color by selecting from the available colors,enter input a hex color value
- for a specific color.
+1. Select a color by selecting from the available colors,enter input a hex color value for a specific color.
1. Select **Create**.
## Edit a label
diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md
index 7e8ef9272d4..13623ebf719 100644
--- a/doc/user/project/merge_requests/code_quality.md
+++ b/doc/user/project/merge_requests/code_quality.md
@@ -557,9 +557,9 @@ GitLab only uses the Code Quality artifact from the latest created job (with the
If multiple jobs in a pipeline generate a code quality artifact, those of earlier jobs are ignored.
To avoid confusion, configure only one job to generate a `gl-code-quality-report.json`.
-### Rubocop errors
+### RuboCop errors
-When using Code Quality jobs on a Ruby project, you can encounter problems running Rubocop.
+When using Code Quality jobs on a Ruby project, you can encounter problems running RuboCop.
For example, the following error can appear when using either a very recent or very old version
of Ruby:
@@ -569,15 +569,15 @@ Unknown Ruby version 2.7 found in `.ruby-version`. (RuboCop::ValidationError)
Supported versions: 2.1, 2.2, 2.3, 2.4, 2.5
```
-This is caused by the default version of Rubocop used by the check engine not covering
+This is caused by the default version of RuboCop used by the check engine not covering
support for the Ruby version in use.
-To use a custom version of Rubocop that
+To use a custom version of RuboCop that
[supports the version of Ruby used by the project](https://docs.rubocop.org/rubocop/compatibility.html#support-matrix),
you can [override the configuration through a `.codeclimate.yml` file](https://docs.codeclimate.com/docs/rubocop#using-rubocops-newer-versions)
created in the project repository.
-For example, to specify using Rubocop release **0.67**:
+For example, to specify using RuboCop release **0.67**:
```yaml
version: "2"