Add latest changes from gitlab-org/gitlab@master

2 files changed, 195 insertions, 208 deletions

diff --git a/doc/ci/components/catalog.md b/doc/ci/components/catalog.md
index e4d4d8c82ab..8a7c333b5db 100644
--- a/doc/ci/components/catalog.md
+++ b/doc/ci/components/catalog.md
@@ -1,52 +1,11 @@
 ---
-stage: Verify
-group: Pipeline Authoring
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+redirect_to: '/ee/ci/components/#cicd-catalog'
+remove_date: '2024-02-24'
 ---
 
-# CI/CD catalog **(PREMIUM ALL EXPERIMENT)**
+This document was moved to [CI/CD components](index.md#cicd-catalog).
 
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407249) in GitLab 16.1.
-
-The CI/CD catalog is a list of [components repositories](index.md#components-repository),
-each containing resources that you can add to your CI/CD pipelines.
-
-Each top level namespace has its own catalog, which contains all the releases from
-components repositories hosted under it. You can create components repositories anywhere
-under the desired top level namespace and the released components are available to
-all projects in that namespace.
-
-## Add a components repository to the Catalog
-
-After components are added to a components repository, they can immediately be [used](index.md#use-a-component-in-a-cicd-configuration)
-to build pipelines in other projects.
-
-However, the repository is not discoverable. You must set the project as a catalog resource
-for it to be visible in the CI/CD Catalog, then other users can discover it. You should only set a repository as a catalog resource when the components are ready for usage.
-
-To set a project as a catalog resource:
-
-1. On the left sidebar, select **Search or go to** and find your project.
-1. On the left sidebar, select **Settings > General**.
-1. Expand **Visibility, project features, permissions**.
-1. Scroll down to **CI/CD Catalog resource** and select the toggle to mark the project as a catalog resource.
-
-Ensure the project has a clear [description](../../user/project/working_with_projects.md#edit-project-name-and-description),
-as the project description is displayed in the component list in the catalog.
-
-NOTE:
-This action is not reversible, and the
-component is always visible in the Catalog unless the repository is deleted. If a component has a bug or other issue, you can [create a new release](index.md#release-a-component) with an updated version.
-
-After the repository is set as a components repository, it appears in the CI/CD Catalog of the namespace.
-
-## View available components in the CI/CD Catalog
-
-To access the CI/CD Catalog and view the published components that are available to you:
-
-1. On the left sidebar, select **Search or go to**.
-1. Select **Explore**.
-1. Select **CI/CD Catalog**.
-
-Alternatively, if you are already in the [pipeline editor](../pipeline_editor/index.md)
-in your project, you can select **Browse CI/CD Catalog**.
+<!-- This redirect file can be deleted after <YYYY-MM-DD>. -->
+<!-- Redirects that point to other docs in the same project expire in three months. -->
+<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. -->
+<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md
index f7c23964606..e0c0b7977f9 100644
--- a/doc/ci/components/index.md
+++ b/doc/ci/components/index.md
@@ -13,25 +13,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 A CI/CD component is a reusable single pipeline configuration unit. Use them to compose an entire pipeline configuration or a small part of a larger pipeline.
 
-A component can optionally take [input parameters](../yaml/inputs.md).
+A component can take [input parameters](../yaml/inputs.md).
 
 CI/CD components are similar to the other kinds of [configuration added with the `include` keyword](../yaml/includes.md), but have several advantages:
 
 - Components can be released and used with a specific version.
-- Multiple components can be combined in the same project and released with a single tag.
-- Components are discoverable in the [CI/CD Catalog](catalog.md).
+- Multiple components can be defined in the same project and versioned together.
+- Components are discoverable in the [CI/CD Catalog](#cicd-catalog).
 
-## Components repository
+## Component project
 
-A components repository is a GitLab project with a repository that hosts one or more pipeline components. All components in the project are versioned and released together.
+A component project is a GitLab project with a repository that hosts one or more components.
+All components in the project are versioned together.
 
-If a component requires different versioning from other components, the component should be migrated to its own components repository.
+If a component requires different versioning from other components, the component should be moved
+to a dedicated component project.
 
 One component repository can have a maximum of 10 components.
 
-## Create a components repository
+### Create a component project
 
-To create a components repository, you must:
+To create a component project, you must:
 
 1. [Create a new project](../../user/project/index.md#create-a-blank-project) with a `README.md` file.
 1. Add a YAML configuration file for each component, following the [required directory structure](#directory-structure).
@@ -51,17 +53,19 @@ To create a components repository, you must:
 
 ### Directory structure
 
-A components repository can host one or more components, and must follow a mandatory file structure.
+The repository must contain:
 
-Component configurations can be saved through the following directory structure, containing:
+- A `README.md` Markdown file documenting the details of all the components in the repository.
+- A top level `templates/` directory that contains all the component configurations.
+  You can define components in this directory:
+  - In single files ending in `.yml` for each component, like `templates/secret-detection.yml`.
+  - In sub-directories containing `template.yml` files as entry points, for components
+    that bundle together multiple related files. For example, `templates/secret-detection/template.yml`.
 
-- A `templates` directory at the top level of your components repository. All component configuration files
-  should be saved under this directory.
-- Files ending in `.yml` containing the component configurations, one file per component.
-- A Markdown `README.md` file explaining the details of all the components in the repository.
+Configure the project's `.gitlab-ci.yml` to [test the components](#test-the-component)
+and [release new versions](#publish-a-new-release).
 
-For example, if the project contains a single component and a pipeline to test the component,
-the file structure should be similar to:
+For example, if the project contains a single component, the directory structure should be similar to:
 
 ```plaintext
 ├── templates/
@@ -70,51 +74,28 @@ the file structure should be similar to:
 └── .gitlab-ci.yml
 ```
 
-This example component could be referenced with a path similar to `gitlab.com/my-namespace/my-project/secret-detection@<version>`,
-if the project is:
-
-- On GitLab.com
-- Named `my-project`
-- In a personal namespace or group named `my-namespace`
-
-The templates directory and the suffix of the configuration file should be excluded from the referenced path.
-
-If the project contains multiple components, then the file structure should be similar to:
+If the project contains multiple components, then the directory structure should be similar to:
 
 ```plaintext
+├── templates/
+│   ├── all-scans.yml
+│   └── secret-detection/
+│       ├── template.yml
+│       ├── Dockerfile
+│       └── test.sh
 ├── README.md
-├── .gitlab-ci.yml
-└── templates/
-    ├── all-scans.yml
-    └── secret-detection.yml
+└── .gitlab-ci.yml
 ```
 
-These components would be referenced with these paths:
+In this example:
 
-- `gitlab.com/my-namespace/my-project/all-scans@<version>`
-- `gitlab.com/my-namespace/my-project/secret-detection@<version>`
-
-You can also have components defined as a directory if you want to bundle together multiple related files.
-In this case GitLab expects a `template.yml` file to be present:
-
-For example:
-
-```plaintext
-├── README.md
-├── .gitlab-ci.yml
-└── templates/
-    └── dast
-        ├── docs.md
-        ├── Dockerfile
-        └── template.yml
-```
-
-In this example, the component could be referenced with `gitlab.com/my-namespace/my-project/dast@<version>`.
+- The `all-scans` component configuration is defined in a single file.
+- The `secret-detection` component configuration contains multiple files in a directory.
 
 #### Component configurations saved in any directory (deprecated)
 
 WARNING:
-Saving component configurations through this directory structure is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/415855) and should be avoided.
+Saving components through the following directory structure is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/415855) and should be avoided.
 
 Components configurations can be saved through the following directory structure, containing:
 
@@ -135,9 +116,6 @@ namespace named `my-namespace`:
   └── .gitlab-ci.yml
   ```
 
-  The `.gitlab-ci.yml` file is not required for a CI/CD component to work, but
-  [testing the component](#test-the-component) in a pipeline in the project is recommended.
-
   This component is referenced with the path `gitlab.com/my-namespace/my-project@<version>`.
 
 - Containing one default component and multiple sub-components, then the file structure
@@ -173,10 +151,93 @@ Nesting of components is not possible. For example:
 │       └── nested_template.yml
 ```
 
-## Release a component
+## Use a component
 
-To create a release for a CI/CD component, use the [`release`](../yaml/index.md#release)
-keyword in a CI/CD pipeline.
+You can use a component in a CI/CD configuration with the `include: component` keyword.
+The component is identified by a unique address formatted as `<fully-qualified-domain-name>/<project-path>/<component-name>@<specific-version>`.
+
+For example:
+
+```yaml
+include:
+  - component: gitlab.example.com/my-org/security-components/secret-detection@1.0
+    inputs:
+      stage: build
+```
+
+In this example:
+
+- `gitlab.example.com` is the Fully Qualified Domain Name (FQDN) matching the GitLab host.
+  You can only reference components in the same GitLab instance as your project.
+- `my-org/security-components` is the full path of the project containing the component.
+- `secret-detection` is the component name that is defined as either a single file `templates/secret-detection.yml`
+  or as a directory `templates/secret-detection/` containing a `template.yml`.
+- `1.0` is the version of the component. In order of highest priority first,
+  the version can be:
+  - A branch name, for example `main`.
+  - A commit SHA, for example `e3262fdd0914fa823210cdb79a8c421e2cef79d8`.
+  - A tag, for example: `1.0`. If a tag and branch exist with the same name, the tag
+    takes precedence over the branch. If a tag and commit SHA exist with the same name,
+    the commit SHA takes precedence over the tag.
+  - `~latest`, which is a special version that always points to the most recent
+    [release published in the CI/CD Catalog](#publish-a-new-release).
+
+NOTE:
+The `~latest` version keyword always returns the most recent published release, not the release with
+the latest semantic version. For example, if you first release `2.0.0`, and later release
+a patch fix like `1.5.1`, then `~latest` returns the `1.5.1` release.
+[Issue #427286](https://gitlab.com/gitlab-org/gitlab/-/issues/427286) proposes to
+change this behavior.
+
+## CI/CD Catalog **(FREE ALL EXPERIMENT)**
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407249) in GitLab 16.1.
+
+The CI/CD Catalog is a list of projects with published CI/CD components you can use to extend your CI/CD workflow.
+
+Anyone can add a [component project](index.md#component-project) to the CI/CD Catalog, or contribute
+to an existing project to improve the available components.
+
+### View the CI/CD Catalog
+
+To access the CI/CD Catalog and view the published components that are available to you:
+
+1. On the left sidebar, select **Search or go to**.
+1. Select **Explore**.
+1. Select **CI/CD Catalog**.
+
+Alternatively, if you are already in the [pipeline editor](../pipeline_editor/index.md)
+in your project, you can select **Browse CI/CD Catalog**.
+
+NOTE:
+Only public and internal projects are discoverable in the CI/CD Catalog.
+
+### Publish a component project
+
+To publish a component project in the CI/CD catalog, you must:
+
+1. Set the project as a catalog resource.
+1. Publish a new release.
+
+#### Set a component project as a catalog resource
+
+To make published versions of a component project visible in the CI/CD catalog,
+you must set the project as a catalog resource:
+
+1. On the left sidebar, select **Search or go to** and find your project.
+1. On the left sidebar, select **Settings > General**.
+1. Expand **Visibility, project features, permissions**.
+1. Scroll down to **CI/CD Catalog resource** and select the toggle to mark the project as a catalog resource.
+
+#### Publish a new release
+
+Components defined in a [component project](#component-project) can be [used](#use-a-component)
+immediately and don't require to be published in the CI/CD catalog. However, having the component
+project published in the catalog makes it discoverable to other users.
+
+After the project is set as a [catalog resource](#set-a-component-project-as-a-catalog-resource),
+add a job to the project's `.gitlab-ci.yml` file that creates a release using the
+[`release`](../yaml/index.md#release) keyword.
 
 For example:
 
@@ -184,75 +245,87 @@ For example:
 create-release:
   stage: deploy
   image: registry.gitlab.com/gitlab-org/release-cli:latest
-  rules:
-    - if: $CI_COMMIT_TAG =~ /^v\d+/
   script: echo "Creating release $CI_COMMIT_TAG"
   release:
     tag_name: $CI_COMMIT_TAG
-    description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH"
+    description: "Release $CI_COMMIT_TAG of components in $CI_PROJECT_PATH"
 ```
 
-In this example, the job runs only for tags formatted as `v` + version number.
-If all previous jobs succeed, the release is created.
+The release fails if the project is missing:
+
+- The [project description](../../user/project/working_with_projects.md#edit-project-name-and-description),
+  to display in the catalog list.
+- A `README.md` file in the root directory for the commit SHA of the tag being released.
+- Any component in the `templates/` directory for the commit SHA of the tag being released.
 
-Like in the [component testing example](#test-the-component), you can set a component to automatically
-be released after all tests pass in pipelines for new tags.
+Create a [new tag](../../user/project/repository/tags/index.md#create-a-tag) for the release,
+which should trigger a tag pipeline that contains the job responsible that creates the release.
+You should configure the tag pipeline to [test the components](#test-the-component) before
+running the release job.
 
-All released versions of the components repositories are displayed in the [CI/CD Catalog](catalog.md),
-providing users with information about official releases.
+The release is created and the new version is published to the CI/CD catalog only if:
 
-Components [can be used](#use-a-component-in-a-cicd-configuration) without being released,
-by using the commit SHA or ref. However, the `~latest` version keyword can only be used with released tags.
+- All jobs before the release job succeed.
+- All component project [requirements](#directory-structure) are satisfied.
+- The component project is [set as a catalog resource](#set-a-component-project-as-a-catalog-resource).
 
 NOTE:
-The `~latest` keyword always returns the most recent release, not the release with
-the latest semantic version. For example, if you first release `v2.0.0`, and later release
-a patch fix like `v1.5.1`, then `~latest` returns the `v1.5.1` release.
-[Issue #427286](https://gitlab.com/gitlab-org/gitlab/-/issues/427286) proposes to
-change this behavior.
+If you disable [catalog resource setting](#set-a-component-project-as-a-catalog-resource),
+the component project and all versions are removed from the catalog. To publish it again,
+you must re-enable the setting and release a new version.
+
+## Best practices
+
+This section describes some best practices for creating high quality component projects.
+
+### Test the component
+
+Testing CI/CD components as part of the development workflow is strongly recommended
+and helps ensure consistent behavior.
 
-## Use a component in a CI/CD configuration
+Test changes in a CI/CD pipeline (like any other project) by creating a `.gitlab-ci.yml`
+in the root directory. Make sure to test both the behavior and potential side-effects
+of the component. You can use the [GitLab API](../../api/rest/index.md) if needed.
 
-You can add a component to a CI/CD configuration with the `include: component` keyword.
 For example:
 
 ```yaml
 include:
-  - component: gitlab.example.com/my-namespace/my-project@1.0
+  # include the component located in the current project from the current SHA
+  - component: gitlab.com/$CI_PROJECT_PATH/my-component@$CI_COMMIT_SHA
     inputs:
       stage: build
-```
-
-The component is identified by a unique address in the form `<fully-qualified-domain-name>/<component-path>@<specific-version>`,
-where:
 
-- `<fully-qualified-domain-name>` matches the GitLab host. You can only reference components
-  in the same GitLab instance as your project.
-- `<component-path>` is the component project's full path and directory where the
-  component YAML file is located.
-- `<specific-version>` is the version of the component. In order of highest priority first,
-  the version can be:
-  - A branch name, for example `main`.
-  - A commit SHA, for example `e3262fdd0914fa823210cdb79a8c421e2cef79d8`.
-  - A tag, for example: `1.0`. If a tag and branch exist with the same name, the tag
-    takes precedence over the branch. If a tag and commit SHA exist with the same name,
-    the commit SHA takes precedence over the tag.
-  - `~latest`, which is a special version that always points to the most recent released tag.
-    Available only if the component has been [released](#release-a-component).
+stages: [build, test, release]
 
-For example, for a component repository located at `gitlab-org/dast` on `gitlab.com`,
-the path:
+# Expect `component-job` is added.
+# This example tests that the included component works as expected.
+# You can inspect data generated by the component, use GitLab API endpoints or third-party tools.
+ensure-job-added:
+  stage: test
+  image: badouralix/curl-jq
+  script:
+    - |
+      route="https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs"
+      count=`curl --silent --header "PRIVATE-TOKEN: $API_TOKEN" $route | jq 'map(select(.name | contains("component-job"))) | length'`
+      if [ "$count" != "1" ]; then
+        exit 1
+      fi
 
-- `gitlab.com/gitlab-org/dast@main` targets the `template.yml` in the root directory
-  on the `main` branch.
-- `gitlab.com/gitlab-org/dast@e3262fdd0914fa823210cdb79a8c421e2cef79d8` targets the same file
-  for the specified commit SHA.
-- `gitlab.com/gitlab-org/dast@1.0` targets the same file for the `1.0` tag.
-- `gitlab.com/gitlab-org/dast@~latest` targets the same file for the latest release.
-- `gitlab.com/gitlab-org/dast/api-scan@main` targets a different file, the `template.yml`
-  in the `/api-scan` directory in the component repository, for the `main` branch.
+# If we are tagging a release with a specific convention ("v" + number) and all
+# previous checks succeeded, we proceed with creating a release automatically.
+create-release:
+  stage: release
+  image: registry.gitlab.com/gitlab-org/release-cli:latest
+  rules:
+    - if: $CI_COMMIT_TAG =~ /^v\d+/
+  script: echo "Creating release $CI_COMMIT_TAG"
+  release:
+    tag_name: $CI_COMMIT_TAG
+    description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH"
+```
 
-## Best practices
+After committing and pushing changes, the pipeline tests the component, then releases the tag it if the test passes.
 
 ### Avoid using global keywords
 
@@ -264,7 +337,9 @@ As an alternative to global keywords, instead:
 
 - Add the configuration directly to each job, even if it creates some duplication
   in the component configuration.
-- Use the [`extends`](../yaml/index.md#extends) keyword in the component.
+- Use the [`extends`](../yaml/index.md#extends) keyword in the component, but use
+  unique names that reduce the risk of naming conflicts when the component is merged
+  into the configuration.
 
 For example, using the `default` keyword is not recommended:
 
@@ -396,7 +471,8 @@ In other cases, CI/CD variables are still preferred, including:
 
 ### Use semantic versioning
 
-When tagging and releasing new versions of components, you should use [semantic versioning](https://semver.org).
+When tagging and [releasing new versions](#publish-a-new-release) of components, you should use
+[semantic versioning](https://semver.org).
 Semantic versioning is the standard for communicating that a change is a major, minor, patch,
 or other kind of change.
 
@@ -410,80 +486,32 @@ Other examples of semantic versioning:
 - `1.0.0-alpha`
 - `3.0.0-rc1`
 
-### Test the component
-
-Testing CI/CD components as part of the development workflow is strongly recommended
-and helps ensure consistent behavior.
-
-Test changes in a CI/CD pipeline (like any other project) by creating a `.gitlab-ci.yml`
-in the root directory.
-
-For example:
-
-```yaml
-include:
-  # include the component located in the current project from the current SHA
-  - component: gitlab.com/$CI_PROJECT_PATH/my-project@$CI_COMMIT_SHA
-    inputs:
-      stage: build
-
-stages: [build, test, release]
-
-# Expect `component-job` is added.
-# This example tests that the included component works as expected.
-# You can inspect data generated by the component, use GitLab API endpoints or third-party tools.
-ensure-job-added:
-  stage: test
-  image: badouralix/curl-jq
-  script:
-    - |
-      route="https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs"
-      count=`curl --silent --header "PRIVATE-TOKEN: $API_TOKEN" $route | jq 'map(select(.name | contains("component-job"))) | length'`
-      if [ "$count" != "1" ]; then
-        exit 1
-      fi
-
-# If we are tagging a release with a specific convention ("v" + number) and all
-# previous checks succeeded, we proceed with creating a release automatically.
-create-release:
-  stage: release
-  image: registry.gitlab.com/gitlab-org/release-cli:latest
-  rules:
-    - if: $CI_COMMIT_TAG =~ /^v\d+/
-  script: echo "Creating release $CI_COMMIT_TAG"
-  release:
-    tag_name: $CI_COMMIT_TAG
-    description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH"
-```
-
-After committing and pushing changes, the pipeline tests the component, then releases it if the test passes.
-
 ## Convert a CI/CD template to a component
 
 Any existing CI/CD template that you use in projects by using the `include:` syntax
 can be converted to a CI/CD component:
 
-1. Decide if you want the component to be part of an existing [components repository](index.md#components-repository)
-   to be grouped with other components, or create and set up a new components repository.
-1. Create a YAML file in the components repository according to the expected [directory structure](index.md#directory-structure).
+1. Decide if you want the component to be part of an existing [component project](index.md#component-project)
+   to be grouped with other components, or [create a new component project](#create-a-component-project).
+1. Create a YAML file in the component project according to the expected [directory structure](index.md#directory-structure).
 1. Copy the content of the original template YAML file into the new component YAML file.
 1. Refactor the new component's configuration to:
    - Follow the [best practices](index.md#best-practices) for components.
    - Improve the configuration, for example by enabling [merge request pipelines](../pipelines/merge_request_pipelines.md)
      or making it [more efficient](../pipelines/pipeline_efficiency.md).
 1. Leverage the `.gitlab-ci.yml` in the components repository to [test changes to the component](index.md#test-the-component).
-1. Tag and [release the component](index.md#release-a-component).
+1. Tag and [release the component](#publish-a-new-release).
 
 ## Troubleshooting
 
 ### `content not found` message
 
 You might receive an error message similar to the following when using the `~latest`
-version qualifier to reference a component hosted by a [catalog resource](catalog.md#add-a-components-repository-to-the-catalog):
+version qualifier to reference a component hosted by a [catalog resource](#set-a-component-project-as-a-catalog-resource):
 
 ```plaintext
 This GitLab CI configuration is invalid: component 'gitlab.com/my-namespace/my-project/my-component@~latest' - content not found`
 ```
 
 The `~latest` behavior [was updated](https://gitlab.com/gitlab-org/gitlab/-/issues/429707)
-in GitLab 16.7. It now refers to the latest published version of the catalog resource. To resolve this issue, [create a new release](#release-a-component).
+in GitLab 16.7. It now refers to the latest published version of the catalog resource. To resolve this issue, [create a new release](#publish-a-new-release).