Add latest changes from gitlab-org/gitlab@16-8-stable-eev16.8.0-rc42

8 files changed, 188 insertions, 6 deletions

diff --git a/doc/development/cicd/cicd_reference_documentation_guide.md b/doc/development/cicd/cicd_reference_documentation_guide.md
index ccd952f586c..3832ba182b6 100644
--- a/doc/development/cicd/cicd_reference_documentation_guide.md
+++ b/doc/development/cicd/cicd_reference_documentation_guide.md
@@ -6,7 +6,7 @@ info: Any user with at least the Maintainer role can merge updates to this conte
 
 # Documenting the `.gitlab-ci.yml` keywords
 
-The [CI/CD YAML reference](../../ci/yaml/index.md) uses a standard style to make it easier to use and update.
+The [CI/CD YAML syntax reference](../../ci/yaml/index.md) uses a standard style to make it easier to use and update.
 
 The reference information should be kept as simple as possible, and expanded details
 and examples should be documented on other pages.
diff --git a/doc/development/cicd/components.md b/doc/development/cicd/components.md
new file mode 100644
index 00000000000..56ab5a24bd1
--- /dev/null
+++ b/doc/development/cicd/components.md
@@ -0,0 +1,80 @@
+---
+stage: Verify
+group: Pipeline Authoring
+info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
+---
+
+# Development guide for GitLab CI/CD components
+
+This document explains how to develop [CI/CD components](../../ci/components/index.md) that are maintained by GitLab.
+
+The official location for all GitLab-maintained component projects is the [`gitlab.com/components`](https://gitlab.com/components) group.
+This group contains all components that are designed to be generic, served to all GitLab users, and maintained by GitLab.
+
+A component project can initially be created under a different group (for example `gitlab-org`)
+but they need to be moved into the `components` group before the first version gets published to the catalog.
+
+Components that are for GitLab internal use only, for example specific to `gitlab-org/gitlab` project, should be
+implemented under `gitlab-org` group.
+
+Component projects that are expected to be published in the [CI/CD catalog](../../ci/components/index.md#cicd-catalog)
+should first be dogfooded it to ensure we stay on top of the project quality and have first-hand
+experience with it.
+
+## Define ownership
+
+GitLab-maintained components are trusted by the community and require a high degree of quality and timely maintenance.
+Components must be kept up to date, monitored for security vulnerabilities, and bugs fixed.
+
+Each component project must have a set of owners and maintainers that are also domain experts.
+Experts can be from any department in GitLab, from Engineering to Support, Customer Success, and Developer Relations.
+
+If a component is related to a GitLab feature (for example Secret Detection), the team that owns the
+feature category or is most closely related to it should maintain the project.
+
+The component project can be created by a separate team or individual initially but it must be transitioned
+to a set of owners before the first version gets published to the catalog.
+
+The `README.md` file in the project repository must indicate the main owners of the project so that
+they can be contacted by the wider community if needed.
+
+NOTE:
+If a set of project owners cannot be guaranteed or the components cannot be dogfooded, we strongly recommend
+not creating a GitLab-maintained component project and instead let the wider community fulfill the demand
+in the catalog.
+
+## Development process
+
+1. Create a project under [`gitlab.com/components`](https://gitlab.com/components)
+   or ask one of the group owners to create an empty project for you.
+1. Follow the [standard guide for creating components](../../ci/components/index.md).
+1. Add a concise project description that clearly describes the capabilities offered by the component project.
+1. Ensure that the [best practices](../../ci/components/index.md#best-practices) are followed.
+1. Use [semantic versioning](https://semver.org) in the form `MAJOR.MINOR` or `MAJOR.MINOR.PATCH`.
+1. Add a `LICENSE.md` file with the MIT license.
+1. The project must have a `.gitlab-ci.yml` file that:
+   - Validates all the components in the project correctly.
+   - Contains a `release` job to publish newly released tags to the catalog.
+1. Ensure that the `README.md` contains at least the sections below (for example, see the [Code quality component](https://gitlab.com/components/code-quality)):
+   - **Overview**: The capabilities offered by the component project.
+   - **Components**: Sub-sections for each component, each with:
+     - **Usage**: Examples with and without inputs (when optional).
+     - **Inputs**: A table showing the input names, types, default values (if any) and descriptions.
+     - **Variables** (when applicable): The variable names, possible values, and descriptions.
+   - **Contribute**: Notes and how to get in touch with the maintainers.
+     Usually the contribution process should follow the [official guide](../../ci/components/index.md).
+1. Upload the [official avatar image](img/avatar_component_project.png) to the component project.
+
+## Review and contribution process
+
+It's possible that components in the project have a related [CI/CD template](templates.md) in the GitLab codebase.
+In that case we need to cross link the component project and CI/CD template:
+
+- Add a comment in the CI/CD template with the location of the related component project.
+- Add a section in the `README.md` of the component project with the location of the existing CI/CD template.
+
+When changes are applied to these components, check whether we can integrate the changes in the CI/CD template too.
+This might not be possible due to the rigidity of versioning in CI/CD templates.
+
+Ping [`@gitlab-org/maintainers/ci-components`](https://gitlab.com/groups/gitlab-org/maintainers/ci-components/-/group_members?with_inherited_permissions=exclude)
+for reviews to ensure that the components are written in consistent style and follow the best practices.
diff --git a/doc/development/cicd/configuration.md b/doc/development/cicd/configuration.md
new file mode 100644
index 00000000000..60f55174651
--- /dev/null
+++ b/doc/development/cicd/configuration.md
@@ -0,0 +1,100 @@
+---
+stage: Verify
+group: Pipeline Authoring
+info: Any user with at least the Maintainer role can merge updates to this content. For details, see https://docs.gitlab.com/ee/development/development_processes.html#development-guidelines-review.
+---
+
+# Contribute to the CI/CD configuration
+
+## Glossary
+
+- **CI/CD configuration**: The YAML file that defines the CI/CD configuration for a project.
+- **keyword**: Each keyword in the CI/CD configuration.
+- **entry**: An `Entry` class that represents a keyword in the CI/CD configuration.
+
+Not every keyword in the CI/CD configuration is represented by an `Entry` class.
+We create `Entry` classes for keywords that have a complex structure or reusable parts.
+
+For example;
+
+- The `image` keyword is represented by the [`Entry::Image`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/image.rb) class.
+- The `name` subkeyword of the `image` keyword is not represented by an `Entry` class.
+- The `pull_policy` subkeyword of the `image` keyword is represented by the [`Entry::PullPolicy`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/pull_policy.rb) class.
+
+## Adding New Keywords
+
+CI config keywords are added in the [`lib/gitlab/ci/config/entry`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/config/entry) directory.
+For EE-specific changes, use the [`ee/lib/gitlab/ci/config/entry`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/ci/config/entry)
+or [`ee/lib/ee/gitlab/ci/config/entry`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/ee/gitlab/ci/config/entry) directory.
+
+### Inheritance
+
+An entry is represented by a class that inherits from;
+
+- `Entry::Node`: for simple keywords.
+  (e.g. [`Entry::Stage`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/stage.rb))
+- `Entry::Simplifiable`: for keywords that have multiple structures.
+  For example, [`Entry::Retry`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/retry.rb) can be a simple number or a hash configuration.
+- `Entry::ComposableArray`: for keywords that have a list of single-type sub-elements.
+  For example, [`Entry::Includes`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/includes.rb) has a list of `Entry::Include` elements.
+- `Entry::ComposableHash`: for keywords that have single-type sub-elements with user-defined keys.
+  For example, [`Entry::Variables`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/config/entry/variables.rb) has a list of `Entry::Variable` elements with user-defined keys.
+
+### Helper Classes
+
+The following helper classes are available for use in entries:
+
+- `Entry::Validatable`: Enables the `validations` block in an entry class and provides validations.
+- `Entry::Attributable`: Enables the `attributes` method in an entry class. It creates these methods for each attribute; `xxx`, `has_xxx?`, `has_xxx_value?`.
+- `Entry::Configurable`: Enables the `entry` method in an entry class. It creates these methods for each entry; `xxx_defined?`, `xxx_entry`, `xxx_value`.
+
+### The `value` Method
+
+The `value` method is the main method of an entry class. It returns the actual value of the entry.
+By default, from the `Entry::Node` class, the `value` method returns the hash configuration of the entry unless it has nested entries.
+It can be useful for simple entries. For example, `Entry::Paths` has an array of strings as its value. So, it can return the array of strings directly.
+
+In some keywords, we override the `value` method. In this method, we return what and how we want to return from the entry.
+The usage of `Entry::Attributable` and `Entry::Configurable` may have a significant role here. For example,
+in `Entry::Secret`, we have this;
+
+```ruby
+attributes %i[vault file token].freeze
+
+entry :vault, Entry::Vault::Secret
+entry :file, ::Gitlab::Config::Entry::Boolean
+
+def value
+  {
+    vault: vault_value,
+    file: file_value,
+    token: token
+  }.compact
+end
+```
+
+- `vault_value` is the value of the nested `vault` entry.
+- `file_value` is the value of the nested `file` entry.
+- `token` is the value of the basic `token` attribute.
+
+**It is important** that we should always use the `xxx_value` method to get the value of a nested entry.
+
+## Feature Flag Usage
+
+When adding new CI/CD configuration keywords, it is important to use feature flags to control the rollout of the change.
+This allows us to test the change in production without affecting all users. For more information, see the [feature flags documentation](../feature_flags/index.md).
+
+### Feature Flag Actor
+
+In entry classes, we have no access to the current project or user. However, it's discouraged to use feature flags without [an actor](../feature_flags/index.md#feature-actors).
+To solve this problem, we have three options;
+
+1. Use `Feature.enabled?(:feature_flag, Feature.current_request)`.
+1. Use `YamlProcessor::FeatureFlags.enabled?(:feature_flag)`
+1. Do not use feature flags in entry classes and use them in other parts of the code.
+
+## Testing and Validation
+
+When adding or modifying an entry, the corresponding spec file must be either added or updated.
+Besides, to have a fully integrated test, it's also important to add/modify tests in the `spec/lib/gitlab/ci/yaml_processor_spec.rb` file or
+the files in `spec/lib/gitlab/ci/yaml_processor/test_cases/*` directory.
diff --git a/doc/development/cicd/img/avatar_component_project.png b/doc/development/cicd/img/avatar_component_project.png
new file mode 100644
index 00000000000..e5c20d108fa
--- /dev/null
+++ b/doc/development/cicd/img/avatar_component_project.png
diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md
index 18781f9315a..a8dfdeb30a3 100644
--- a/doc/development/cicd/index.md
+++ b/doc/development/cicd/index.md
@@ -9,10 +9,12 @@ info: Any user with at least the Maintainer role can merge updates to this conte
 Development guides that are specific to CI/CD are listed here:
 
 - If you are creating new CI/CD templates, read [the development guide for GitLab CI/CD templates](templates.md).
-- If you are adding a new keyword or changing the CI schema, check the [CI schema guide](schema.md)
+- If you are adding a new keyword or changing the CI schema, refer to the following guides:
+  - [The CI configuration guide](configuration.md)
+  - [The CI schema guide](schema.md)
 
 See the [CI/CD YAML reference documentation guide](cicd_reference_documentation_guide.md)
-to learn how to update the [reference page](../../ci/yaml/index.md).
+to learn how to update the [CI/CD YAML syntax reference page](../../ci/yaml/index.md).
 
 ## Examples of CI/CD usage
 
diff --git a/doc/development/cicd/pipeline_wizard.md b/doc/development/cicd/pipeline_wizard.md
index 0c0c0f3cc45..b534fede6e9 100644
--- a/doc/development/cicd/pipeline_wizard.md
+++ b/doc/development/cicd/pipeline_wizard.md
@@ -138,7 +138,7 @@ is planned to add the ability to create a MR from here.
 ### Events
 
 - `done` - Emitted after the file has been committed. Use this to redirect the
-user to the pipeline, for example.
+  user to the pipeline, for example.
 
 ### Template file location
 
diff --git a/doc/development/cicd/schema.md b/doc/development/cicd/schema.md
index e9a0b93b5f3..c24b7d21286 100644
--- a/doc/development/cicd/schema.md
+++ b/doc/development/cicd/schema.md
@@ -30,7 +30,7 @@ a step-by-step introduction on how to work with JSON schemas.
 
 The CI/CD schema is at [`app/assets/javascripts/editor/schema/ci.json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json).
 It contains all the keywords available for authoring CI/CD configuration files.
-Check the [keyword reference](../../ci/yaml/index.md) for a comprehensive list of
+Check the [CI/CD YAML syntax reference](../../ci/yaml/index.md) for a comprehensive list of
 all available keywords.
 
 All keywords are defined under `definitions`. We use these definitions as
diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md
index a2b490b9106..bd3023ebf8d 100644
--- a/doc/development/cicd/templates.md
+++ b/doc/development/cicd/templates.md
@@ -13,7 +13,7 @@ we encourage team members to create [CI/CD components](../../ci/components/index
 for the catalog. This transition enhances the modularity and maintainability of our
 shared CI/CD resources, and avoids the complexities of contributing new CI/CD templates.
 If you need to update an existing template, you must also update the matching CI/CD component.
-If no component exists that matches the CI/CD template yet, consider creating the matching component.
+If no component exists that matches the CI/CD template yet, consider [creating the matching component](components.md).
 This ensures that template and component functionality remain in sync, aligning with
 our new development practices.