diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-14 21:11:02 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2023-12-14 21:11:02 +0300 |
commit | 0ce623783c5970e2439cda2a5eab8cbb81c194c3 (patch) | |
tree | 2bd6904b7e7a6b3c3539de89e86e60420c430452 /doc/ci/components | |
parent | d7e72d98df261209772ce059e09381d36226913f (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/ci/components')
-rw-r--r-- | doc/ci/components/index.md | 253 |
1 files changed, 135 insertions, 118 deletions
diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md index e67f636b14f..f20f2fd5e3f 100644 --- a/doc/ci/components/index.md +++ b/doc/ci/components/index.md @@ -11,33 +11,36 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Feature flag `ci_namespace_catalog_experimental` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/394772) in GitLab 16.3. > - [Moved](https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/130824) to [Beta status](../../policy/experiment-beta-support.md) in GitLab 16.6. -A CI/CD component is a reusable single pipeline configuration unit. Use them to compose an entire pipeline configuration or a small part of a larger pipeline. +A CI/CD component is a reusable single pipeline configuration unit. Use components +to create a small part of a larger pipeline, or even to compose a complete pipeline configuration. -A component can take [input parameters](../yaml/inputs.md). +A component can be configured with [input parameters](../yaml/inputs.md) for more +dynamic behavior. -CI/CD components are similar to the other kinds of [configuration added with the `include` keyword](../yaml/includes.md), but have several advantages: +CI/CD components are similar to the other kinds of [configuration added with the `include` keyword](../yaml/includes.md), +but have several advantages: +- Components can be listed in the [CI/CD Catalog](#cicd-catalog). - Components can be released and used with a specific version. - Multiple components can be defined in the same project and versioned together. -- Components are discoverable in the [CI/CD Catalog](#cicd-catalog). + +Instead of creating your own components, you can also search for published components +that have the functionality you need in the [CI/CD Catalog](#cicd-catalog). ## Component project A component project is a GitLab project with a repository that hosts one or more components. -All components in the project are versioned together. +All components in the project are versioned together, with a maximum of 10 components per project. If a component requires different versioning from other components, the component should be moved to a dedicated component project. -One component repository can have a maximum of 10 components. - ### Create a component project To create a component project, you must: 1. [Create a new project](../../user/project/index.md#create-a-blank-project) with a `README.md` file. 1. Add a YAML configuration file for each component, following the [required directory structure](#directory-structure). - For example: ```yaml @@ -51,6 +54,9 @@ To create a component project, you must: stage: $[[ inputs.stage ]] ``` +You can [use the component](#use-a-component) immediately, but you might want to consider +publishing the component to the [CI/CD catalog](#cicd-catalog). + ### Directory structure The repository must contain: @@ -65,39 +71,40 @@ The repository must contain: Configure the project's `.gitlab-ci.yml` to [test the components](#test-the-component) and [release new versions](#publish-a-new-release). -For example, if the project contains a single component, the directory structure should be similar to: +For example: -```plaintext -├── templates/ -│ └── secret-detection.yml -├── README.md -└── .gitlab-ci.yml -``` +- If the project contains a single component, the directory structure should be similar to: -If the project contains multiple components, then the directory structure should be similar to: + ```plaintext + ├── templates/ + │ └── my-component.yml + ├── README.md + └── .gitlab-ci.yml + ``` -```plaintext -├── templates/ -│ ├── all-scans.yml -│ └── secret-detection/ -│ ├── template.yml -│ ├── Dockerfile -│ └── test.sh -├── README.md -└── .gitlab-ci.yml -``` +- If the project contains multiple components, then the directory structure should be similar to: + + ```plaintext + ├── templates/ + │ ├── my-simple-component.yml + │ └── my-complex-component/ + │ ├── template.yml + │ ├── Dockerfile + │ └── test.sh + ├── README.md + └── .gitlab-ci.yml + ``` -In this example: + In this example: -- The `all-scans` component configuration is defined in a single file. -- The `secret-detection` component configuration contains multiple files in a directory. + - The `my-simple-component` component's configuration is defined in a single file. + - The `my-complex-component` component's configuration contains multiple files in a directory. ## Use a component -You can use a component in a CI/CD configuration with the `include: component` keyword. -The component is identified by a unique address formatted as `<fully-qualified-domain-name>/<project-path>/<component-name>@<specific-version>`. - -For example: +To add a component to a project's CI/CD configuration, use the [`include: component`](../yaml/index.md#includecomponent) +keyword. The component reference is formatted as `<fully-qualified-domain-name>/<project-path>/<component-name>@<specific-version>`, +for example: ```yaml include: @@ -113,15 +120,22 @@ In this example: - `my-org/security-components` is the full path of the project containing the component. - `secret-detection` is the component name that is defined as either a single file `templates/secret-detection.yml` or as a directory `templates/secret-detection/` containing a `template.yml`. -- `1.0` is the version of the component. In order of highest priority first, - the version can be: - - A branch name, for example `main`. - - A commit SHA, for example `e3262fdd0914fa823210cdb79a8c421e2cef79d8`. - - A tag, for example: `1.0`. If a tag and branch exist with the same name, the tag - takes precedence over the branch. If a tag and commit SHA exist with the same name, - the commit SHA takes precedence over the tag. - - `~latest`, which is a special version that always points to the most recent - [release published in the CI/CD Catalog](#publish-a-new-release). +- `1.0` is the [version](#component-versions) of the component. + +When GitLab creates a new pipeline, the component's configuration is fetched and added to +the pipeline's configuration. + +### Component versions + +In order of highest priority first, the component version can be: + +- A commit SHA, for example `e3262fdd0914fa823210cdb79a8c421e2cef79d8`. +- A tag, for example: `1.0`. If a tag and commit SHA exist with the same name, + the commit SHA takes precedence over the tag. +- A branch name, for example `main`. If a branch and tag exist with the same name, + the tag takes precedence over the branch. +- `~latest`, which is a special version that always points to the most recent + [release published in the CI/CD Catalog](#publish-a-new-release). NOTE: The `~latest` version keyword always returns the most recent published release, not the release with @@ -135,10 +149,11 @@ change this behavior. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/407249) in GitLab 16.1 as an [experiment](../../policy/experiment-beta-support.md#experiment). > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/432045) to [beta](../../policy/experiment-beta-support.md#beta) in GitLab 16.7. -The CI/CD Catalog is a list of projects with published CI/CD components you can use to extend your CI/CD workflow. +The CI/CD Catalog is a list of projects with published CI/CD components you can use +to extend your CI/CD workflow. -Anyone can add a [component project](index.md#component-project) to the CI/CD Catalog, or contribute -to an existing project to improve the available components. +Anyone can [create a component project](#create-a-component-project) and add it to +the CI/CD Catalog, or contribute to an existing project to improve the available components. ### View the CI/CD Catalog @@ -164,62 +179,64 @@ To publish a component project in the CI/CD catalog, you must: #### Set a component project as a catalog resource To make published versions of a component project visible in the CI/CD catalog, -you must set the project as a catalog resource: +you must set the project as a catalog resource. + +Prerequisites: + +- You must have the Owner role in the project. + +To set the project as a catalog resource: 1. On the left sidebar, select **Search or go to** and find your project. 1. Select **Settings > General**. 1. Expand **Visibility, project features, permissions**. -1. Scroll down to **CI/CD Catalog resource** and select the toggle to mark the project as a catalog resource. +1. Turn on the **CI/CD Catalog resource** toggle. -#### Publish a new release +The project only becomes findable in the catalog after you publish a new release. -Components defined in a [component project](#component-project) can be [used](#use-a-component) -immediately and don't require to be published in the CI/CD catalog. However, having the component -project published in the catalog makes it discoverable to other users. +#### Publish a new release -After the project is set as a [catalog resource](#set-a-component-project-as-a-catalog-resource), -add a job to the project's `.gitlab-ci.yml` file that creates a release using the -[`release`](../yaml/index.md#release) keyword. +CI/CD components can be [used](#use-a-component) without being listed in the CI/CD catalog. +However, publishing a component's releases in the catalog makes it discoverable to other users. -For example: - -```yaml -create-release: - stage: deploy - image: registry.gitlab.com/gitlab-org/release-cli:latest - script: echo "Creating release $CI_COMMIT_TAG" - release: - tag_name: $CI_COMMIT_TAG - description: "Release $CI_COMMIT_TAG of components in $CI_PROJECT_PATH" -``` +Prerequisites: -The release fails if the project is missing: +- The project must: + - Be set as a [catalog resource](#set-a-component-project-as-a-catalog-resource). + - Have a [project description](../../user/project/working_with_projects.md#edit-project-name-and-description) defined. + - Have a `README.md` file in the root directory for the commit SHA of the tag being released. + - Have at least one [CI/CD component in the `templates/` directory](#directory-structure) + for the commit SHA of the tag being released. -- The [project description](../../user/project/working_with_projects.md#edit-project-name-and-description), - to display in the catalog list. -- A `README.md` file in the root directory for the commit SHA of the tag being released. -- Any component in the `templates/` directory for the commit SHA of the tag being released. +To publish a new version of the component to the catalog: -Create a [new tag](../../user/project/repository/tags/index.md#create-a-tag) for the release, -which should trigger a tag pipeline that contains the job responsible that creates the release. -You should configure the tag pipeline to [test the components](#test-the-component) before -running the release job. +1. Add a job to the project's `.gitlab-ci.yml` file that uses the [`release`](../yaml/index.md#release) + keyword to create the new release. For example: -The release is created and the new version is published to the CI/CD catalog only if: + ```yaml + create-release: + stage: deploy + image: registry.gitlab.com/gitlab-org/release-cli:latest + script: echo "Creating release $CI_COMMIT_TAG" + rules: + - if: $CI_COMMIT_TAG + release: + tag_name: $CI_COMMIT_TAG + description: "Release $CI_COMMIT_TAG of components in $CI_PROJECT_PATH" + ``` -- All jobs before the release job succeed. -- All component project [requirements](#directory-structure) are satisfied. -- The component project is [set as a catalog resource](#set-a-component-project-as-a-catalog-resource). +1. Create a [new tag](../../user/project/repository/tags/index.md#create-a-tag) for the release, + which should trigger a tag pipeline that contains the job responsible for creating the release. + You should configure the tag pipeline to [test the components](#test-the-component) before + running the release job. -NOTE: -If you disable [catalog resource setting](#set-a-component-project-as-a-catalog-resource), -the component project and all versions are removed from the catalog. To publish it again, -you must re-enable the setting and release a new version. +After the release job completes successfully, the release is created and the new version +is published to the CI/CD catalog. ### Unpublish a component project -To remove a component project from the catalog, turn off the [**CI/CD Catalog resource**](#set-a-component-project-as-a-catalog-resource) toggle. -in the project settings. +To remove a component project from the catalog, turn off the [**CI/CD Catalog resource**](#set-a-component-project-as-a-catalog-resource) +toggle in the project settings. WARNING: This action destroys the metadata about the component project and its versions published @@ -251,9 +268,9 @@ include: stages: [build, test, release] -# Expect `component-job` is added. -# This example tests that the included component works as expected. -# You can inspect data generated by the component, use GitLab API endpoints or third-party tools. +# Check if `component-job` is added. +# This example job could also test that the included component works as expected. +# You can inspect data generated by the component, use GitLab API endpoints, or third-party tools. ensure-job-added: stage: test image: badouralix/curl-jq @@ -265,8 +282,8 @@ ensure-job-added: exit 1 fi -# If we are tagging a release with a semantic version and all previous checks succeeded, -# we proceed with creating a release automatically. +# If the pipeline is for a new tag with a semantic version, and all previous jobs succeed, +# create the release. create-release: stage: release image: registry.gitlab.com/gitlab-org/release-cli:latest @@ -278,7 +295,8 @@ create-release: description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH" ``` -After committing and pushing changes, the pipeline tests the component, then releases the tag it if the test passes. +After committing and pushing changes, the pipeline tests the component, then creates +a release if the earlier jobs pass. ### Avoid using global keywords @@ -286,7 +304,7 @@ Avoid using [global keywords](../yaml/index.md#global-keywords) in a component. Using these keywords in a component affects all jobs in a pipeline, including jobs directly defined in the main `.gitlab-ci.yml` or in other included components. -As an alternative to global keywords, instead: +As an alternative to global keywords: - Add the configuration directly to each job, even if it creates some duplication in the component configuration. @@ -294,7 +312,7 @@ As an alternative to global keywords, instead: unique names that reduce the risk of naming conflicts when the component is merged into the configuration. -For example, using the `default` keyword is not recommended: +For example, avoid using the `default` global keyword: ```yaml # Not recommended @@ -310,7 +328,7 @@ rspec-2: Instead, you can: -- Add the configuration to each job: +- Add the configuration to each job explicitly: ```yaml rspec-1: @@ -339,21 +357,21 @@ Instead, you can: script: bundle exec rspec dir2/ ``` -### Replace hard-coded values with inputs +### Replace hardcoded values with inputs -Avoid hard-coding values in CI/CD components. Hard-coded values might force +Avoid using hardcoded values in CI/CD components. Hardcoded values might force component users to need to review the component's internal details and adapt their pipeline to work with the component. A common keyword with problematic hard-coded values is `stage`. If a component job's -stage is set to a specific value, the pipeline using the component **must** define -the exact same stage. Additionally, if the component user wants to use a different stage, -they must [override](../yaml/includes.md#override-included-configuration-values) the configuration. +stage is hardcoded, all pipelines using the component **must** either define +the exact same stage, or [override](../yaml/includes.md#override-included-configuration-values) +the configuration. -The preferred method is to use the [`input` keyword](../yaml/inputs.md). -The component user can specify the exact value they need. +The preferred method is to use the [`input` keyword](../yaml/inputs.md) for dynamic +component configuration. The component user can specify the exact value they need. -For example: +For example, to create a component with `stage` configuration that can be defined by users: - In the component configuration: @@ -372,28 +390,28 @@ For example: script: echo integration tests ``` -- In the project using the component: +- In a project using the component: ```yaml + stages: [verify, deploy] + include: - component: gitlab.com/gitlab-org/ruby-test@1.0 inputs: stage: verify - - stages: [verify, deploy] ``` ### Replace custom CI/CD variables with inputs When using CI/CD variables in a component, evaluate if the `inputs` keyword -should be used instead. Avoid requiring a user to define custom variables to change a component's -behavior. You should try to use `inputs` for any component customization. +should be used instead. Avoid asking users to define custom variables to configure +components when `inputs` is a better solution. -Inputs are explicitly defined in the component's specs, and are better validated than variables. +Inputs are explicitly defined in the component's specs, and have better validation than variables. For example, if a required input is not passed to the component, GitLab returns a pipeline error. By contrast, if a variable is not defined, its value is empty, and there is no error. -For example, use `inputs` instead of variables to let users change a scanner's output format: +For example, use `inputs` instead of variables to configure a scanner's output format: - In the component configuration: @@ -416,18 +434,17 @@ For example, use `inputs` instead of variables to let users change a scanner's o scanner-output: yaml ``` -In other cases, CI/CD variables are still preferred, including: +In other cases, CI/CD variables might still be preferred. For example: -- Using [predefined variables](../variables/predefined_variables.md) to automatically configure +- Use [predefined variables](../variables/predefined_variables.md) to automatically configure a component to match a user's project. -- Requiring tokens or other sensitive values to be stored as [masked or protected variables in project settings](../variables/index.md#define-a-cicd-variable-in-the-ui). +- Ask users to store sensitive values as [masked or protected CI/CD variables in project settings](../variables/index.md#define-a-cicd-variable-in-the-ui). ### Use semantic versioning -When tagging and [releasing new versions](#publish-a-new-release) of components, you should use -[semantic versioning](https://semver.org). -Semantic versioning is the standard for communicating that a change is a major, minor, patch, -or other kind of change. +When tagging and [releasing new versions](#publish-a-new-release) of components, +you should use [semantic versioning](https://semver.org). Semantic versioning is the standard +for communicating that a change is a major, minor, patch, or other kind of change. You should use at least the `major.minor` format, as this is widely understood. For example, `2.0` or `2.1`. @@ -444,9 +461,9 @@ Other examples of semantic versioning: Any existing CI/CD template that you use in projects by using the `include:` syntax can be converted to a CI/CD component: -1. Decide if you want the component to be part of an existing [component project](index.md#component-project) - to be grouped with other components, or [create a new component project](#create-a-component-project). -1. Create a YAML file in the component project according to the expected [directory structure](index.md#directory-structure). +1. Decide if you want the component to be grouped with other components as part of + an existing [component project](index.md#component-project), or [create a new component project](#create-a-component-project). +1. Create a YAML file in the component project according to the [directory structure](index.md#directory-structure). 1. Copy the content of the original template YAML file into the new component YAML file. 1. Refactor the new component's configuration to: - Follow the [best practices](index.md#best-practices) for components. |