diff options
Diffstat (limited to 'doc/ci/yaml/includes.md')
| -rw-r--r-- | doc/ci/yaml/includes.md | 153 |
1 files changed, 121 insertions, 32 deletions
diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index bf0b7444e78..35b302d0373 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -5,7 +5,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: reference --- -# GitLab CI/CD include examples **(FREE)** +# Use CI/CD configuration from other files **(FREE)** You can use [`include`](index.md#include) to include external YAML files in your CI/CD jobs. @@ -20,13 +20,6 @@ To include a single configuration file, use either of these syntax options: include: '/templates/.after-script-template.yml' ``` -- `include` with a single file, and you specify the `include` type: - - ```yaml - include: - remote: 'https://gitlab.com/awesome-project/raw/main/.before-script-template.yml' - ``` - ## Include an array of configuration files You can include an array of configuration files: @@ -159,12 +152,18 @@ do not change. This method is called *merging*. ### Merge method for `include` -For a file containing `include` directives, the included files are read in order (possibly -recursively), and the configuration in these files is likewise merged in order. If the parameters overlap, the last included file takes precedence. Finally, the directives in the -file itself are merged with the configuration from the included files. +The `include` configuration merges with the main configuration file with this process: + +- Included files are read in the order defined in the configuration file, and + the included configuration is merged together in the same order. +- If an included file also uses `include`, that nested `include` configuration is merged first (recursively). +- If parameters overlap, the last included file takes precedence when merging the configuration + from the included files. +- After all configuration added with `include` is merged together, the main configuration + is merged with the included configuration. This merge method is a _deep merge_, where hash maps are merged at any depth in the -configuration. To merge hash map A (containing the configuration merged so far) and B (the next piece +configuration. To merge hash map "A" (that contains the configuration merged so far) and "B" (the next piece of configuration), the keys and values are processed as follows: - When the key only exists in A, use the key and value from A. @@ -172,9 +171,7 @@ of configuration), the keys and values are processed as follows: - When the key exists in both A and B, and one of the values is not a hash map, use the value from B. - Otherwise, use the key and value from B. -For example: - -We have a configuration consisting of two files. +For example, with a configuration that consists of two files: - The `.gitlab-ci.yml` file: @@ -211,7 +208,7 @@ We have a configuration consisting of two files. dotenv: deploy.env ``` -The merged result: +The merged result is: ```yaml variables: @@ -374,7 +371,7 @@ In `include` sections in your `.gitlab-ci.yml` file, you can use: - [Project variables](../variables/index.md#for-a-project). - [Group variables](../variables/index.md#for-a-group). - [Instance variables](../variables/index.md#for-an-instance). -- Project [predefined variables](../variables/predefined_variables.md). +- Project [predefined variables](../variables/predefined_variables.md) (`CI_PROJECT_*`). - In GitLab 14.2 and later, the `$CI_COMMIT_REF_NAME` [predefined variable](../variables/predefined_variables.md). When used in `include`, the `CI_COMMIT_REF_NAME` variable returns the full @@ -386,15 +383,7 @@ In GitLab 14.5 and later, you can also use: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](../pipelines/index.md#run-a-pipeline-manually). -- Pipeline [predefined variables](../variables/predefined_variables.md). - - YAML files are parsed before the pipeline is created, so the following pipeline predefined variables - are **not** available: - - - `CI_PIPELINE_ID` - - `CI_PIPELINE_URL` - - `CI_PIPELINE_IID` - - `CI_PIPELINE_CREATED_AT` +- The `CI_PIPELINE_SOURCE` and `CI_PIPELINE_TRIGGERED` [predefined variables](../variables/predefined_variables.md). For example: @@ -416,7 +405,8 @@ see this [CI/CD variable demo](https://youtu.be/4XR8gw3Pkos). > - Introduced in GitLab 14.2 [with a flag](../../administration/feature_flags.md) named `ci_include_rules`. Disabled by default. > - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.3. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/337507) in GitLab 14.4. Feature flag `ci_include_rules` removed. -> - [Support for `exists` keyword added](https://gitlab.com/gitlab-org/gitlab/-/issues/341511) in GitLab 14.5. +> - Support for `exists` keyword [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/341511) in GitLab 14.5. +> - Support for `needs` job dependency [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) in GitLab 15.11. You can use [`rules`](index.md#rules) with `include` to conditionally include other configuration files. @@ -426,11 +416,6 @@ these keywords: - [`rules:if`](index.md#rulesif). - [`rules:exists`](index.md#rulesexists). -You cannot use [`needs:`](index.md#needs) to create a job dependency that points to -a job added with `include:local:rules`. When the configuration is validated, -GitLab returns `undefined need: <job-name>`. [Issue 345377](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) -proposes improving this behavior. - ### `include` with `rules:if` Use [`rules:if`](index.md#rulesif) to conditionally include other configuration files @@ -517,3 +502,107 @@ When the pipeline runs, GitLab: # This matches all `.yml` files only in subfolders of `configs`. include: 'configs/**/*.yml' ``` + +## Define inputs for configuration added with `include` (Beta) + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/391331) in GitLab 15.11 as a Beta feature. + +FLAG: +`spec` and `with` are experimental [Open Beta features](../../policy/alpha-beta-support.md#beta) +and subject to change without notice. + +### Define input parameters with `spec:inputs` + +Use `spec:inputs` to define input parameters for CI/CD configuration intended to be added +to a pipeline with `include`. Use [`include:inputs`](#set-input-parameter-values-with-includeinputs) +to define the values to use when the pipeline runs. + +The specs must be declared at the top of the configuration file, in a header section. +Separate the header from the rest of the configuration with `---`. + +Use the interpolation format `$[[ input.input-id ]]` to reference the values outside of the header section. +The inputs are evaluated and interpolated once, when the configuration is fetched +during pipeline creation, but before the configuration is merged with the contents of the `.gitlab-ci.yml`. + +```yaml +spec: + inputs: + environment: + job-stage: +--- + +scan-website: + stage: $[[ inputs.job-stage ]] + script: ./scan-website $[[ inputs.environment ]] +``` + +When using `spec:inputs`: + +- Defined inputs are mandatory by default. +- Inputs can be made optional by specifying a `default`. Use `default: null` to have no default value. +- A string containing an interpolation block must not exceed 1 MB. +- The string inside an interpolation block must not exceed 1 KB. + +For example, a `custom_configuration.yml`: + +```yaml +spec: + inputs: + website: + user: + default: 'test-user' + flags: + default: null +--- + +# The pipeline configuration would follow... +``` + +In this example: + +- `website` is mandatory and must be defined. +- `user` is optional. If not defined, the value is `test-user`. +- `flags` is optional. If not defined, it has no value. + +### Set input parameter values with `include:inputs` + +> `include:with` [renamed to `include:inputs`](https://gitlab.com/gitlab-org/gitlab/-/issues/406780) in GitLab 16.0. + +Use `include:inputs` to set the values for the parameters when the included configuration +is added to the pipeline. + +For example, to include a `custom_configuration.yml` that has the same specs +as the [example above](#define-input-parameters-with-specinputs): + +```yaml +include: + - local: 'custom_configuration.yml' + inputs: + website: "My website" +``` + +In this example: + +- `website` has a value of `My website` for the included configuration. +- `user` has a value of `test-user`, because that is the default when not specified. +- `flags` has no value, because it is optional and has no default when not specified. + +## Troubleshooting + +### `Maximum of 150 nested includes are allowed!` error + +The maximum number of [nested included files](#use-nested-includes) for a pipeline is 150. +If you receive the `Maximum 150 includes are allowed` error message in your pipeline, +it's likely that either: + +- Some of the nested configuration includes an overly large number of additional nested `include` configuration. +- There is an accidental loop in the nested includes. For example, `include1.yml` includes + `include2.yml` which includes `include1.yml`, creating a recursive loop. + +To help reduce the risk of this happening, edit the pipeline configuration file +with the [pipeline editor](../pipeline_editor/index.md), which validates if the +limit is reached. You can remove one included file at a time to try to narrow down +which configuration file is the source of the loop or excessive included files. + +In [GitLab 16.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/207270) self-managed users can +change the [maximum includes](../../user/admin_area/settings/continuous_integration.md#maximum-includes) value. |