Add latest changes from gitlab-org/gitlab@master

1 files changed, 58 insertions, 355 deletions

diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md
index 2e4049f91af..5c11e823c68 100644
--- a/doc/ci/yaml/README.md
+++ b/doc/ci/yaml/README.md
@@ -2800,6 +2800,32 @@ URI-encoded `%2F`. A value made only of dots (`.`, `%2E`) is also forbidden.
 
 You can specify a [fallback cache key](#fallback-cache-key) to use if the specified `cache:key` is not found.
 
+#### Fallback cache key
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4.
+
+You can use the `$CI_COMMIT_REF_SLUG` [variable](#variables) to specify your [`cache:key`](#cachekey).
+For example, if your `$CI_COMMIT_REF_SLUG` is `test` you can set a job
+to download cache that's tagged with `test`.
+
+If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to
+specify a cache to use when none exists.
+
+For example:
+
+```yaml
+variables:
+  CACHE_FALLBACK_KEY: fallback-key
+
+cache:
+  key: "$CI_COMMIT_REF_SLUG"
+  paths:
+    - binaries/
+```
+
+In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined
+by the `CACHE_FALLBACK_KEY` variable.
+
 ##### `cache:key:files`
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18986) in GitLab v12.5.
@@ -3483,7 +3509,7 @@ Possible values for `when` are:
 - `scheduler_failure`: Retry if the scheduler failed to assign the job to a runner.
 - `data_integrity_failure`: Retry if there was a structural integrity problem detected.
 
-You can specify the number of [retry attempts for certain stages of job execution](#job-stages-attempts) using variables.
+You can specify the number of [retry attempts for certain stages of job execution](../runners/README.md#job-stages-attempts) using variables.
 
 ### `timeout`
 
@@ -4145,8 +4171,8 @@ Read more on [GitLab Pages user documentation](../../user/project/pages/index.md
 
 > Introduced in GitLab Runner v0.5.0.
 
-Variables are configurable values that are passed to jobs. They can be set
-globally and per-job.
+[CI/CD variables](../variables/README.md) are configurable values that are passed to jobs.
+They can be set globally and per-job.
 
 There are two types of variables.
 
@@ -4163,372 +4189,49 @@ Variables are meant for non-sensitive project configuration, for example:
 
 ```yaml
 variables:
-  DATABASE_URL: "postgres://postgres@postgres/my_database"
+  DEPLOY_SITE: "https://example.com/"
+
+deploy_job:
+  stage: deploy
+  script:
+    - deploy-script --url $DEPLOY_SITE --path "/"
+
+deploy_review_job:
+  stage: deploy
+  variables:
+    REVIEW_PATH: "/review"
+  script:
+    - deploy-review-script --url $DEPLOY_SITE --path $REVIEW_PATH
 ```
 
-You can use integers and strings for the variable's name and value.
-You cannot use floats.
+You can use only integers and strings for the variable's name and value.
 
 If you define a variable at the top level of the `gitlab-ci.yml` file, it is global,
-meaning it applies to all jobs.
-
-If you define a variable within a job, it's available to that job only.
+meaning it applies to all jobs. If you define a variable within a job, it's available
+to that job only.
 
 If a variable of the same name is defined globally and for a specific job, the
 [job-specific variable is used](../variables/README.md#priority-of-environment-variables).
 
 All YAML-defined variables are also set to any linked
-[service containers](../docker/using_docker_images.md#what-is-a-service).
-
-[YAML anchors for variables](#yaml-anchors-for-variables) are available.
-
-Learn more about [variables and their priority](../variables/README.md).
-
-### Git strategy
-
-> - Introduced in GitLab 8.9 as an experimental feature.
-> - `GIT_STRATEGY=none` requires GitLab Runner v1.7+.
-
-You can set the `GIT_STRATEGY` used for getting recent application code, either
-globally or per-job in the [`variables`](#variables) section. If left
-unspecified, the default from the project settings is used.
-
-There are three possible values: `clone`, `fetch`, and `none`.
-
-`clone` is the slowest option. It clones the repository from scratch for every
-job, ensuring that the local working copy is always pristine.
-
-```yaml
-variables:
-  GIT_STRATEGY: clone
-```
-
-`fetch` is faster as it re-uses the local working copy (falling back to `clone`
-if it does not exist). `git clean` is used to undo any changes made by the last
-job, and `git fetch` is used to retrieve commits made since the last job ran.
-
-```yaml
-variables:
-  GIT_STRATEGY: fetch
-```
-
-`none` also re-uses the local working copy. However, it skips all Git operations,
-including GitLab Runner's pre-clone script, if present.
-
-It's useful for jobs that operate exclusively on artifacts, like a deployment job.
-Git repository data may be present, but it's likely out-of-date. You should only
-rely on files brought into the local working copy from cache or artifacts.
-
-```yaml
-variables:
-  GIT_STRATEGY: none
-```
-
-NOTE: **Note:**
-`GIT_STRATEGY` is not supported for
-[Kubernetes executor](https://docs.gitlab.com/runner/executors/kubernetes.html),
-but may be in the future. See the [support Git strategy with Kubernetes executor feature proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3847)
-for updates.
-
-### Git submodule strategy
-
-> Requires GitLab Runner v1.10+.
-
-The `GIT_SUBMODULE_STRATEGY` variable is used to control if / how Git
-submodules are included when fetching the code before a build. You can set them
-globally or per-job in the [`variables`](#variables) section.
-
-There are three possible values: `none`, `normal`, and `recursive`:
-
-- `none` means that submodules are not included when fetching the project
-  code. This is the default, which matches the pre-v1.10 behavior.
-
-- `normal` means that only the top-level submodules are included. It's
-  equivalent to:
-
-  ```shell
-  git submodule sync
-  git submodule update --init
-  ```
-
-- `recursive` means that all submodules (including submodules of submodules)
-  are included. This feature needs Git v1.8.1 and later. When using a
-  GitLab Runner with an executor not based on Docker, make sure the Git version
-  meets that requirement. It's equivalent to:
-
-  ```shell
-  git submodule sync --recursive
-  git submodule update --init --recursive
-  ```
-
-For this feature to work correctly, the submodules must be configured
-(in `.gitmodules`) with either:
-
-- the HTTP(S) URL of a publicly-accessible repository, or
-- a relative path to another repository on the same GitLab server. See the
-  [Git submodules](../git_submodules.md) documentation.
-
-### Git checkout
-
-> Introduced in GitLab Runner 9.3.
-
-The `GIT_CHECKOUT` variable can be used when the `GIT_STRATEGY` is set to either
-`clone` or `fetch` to specify whether a `git checkout` should be run. If not
-specified, it defaults to true. You can set them globally or per-job in the
-[`variables`](#variables) section.
-
-If set to `false`, the runner:
-
-- when doing `fetch` - updates the repository and leaves the working copy on
-  the current revision,
-- when doing `clone` - clones the repository and leaves the working copy on the
-  default branch.
-
-If `GIT_CHECKOUT` is set to `true`, both `clone` and `fetch` work the same way.
-The runner checks out the working copy of a revision related
-to the CI pipeline:
-
-```yaml
-variables:
-  GIT_STRATEGY: clone
-  GIT_CHECKOUT: "false"
-script:
-  - git checkout -B master origin/master
-  - git merge $CI_COMMIT_SHA
-```
-
-### Git clean flags
-
-> Introduced in GitLab Runner 11.10
-
-The `GIT_CLEAN_FLAGS` variable is used to control the default behavior of
-`git clean` after checking out the sources. You can set it globally or per-job in the
-[`variables`](#variables) section.
-
-`GIT_CLEAN_FLAGS` accepts all possible options of the [`git clean`](https://git-scm.com/docs/git-clean)
-command.
-
-`git clean` is disabled if `GIT_CHECKOUT: "false"` is specified.
-
-If `GIT_CLEAN_FLAGS` is:
-
-- Not specified, `git clean` flags default to `-ffdx`.
-- Given the value `none`, `git clean` is not executed.
-
-For example:
-
-```yaml
-variables:
-  GIT_CLEAN_FLAGS: -ffdx -e cache/
-script:
-  - ls -al cache/
-```
-
-### Git fetch extra flags
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4142) in GitLab Runner 13.1.
-
-The `GIT_FETCH_EXTRA_FLAGS` variable is used to control the behavior of
-`git fetch`. You can set it globally or per-job in the [`variables`](#variables) section.
-
-`GIT_FETCH_EXTRA_FLAGS` accepts all options of the [`git fetch`](https://git-scm.com/docs/git-fetch) command. However, `GIT_FETCH_EXTRA_FLAGS` flags are appended after the default flags that can't be modified.
-
-The default flags are:
+[Docker service containers](../docker/using_docker_images.md#what-is-a-service).
 
-- [GIT_DEPTH](#shallow-cloning).
-- The list of [refspecs](https://git-scm.com/book/en/v2/Git-Internals-The-Refspec).
-- A remote called `origin`.
+You can use [YAML anchors for variables](#yaml-anchors-for-variables).
 
-If `GIT_FETCH_EXTRA_FLAGS` is:
+### Configure runner behavior with variables
 
-- Not specified, `git fetch` flags default to `--prune --quiet` along with the default flags.
-- Given the value `none`, `git fetch` is executed only with the default flags.
+You can use [CI/CD variables](../variables/README.md) to configure runner Git behavior:
 
-For example, the default flags are `--prune --quiet`, so you can make `git fetch` more verbose by overriding this with just `--prune`:
-
-```yaml
-variables:
-  GIT_FETCH_EXTRA_FLAGS: --prune
-script:
-  - ls -al cache/
-```
-
-The configuration above results in `git fetch` being called this way:
-
-```shell
-git fetch origin $REFSPECS --depth 50  --prune
-```
-
-Where `$REFSPECS` is a value provided to the runner internally by GitLab.
-
-### Job stages attempts
-
-> Introduced in GitLab, it requires GitLab Runner v1.9+.
-
-You can set the number of attempts that the running job tries to execute
-the following stages:
-
-| Variable                          | Description                                            |
-|-----------------------------------|--------------------------------------------------------|
-| **ARTIFACT_DOWNLOAD_ATTEMPTS**    | Number of attempts to download artifacts running a job |
-| **EXECUTOR_JOB_SECTION_ATTEMPTS** | [In GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) and later, the number of attempts to run a section in a job after a [`No Such Container`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) error ([Docker executor](https://docs.gitlab.com/runner/executors/docker.html) only). |
-| **GET_SOURCES_ATTEMPTS**          | Number of attempts to fetch sources running a job      |
-| **RESTORE_CACHE_ATTEMPTS**        | Number of attempts to restore the cache running a job  |
-
-The default is one single attempt.
-
-Example:
-
-```yaml
-variables:
-  GET_SOURCES_ATTEMPTS: 3
-```
-
-You can set them globally or per-job in the [`variables`](#variables) section.
-
-### Fallback cache key
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1534) in GitLab Runner 13.4.
-
-You can use the `$CI_COMMIT_REF_SLUG` variable to specify your [`cache:key`](#cachekey).
-For example, if your `$CI_COMMIT_REF_SLUG` is `test` you can set a job
-to download cache that's tagged with `test`.
-
-If a cache with this tag is not found, you can use `CACHE_FALLBACK_KEY` to
-specify a cache to use when none exists.
-
-For example:
-
-```yaml
-variables:
-  CACHE_FALLBACK_KEY: fallback-key
-
-cache:
-  key: "$CI_COMMIT_REF_SLUG"
-  paths:
-    - binaries/
-```
-
-In this example, if the `$CI_COMMIT_REF_SLUG` is not found, the job uses the key defined
-by the `CACHE_FALLBACK_KEY` variable.
-
-### Shallow cloning
-
-> Introduced in GitLab 8.9 as an experimental feature.
-
-You can specify the depth of fetching and cloning using `GIT_DEPTH`.
-`GIT_DEPTH` does a shallow clone of the repository and can significantly speed up cloning.
-It can be helpful for repositories with a large number of commits or old, large binaries. The value is
-passed to `git fetch` and `git clone`.
-
-In GitLab 12.0 and later, newly-created projects automatically have a
-[default `git depth` value of `50`](../pipelines/settings.md#git-shallow-clone).
-
-If you use a depth of `1` and have a queue of jobs or retry
-jobs, jobs may fail.
-
-Git fetching and cloning is based on a ref, such as a branch name, so runners
-can't clone a specific commit SHA. If multiple jobs are in the queue, or
-you're retrying an old job, the commit to be tested must be within the
-Git history that is cloned. Setting too small a value for `GIT_DEPTH` can make
-it impossible to run these old commits and `unresolved reference` is displayed in
-job logs. You should then reconsider changing `GIT_DEPTH` to a higher value.
-
-Jobs that rely on `git describe` may not work correctly when `GIT_DEPTH` is
-set since only part of the Git history is present.
-
-To fetch or clone only the last 3 commits:
-
-```yaml
-variables:
-  GIT_DEPTH: "3"
-```
-
-You can set it globally or per-job in the [`variables`](#variables) section.
-
-### Custom build directories
-
-> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2211) in GitLab Runner 11.10.
-
-By default, GitLab Runner clones the repository in a unique subpath of the
-`$CI_BUILDS_DIR` directory. However, your project might require the code in a
-specific directory (Go projects, for example). In that case, you can specify
-the `GIT_CLONE_PATH` variable to tell the runner the directory to clone the
-repository in:
-
-```yaml
-variables:
-  GIT_CLONE_PATH: $CI_BUILDS_DIR/project-name
-
-test:
-  script:
-    - pwd
-```
-
-The `GIT_CLONE_PATH` has to always be within `$CI_BUILDS_DIR`. The directory set in `$CI_BUILDS_DIR`
-is dependent on executor and configuration of [runners.builds_dir](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section)
-setting.
-
-This can only be used when `custom_build_dir` is enabled in the
-[runner's configuration](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscustom_build_dir-section).
-This is the default configuration for the `docker` and `kubernetes` executors.
-
-#### Handling concurrency
-
-An executor that uses a concurrency greater than `1` might lead
-to failures. Multiple jobs might be working on the same directory if the `builds_dir`
-is shared between jobs.
-
-The runner does not try to prevent this situation. It's up to the administrator
-and developers to comply with the requirements of runner configuration.
-
-To avoid this scenario, you can use a unique path within `$CI_BUILDS_DIR`, because runner
-exposes two additional variables that provide a unique `ID` of concurrency:
-
-- `$CI_CONCURRENT_ID`: Unique ID for all jobs running within the given executor.
-- `$CI_CONCURRENT_PROJECT_ID`: Unique ID for all jobs running within the given executor and project.
-
-The most stable configuration that should work well in any scenario and on any executor
-is to use `$CI_CONCURRENT_ID` in the `GIT_CLONE_PATH`. For example:
-
-```yaml
-variables:
-  GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_CONCURRENT_ID/project-name
-
-test:
-  script:
-    - pwd
-```
-
-The `$CI_CONCURRENT_PROJECT_ID` should be used in conjunction with `$CI_PROJECT_PATH`
-as the `$CI_PROJECT_PATH` provides a path of a repository. That is, `group/subgroup/project`. For example:
-
-```yaml
-variables:
-  GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_CONCURRENT_ID/$CI_PROJECT_PATH
-
-test:
-  script:
-    - pwd
-```
-
-#### Nested paths
-
-The value of `GIT_CLONE_PATH` is expanded once and nesting variables
-within is not supported.
-
-For example, you define both the variables below in your
-`.gitlab-ci.yml` file:
-
-```yaml
-variables:
-  GOPATH: $CI_BUILDS_DIR/go
-  GIT_CLONE_PATH: $GOPATH/src/namespace/project
-```
+- [`GIT_STRATEGY`](../runners/README.md#git-strategy)
+- [`GIT_SUBMODULE_STRATEGY`](../runners/README.md#git-submodule-strategy)
+- [`GIT_CHECKOUT`](../runners/README.md#git-checkout)
+- [`GIT_CLEAN_FLAGS`](../runners/README.md#git-clean-flags)
+- [`GIT_FETCH_EXTRA_FLAGS`](../runners/README.md#git-fetch-extra-flags)
+- [`GIT_DEPTH`](../runners/README.md#shallow-cloning) (shallow cloning)
+- [`GIT_CLONE_PATH`](../runners/README.md#custom-build-directories) (custom build directories)
 
-The value of `GIT_CLONE_PATH` is expanded once into
-`$CI_BUILDS_DIR/go/src/namespace/project`, and results in failure
-because `$CI_BUILDS_DIR` is not expanded.
+You can also use variables to configure how many times a runner
+[attempts certain stages of job execution](../runners/README.md#job-stages-attempts).
 
 ## Special YAML features