Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/doc/ci
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2023-05-10 19:46:51 +0300
committerGitLab Bot <gitlab-bot@gitlab.com>2023-05-10 19:46:51 +0300
commit04ac4180cb5ba0df460034c7e64862056fd498b3 (patch)
treeca5267d12fa2aab80c1c35593b96c8cfceaa747b /doc/ci
parent4e9ceea7f8ff3e097ad45f6f54c1b7165248e848 (diff)
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/ci')
-rw-r--r--doc/ci/cloud_services/aws/index.md4
-rw-r--r--doc/ci/cloud_services/azure/index.md4
-rw-r--r--doc/ci/cloud_services/google_cloud/index.md4
-rw-r--r--doc/ci/cloud_services/index.md12
-rw-r--r--doc/ci/components/index.md201
-rw-r--r--doc/ci/examples/authenticating-with-hashicorp-vault/index.md10
-rw-r--r--doc/ci/secrets/id_token_authentication.md36
-rw-r--r--doc/ci/variables/predefined_variables.md6
8 files changed, 244 insertions, 33 deletions
diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md
index 733105e9ddf..e3655e67c7f 100644
--- a/doc/ci/cloud_services/aws/index.md
+++ b/doc/ci/cloud_services/aws/index.md
@@ -6,6 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Configure OpenID Connect in AWS to retrieve temporary credentials **(FREE)**
+WARNING:
+`CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
+and is scheduled to be removed in GitLab 16.5. Use [ID tokens](../../yaml/index.md#id_tokens) instead.
+
In this tutorial, we'll show you how to use a GitLab CI/CD job with a JSON web token (JWT) to retrieve temporary credentials from AWS without needing to store secrets.
To do this, you must configure OpenID Connect (OIDC) for ID federation between GitLab and AWS. For background and requirements for integrating GitLab using OIDC, see [Connect to cloud services](../index.md).
diff --git a/doc/ci/cloud_services/azure/index.md b/doc/ci/cloud_services/azure/index.md
index c99e30e8bac..912e6597985 100644
--- a/doc/ci/cloud_services/azure/index.md
+++ b/doc/ci/cloud_services/azure/index.md
@@ -6,6 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Configure OpenID Connect in Azure to retrieve temporary credentials **(FREE)**
+WARNING:
+`CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
+and is scheduled to be removed in GitLab 16.5. Use [ID tokens](../../yaml/index.md#id_tokens) instead.
+
This tutorial demonstrates how to use a JSON web token (JWT) in a GitLab CI/CD job
to retrieve temporary credentials from Azure without needing to store secrets.
diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md
index 352b56cd8c6..5ed22883518 100644
--- a/doc/ci/cloud_services/google_cloud/index.md
+++ b/doc/ci/cloud_services/google_cloud/index.md
@@ -6,9 +6,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w
# Configure OpenID Connect with GCP Workload Identity Federation **(FREE)**
-NOTE:
+WARNING:
`CI_JOB_JWT_V2` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
-and is scheduled to be removed in GitLab 16.0. Use [ID tokens](../../yaml/index.md#id_tokens) instead.
+and is scheduled to be removed in GitLab 16.5. Use [ID tokens](../../yaml/index.md#id_tokens) instead.
This tutorial demonstrates authenticating to Google Cloud from a GitLab CI/CD job
using a JSON Web Token (JWT) token and Workload Identity Federation. This configuration
diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md
index 500a3bb702d..54cadc9e1b6 100644
--- a/doc/ci/cloud_services/index.md
+++ b/doc/ci/cloud_services/index.md
@@ -10,6 +10,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w
> - `CI_JOB_JWT_V2` variable to support additional OIDC providers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346737) in GitLab 14.7.
> - [ID tokens](../yaml/index.md#id_tokens) to support any OIDC provider, including HashiCorp Vault, [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356986) in GitLab 15.7.
+WARNING:
+`CI_JOB_JWT` and `CI_JOB_JWT_V2` were [deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
+and are scheduled to be removed in GitLab 16.5. Use [ID tokens](../yaml/index.md#id_tokens) instead.
+
GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) to
give your build and deployment jobs access to cloud credentials and services.
Historically, teams stored secrets in projects or applied permissions on the GitLab Runner
@@ -19,24 +23,18 @@ in the CI/CD job allowing you to follow a scalable and least-privilege security
In GitLab 15.6 and earlier, you must use `CI_JOB_JWT_V2` instead of an ID token,
but it is not customizable. In GitLab 14.6 an earlier you must use the `CI_JOB_JWT`, which has limited support.
-NOTE:
-`CI_JOB_JWT` and `CI_JOB_JWT_V2` were [deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
-and are scheduled to be removed in GitLab 16.5. Use [ID tokens](../yaml/index.md#id_tokens) instead.
-
## Prerequisites
- Account on GitLab.
- Access to a cloud provider that supports OIDC to configure authorization and create roles.
-ID tokens and `CI_JOB_JWT_V2` support cloud providers with OIDC, including:
+ID tokens support cloud providers with OIDC, including:
- AWS
- Azure
- GCP
- HashiCorp Vault
-The `CI_JOB_JWT` only supports the [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/index.md).
-
NOTE:
Configuring OIDC enables JWT token access to the target environments for all pipelines.
When you configure OIDC for a pipeline, you should complete a software supply chain security
diff --git a/doc/ci/components/index.md b/doc/ci/components/index.md
new file mode 100644
index 00000000000..82040d5990c
--- /dev/null
+++ b/doc/ci/components/index.md
@@ -0,0 +1,201 @@
+---
+stage: Verify
+group: Pipeline Authoring
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
+type: reference
+---
+
+# CI/CD Components (Experimental)
+
+> Introduced in GitLab 16.0 as an [experimental feature](../../policy/alpha-beta-support.md).
+
+FLAG:
+On self-managed GitLab, by default this feature is not available.
+To make it available, ask an administrator to enable the feature flag named `ci_namespace_catalog_experimental`.
+On GitLab.com, this feature is not available. This feature is not ready for production use.
+
+This feature is an experimental feature and [an epic exists](https://gitlab.com/groups/gitlab-org/-/epics/9897)
+to track future work. Tell us about your use case by leaving comments in the epic.
+
+## Components Repository
+
+A components repository is a GitLab project with a repository that hosts one or more pipeline components. A pipeline component is a reusable single pipeline configuration unit. You can use them to compose an entire pipeline configuration or a small part of a larger pipeline. It can optionally take [input parameters](../yaml/includes.md#define-input-parameters-with-specinputs).
+
+### Create a components repository
+
+To create a components repository, you must:
+
+1. [Create a new project](../../user/project/index.md#create-a-blank-project) with a `README.md` file.
+
+1. Create a `template.yml` file inside the project's root directory that contains the configuration you want to provide as a component. For example:
+
+ ```yaml
+ spec:
+ inputs:
+ stage:
+ default: test
+ ---
+ component-job:
+ script: echo job 1
+ stage: $[[ inputs.stage ]]
+ ```
+
+### Directory structure
+
+A components repository can host one or more components.
+
+Components repositories must follow a mandatory file structure, containing:
+
+- `template.yml`: The component configuration, one file per component. If there is
+ only one component, this file can be in the root of the project. If there are multiple
+ components, each file must be in a separate subdirectory.
+- `README.md`: A documentation file explaining the details of the all the components in the repository.
+
+For example, if the project is on GitLab.com, named `my-component`, and in a personal
+namespace named `my-username`:
+
+- Containing a single component and a simple pipeline to test the component, then
+ the file structure might be:
+
+ ```plaintext
+ ├── template.yml
+ ├── README.md
+ └── .gitlab-ci.yml
+ ```
+
+ This component is referenced with the path `gitlab.com/my-username/my-component@<version>`.
+
+- Containing one default component and multiple sub-components, then the file structure
+ might be:
+
+ ```plaintext
+ ├── template.yml
+ ├── README.md
+ ├── .gitlab-ci.yml
+ ├── unit/
+ │ └── template.yml
+ └── integration/
+ └── template.yml
+ ```
+
+ These components are identified by these paths:
+
+ - `gitlab.com/my-username/my-component`
+ - `gitlab.com/my-username/my-component/unit`
+ - `gitlab.com/my-username/my-component/integration`
+
+It is possible to have a components repository with no default component, by having
+no `template.yml` in the root directory.
+
+**Additional notes:**
+
+Nesting of components is not possible. For example:
+
+```plaintext
+├── unit/
+│ └── template.yml
+│ └── another_folder/
+│ └── nested_template.yml
+```
+
+### Test a component
+
+Testing components as part of the development workflow to ensure that quality maintains high standards is strongly recommended.
+
+Testing changes in a CI/CD pipeline can be done, like any other project, by creating a `.gitlab-ci.yml` in the root directory.
+
+For example:
+
+```yaml
+include:
+ # include the component located in the current project from the current SHA
+ - component: gitlab.com/$CI_PROJECT_PATH@$CI_COMMIT_SHA
+ inputs:
+ stage: build
+
+stages: [build, test, release]
+
+# Expect `component-job` is added.
+# This is an example of testing that the included component works as expected.
+# You can leverage GitLab API endpoints or 3rd party tools to inspect data generated by the component.
+ensure-job-added:
+ stage: test
+ image: badouralix/curl-jq
+ script:
+ - |
+ route="https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs"
+ count=`curl --silent --header "PRIVATE-TOKEN: $API_TOKEN" $route | jq 'map(select(.name | contains("component-job"))) | length'`
+ if [ "$count" != "1" ]; then
+ exit 1
+ fi
+
+# If we are tagging a release with a specific convention ("v" + number) and all
+# previous checks succeeded, we proceed with creating a release automatically.
+create-release:
+ stage: release
+ image: registry.gitlab.com/gitlab-org/release-cli:latest
+ rules:
+ - if: $CI_COMMIT_TAG =~ /^v\d+/
+ script: echo "Creating release $CI_COMMIT_TAG"
+ release:
+ tag_name: $CI_COMMIT_TAG
+ description: "Release $CI_COMMIT_TAG of components repository $CI_PROJECT_PATH"
+```
+
+After committing and pushing changes, the pipeline tests the component then releases it if the test passes.
+
+### Release a component
+
+Component repositories are released using the [`release`](../yaml/index.md#release) keyword within a CI pipeline.
+
+Like in the [example above](#test-a-component), after all tests pass in a pipeline running for a tag ref, we can release a new version of the components repository.
+
+All released versions of the components repository are displayed in the Components Catalog page for the given resource, providing users with information about official releases.
+
+### Use a component in a CI/CD configuration
+
+A pipeline component is identified by a unique address in the form `<fully-qualified-doman-name>/<component-path>@<version>`
+containing:
+
+- **A fully qualified domain name (FQDN)**: The FQDN must match the GitLab host.
+- **A specific version**: The version of the component can be (in order of highest priority first):
+ - A commit SHA, for example `gitlab.com/gitlab-org/dast@e3262fdd0914fa823210cdb79a8c421e2cef79d8`.
+ - A tag. for example: `gitlab.com/gitlab-org/dast@1.0`.
+ - `~latest`, which is a special version that always points to the most recent released tag,
+ for example `gitlab.com/gitlab-org/dast@~latest`.
+ - A branch name, for example `gitlab.com/gitlab-org/dast@main`.
+- **A component path**: Contains the project's full path and the directory where the component YAML file `template.yml` is located.
+
+For example, for a component repository located at `gitlab-org/dast` on `gitlab.com`:
+
+- The path `gitlab.com/gitlab-org/dast` tries to load the `template.yml` from the root directory.
+- The path `gitalb.com/gitlab-org/dast/api-scan` tries to load the `template.yml` from the `/api-scan` directory.
+
+**Additional notes:**
+
+- If a tag and branch exist with the same name, the tag takes precedence over the branch.
+- If a tag is named the same as a commit SHA that exists, like `e3262fdd0914fa823210cdb79a8c421e2cef79d8`,
+ the commit SHA takes precedence over the tag.
+
+## Components catalog
+
+The CI/CD Catalog is a list of [components repositories](#components-repository),
+each containing resources that you can add to your CI/CD pipelines.
+
+### Mark the project as a catalog resource
+
+After components are added to a components repository, they can immediately be [used](#use-a-component-in-a-cicd-configuration) to build pipelines in other projects.
+
+However, this repository is not discoverable. You must mark this project as a catalog resource to allow it to be visible in the CI Catalog
+so other users can discover it.
+
+To mark a project as a catalog resource, run the following [graphQL](../../api/graphql/index.md)
+mutation:
+
+```graphql
+mutation {
+ catalogResourcesCreate(input: { projectPath: "path-to-project"}) {
+ errors
+ }
+}
+```
diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
index a9922920c05..a5289ca0a4d 100644
--- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
+++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md
@@ -7,11 +7,13 @@ type: tutorial
# Authenticating and reading secrets with HashiCorp Vault (Deprecated) **(PREMIUM)**
-This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD.
-
-NOTE:
+WARNING:
Authenticating with HashiCorp Vault by using `CI_JOB_JWT` was [deprecated in GitLab 15.9](../../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated)
-and the token is scheduled to be removed in GitLab 16.0. Use [ID tokens to authenticate with HashiCorp Vault](../../secrets/id_token_authentication.md#automatic-id-token-authentication-with-hashicorp-vault) instead.
+and the token is scheduled to be removed in GitLab 16.5. This change is a breaking change.
+Use [ID tokens to authenticate with HashiCorp Vault](../../secrets/id_token_authentication.md#automatic-id-token-authentication-with-hashicorp-vault)
+instead.
+
+This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD.
## Prerequisites
diff --git a/doc/ci/secrets/id_token_authentication.md b/doc/ci/secrets/id_token_authentication.md
index 6c02c29628e..1ff2a6efbcf 100644
--- a/doc/ci/secrets/id_token_authentication.md
+++ b/doc/ci/secrets/id_token_authentication.md
@@ -7,6 +7,8 @@ type: tutorial
# OpenID Connect (OIDC) Authentication Using ID Tokens **(FREE)**
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/356986) in GitLab 15.7.
+
You can authenticate with third party services using GitLab CI/CD's
[ID tokens](../yaml/index.md#id_tokens).
@@ -130,23 +132,6 @@ manual_authentication:
You can use ID tokens to automatically fetch secrets from HashiCorp Vault with the
[`secrets`](../yaml/index.md#secrets) keyword.
-<!--- start_remove The following content will be removed on remove_date: '2023-08-22' -->
-
-### Enable automatic ID token authentication (deprecated)
-
-WARNING:
-This setting was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/391886) in GitLab 16.0.
-ID token authentication is now always available, and JSON Web Token access is always limited.
-
-To enable automatic ID token authentication:
-
-1. On the top bar, select **Main menu > Projects** and find your project.
-1. On the left sidebar, select **Settings > CI/CD**.
-1. Expand **Token Access**.
-1. Toggle **Limit JSON Web Token (JWT) access** to enabled.
-
-<!--- end_remove -->
-
### Configure automatic ID Token authentication
If one ID token is defined, the `secrets` keyword automatically uses it to authenticate with Vault. For example:
@@ -183,3 +168,20 @@ job_with_secrets:
- access-first-db.sh --token $FIRST_DB_PASSWORD
- access-second-db.sh --token $SECOND_DB_PASSWORD
```
+
+<!--- start_remove The following content will be removed on remove_date: '2023-08-22' -->
+
+### Enable automatic ID token authentication (deprecated)
+
+WARNING:
+This setting was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/391886) in GitLab 16.0.
+ID token authentication is now always available, and JSON Web Token access is always limited.
+
+To enable automatic ID token authentication:
+
+1. On the top bar, select **Main menu > Projects** and find your project.
+1. On the left sidebar, select **Settings > CI/CD**.
+1. Expand **Token Access**.
+1. Toggle **Limit JSON Web Token (JWT) access** to enabled.
+
+<!--- end_remove -->
diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md
index 3de36479de7..001a599776a 100644
--- a/doc/ci/variables/predefined_variables.md
+++ b/doc/ci/variables/predefined_variables.md
@@ -68,9 +68,9 @@ as it can cause the pipeline to behave unexpectedly.
| `CI_HAS_OPEN_REQUIREMENTS` | 13.1 | all | Only available if the pipeline's project has an open [requirement](../../user/project/requirements/index.md). `true` when available. |
| `CI_JOB_ID` | 9.0 | all | The internal ID of the job, unique across all jobs in the GitLab instance. |
| `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. |
-| `CI_JOB_JWT` (Deprecated) | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
-| `CI_JOB_JWT_V1` (Deprecated) | 14.6 | all | The same value as `CI_JOB_JWT`. [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
-| `CI_JOB_JWT_V2` (Deprecated) | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. The `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.0. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
+| `CI_JOB_JWT` (Deprecated) | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.5. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
+| `CI_JOB_JWT_V1` (Deprecated) | 14.6 | all | The same value as `CI_JOB_JWT`. [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.5. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
+| `CI_JOB_JWT_V2` (Deprecated) | 14.6 | all | A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. The `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). [Deprecated in GitLab 15.9](../../update/deprecations.md#old-versions-of-json-web-tokens-are-deprecated) and scheduled to be removed in GitLab 16.5. Use [ID tokens](../yaml/index.md#id_tokens) instead. |
| `CI_JOB_MANUAL` | 8.12 | all | Only available if the job was started manually. `true` when available. |
| `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. |
| `CI_JOB_NAME_SLUG` | 15.4 | all | `CI_JOB_NAME_SLUG` in lowercase, shortened to 63 bytes, and with everything except `0-9` and `a-z` replaced with `-`. No leading / trailing `-`. Use in paths. |