diff options
Diffstat (limited to 'doc/ci')
58 files changed, 787 insertions, 373 deletions
diff --git a/doc/ci/chatops/index.md b/doc/ci/chatops/index.md index 698b467a813..c785e2f29ea 100644 --- a/doc/ci/chatops/index.md +++ b/doc/ci/chatops/index.md @@ -43,7 +43,7 @@ job completes: [Slack API](https://api.slack.com/) to send data to the channel. To use the `run` command, you must have at least the -[Developer role](../../user/permissions.md#project-members-permissions). +Developer role. If a job shouldn't be able to be triggered from chat, you can set the job to `except: [chat]`. ## Best practices for ChatOps CI jobs @@ -55,7 +55,7 @@ functions available. Consider these best practices when creating ChatOps jobs: of the standard CI pipeline. - If the job is set to `when: manual`, ChatOps creates the pipeline, but the job waits to be started. - ChatOps provides limited support for access control. If the user triggering the - slash command has at least the [Developer role](../../user/permissions.md#project-members-permissions) + slash command has at least the Developer role in the project, the job runs. The job itself can use existing [CI/CD variables](../variables/index.md#predefined-cicd-variables) like `GITLAB_USER_ID` to perform additional rights validation, but diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 4e4b7420697..a906f213626 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -16,7 +16,7 @@ Watch a video on [Using GitLab CI/CD pipelines with GitHub repositories](https:/ NOTE: Because of [GitHub limitations](https://gitlab.com/gitlab-org/gitlab/-/issues/9147), -[GitHub OAuth](../../integration/github.md#enabling-github-oauth) +[GitHub OAuth](../../integration/github.md#enable-github-oauth-in-gitlab) cannot be used to authenticate with GitHub as an external CI/CD repository. ## Connect with Personal Access Token diff --git a/doc/ci/cloud_services/aws/index.md b/doc/ci/cloud_services/aws/index.md index f09b23db2c5..aa38562c866 100644 --- a/doc/ci/cloud_services/aws/index.md +++ b/doc/ci/cloud_services/aws/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Verify +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -30,7 +30,7 @@ Include the following information: After you create the identity provider, configure a [web identity role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-idp_oidc.html) with conditions for limiting access to GitLab resources. Temporary credentials are obtained using [AWS Security Token Service](https://docs.aws.amazon.com/STS/latest/APIReference/welcome.html), so set the `Action` to [sts:AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRoleWithWebIdentity.html). -For the full list of supported filtering types, see [Connect to cloud services](../index.md). +For the full list of supported filtering types, see [Connect to cloud services](../index.md#configure-a-conditional-role-with-oidc-claims). ```json { diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md new file mode 100644 index 00000000000..14928f91816 --- /dev/null +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -0,0 +1,182 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Configure OpenID Connect with GCP Workload Identity Federation + +WARNING: +The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. + +This tutorial demonstrates authenticating to Google Cloud from a GitLab CI/CD job +using a JSON Web Token (JWT) token and Workload Identity Federation. This configuration +generates on-demand, short-lived credentials without needing to store any secrets. + +To get started, configure OpenID Connect (OIDC) for identity federation between GitLab +and Google Cloud. For more information on using OIDC with GitLab, read +[Connect to cloud services](../index.md). + +This tutorial assumes you have a Google Cloud account and a Google Cloud project. +Your account must have at least the **Workload Identity Pool Admin** permission +on the Google Cloud project. + +To complete this tutorial: + +1. [Create the Google Cloud Workload Identity Pool](#create-the-google-cloud-workload-identity-pool). +1. [Create a Workload Identity Provider](#create-a-workload-identity-provider). +1. [Grant permissions for service account impersonation](#grant-permissions-for-service-account-impersonation). +1. [Retrieve a temporary credential](#retrieve-a-temporary-credential). + +## Create the Google Cloud Workload Identity Pool + +[Create a new Google Cloud Workload Identity Pool](https://cloud.google.com/iam/docs/configuring-workload-identity-federation#oidc) with the following options: + +- **Name**: Human-friendly name for the Workload Identity Pool, such as `GitLab`. +- **Pool ID**: Unique ID in the Google Cloud project for the Workload Identity Pool, + such as `gitlab`. This value is used to refer to the pool. and appears in URLs. +- **Description**: Optional. A description of the pool. +- **Enabled Pool**: Ensure this option is `true`. + +We recommend creating a single _pool_ per GitLab installation per Google Cloud project. If you have multiple GitLab repositories and CI/CD jobs on the same GitLab instance, they can authenticate using different _providers_ against the same _pool_. + +## Create a Workload Identity Provider + +[Create a new Google Cloud Workload Identity Provider](https://cloud.google.com/iam/docs/configuring-workload-identity-federation#create_the_workload_identity_pool_and_provider) +inside the Workload Identity Pool created in the previous step, using the following options: + +- **Provider type**: OpenID Connect (OIDC). +- **Provider name**: Human-friendly name for the Workload Identity Provider, + such as `gitlab/gitlab`. +- **Provider ID**: Unique ID in the pool for the Workload Identity Provider, + such as `gitlab-gitlab`. This value is used to refer to the provider, and appears in URLs. +- **Issuer (URL)**: The address of your GitLab instance, such as `https://gitlab.com` or + `https://gitlab.example.com`. + - The address must use the `https://` protocol. + - The address must end in a trailing slash. +- **Audiences**: Manually set the allowed audiences list to the address of your + GitLab instance, such as `https://gitlab.com` or `https://gitlab.example.com`. + - The address must use the `https://` protocol. + - The address must not end in a trailing slash. +- **Provider attributes mapping**: Create the following mappings, where `attribute.X` is the + name of the attribute you would like to be present on Google's claims, and `assertion.X` + is the value to extract from the [GitLab claim](../index.md#how-it-works): + + | Attribute (on Google) | Assertion (from GitLab) | + | --- | --- | + | `google.subject` | `assertion.sub` | + | `attribute.X` | `assertion.X` | + + You can also [build complex attributes](https://cloud.google.com/iam/help/workload-identity/attribute-mapping) + using Common Expression Language (CEL). + + You must map every attribute that you want to use for permission granting. For example, if you want to map permissions in the next step based on the user's email address, you must map `attribute.user_email` to `assertion.user_email`. + +## Grant permissions for Service Account impersonation + +Creating the Workload Identity Pool and Workload Identity Provider defines the _authentication_ +into Google Cloud. At this point, you can authenticate from GitLab CI/CD job into Google Cloud. +However, you have no permissions on Google Cloud (_authorization_). + +To grant your GitLab CI/CD job permissions on Google Cloud, you must: + +1. [Create a Google Cloud Service Account](https://www.google.com/search?q=google+cloud+create+service+account). + You can use whatever name and ID you prefer. +1. [Grant IAM permissions](https://cloud.google.com/iam/docs/granting-changing-revoking-access) to your + service account on Google Cloud resources. These permissions vary significantly based on + your use case. In general, grant this service account the permissions on your Google Cloud + project and resources you want your GitLab CI/CD job to be able to use. For example, if you needed to upload a file to a Google Cloud Storage bucket in your GitLab CI/CD job, you would grant this Service Account the `roles/storage.objectCreator` role on your Cloud Storage bucket. +1. [Grant the external identity permissions](https://cloud.google.com/iam/docs/using-workload-identity-federation#impersonate) + to impersonate that Service Account. This step enables a GitLab CI/CD job to _authorize_ + to Google Cloud, via Service Account impersonation. This step grants an IAM permission + _on the Service Account itself_, giving the external identity permissions to act as that + service account. External identities are expressed using the `principalSet://` protocol. + +Much like the previous step, this step depends heavily on your desired configuration. +For example, to allow a GitLab CI/CD job to impersonate a Service Account named +`my-service-account` if the GitLab CI/CD job was initiated by a GitLab user with the +username `chris`, you would grant the `roles/iam.workloadIdentityUser` IAM role to the +external identity on `my-service-account`. The external identity takes the format: + +```plaintext +principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/attribute.user_login/chris +``` + +where `PROJECT_NUMBER` is your Google Cloud project number, and `POOL_ID` is the +ID (not name) of the Workload Identity Pool created in the first section. + +This configuration also assumes you added `user_login` as an attribute mapped from +the assertion in the previous section. + +## Retrieve a temporary credential + +After you configure the OIDC and role, the GitLab CI/CD job can retrieve a temporary credential from the +[Google Cloud Security Token Service (STS)](https://cloud.google.com/iam/docs/reference/sts/rest). + +```shell +PAYLOAD="$(cat <<EOF +{ + "audience": "//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID", + "grantType": "urn:ietf:params:oauth:grant-type:token-exchange", + "requestedTokenType": "urn:ietf:params:oauth:token-type:access_token", + "scope": "https://www.googleapis.com/auth/cloud-platform", + "subjectTokenType": "urn:ietf:params:oauth:token-type:jwt", + "subjectToken": "${CI_JOB_JWT_V2}" +} +EOF +)" +``` + +```shell +FEDERATED_TOKEN="$(curl --fail "https://sts.googleapis.com/v1/token" \ + --header "Accept: application/json" \ + --header "Content-Type: application/json" \ + --data "${PAYLOAD}" \ + | jq -r '.access_token' +)" +``` + +Where: + +- `PROJECT_NUMBER` is your Google Cloud project number (not name). +- `POOL_ID` is the ID of the Workload Identity Pool created in the first section. +- `PROVIDER_ID` is the ID of the Workload Identity Provider created in the second section. +- `CI_JOB_JWT_V2` is injected into the CI/CD job by GitLab. For more information about + this variable, read [Connect to cloud services](../index.md). + +You can then use the resulting federated token to impersonate the service account created +in the previous section: + +```shell +ACCESS_TOKEN="$(curl --fail "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SERVICE_ACCOUNT_EMAIL:generateAccessToken" \ + --header "Accept: application/json" \ + --header "Content-Type: application/json" \ + --header "Authorization: Bearer FEDERATED_TOKEN" \ + --data '{"scope": ["https://www.googleapis.com/auth/cloud-platform"]}' \ + | jq -r '.accessToken' +)" +``` + +Where: + +- `SERVICE_ACCOUNT_EMAIL` is the full email address of the service account to impersonate, + created in the previous section. +- `FEDERATED_TOKEN` is the federated token retrieved from the previous step. + +The result is a Google Cloud OAuth 2.0 access token, which you can use to authenticate to +most Google Cloud APIs and services when used as a bearer token. You can also pass this +value to the `gcloud` CLI by setting the environment variable `CLOUDSDK_AUTH_ACCESS_TOKEN`. + +## Working example + +Review this +[reference project](https://gitlab.com/guided-explorations/gcp/configure-openid-connect-in-gcp) +for provisioning OIDC in GCP using Terraform and a sample script to retrieve temporary credentials. + +## Troubleshooting + +- When debugging `curl` responses, install the latest version of curl. Use `--fail-with-body` + instead of `-f`. This command prints the entire body, which can contain helpful error messages. + +- Review Google Cloud's documentation for + [Troubleshooting Workload Identity Federation](https://cloud.google.com/iam/docs/troubleshooting-workload-identity-federation). diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index 73e726ea8a9..a80231a04c2 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Verify +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -19,7 +19,7 @@ GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) t The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. WARNING: -The `CI_JOB_JWT_V2` variable is under development [(alpha)](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha) and is not yet suitable for production use. +The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. ## Use cases @@ -99,9 +99,9 @@ sequenceDiagram Note right of Cloud: Decode & verify JWT with public key (https://gitlab/-/jwks) Note right of Cloud: Validate audience defined in OIDC Note right of Cloud: Validate conditional (sub, aud) role - Note right of Cloud: Generate credential or fetch secret + Note right of Cloud: Generate credential or fetch secret Cloud->>GitLab: Return temporary credential - Note left of GitLab: Perform operation + Note left of GitLab: Perform operation ``` @@ -131,3 +131,4 @@ To configure the trust between GitLab and OIDC, you must create a conditional ro To connect with your cloud provider, see the following tutorials: - [Configure OpenID Connect in AWS](aws/index.md) +- [Configure OpenID Connect in Google Cloud](google_cloud/index.md) diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 5bd9293924d..7edff334134 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -450,3 +450,26 @@ To configure access for `<aws_account_id>.dkr.ecr.<region>.amazonaws.com`, follo You can add configuration for as many registries as you want, adding more registries to the `"credHelpers"` hash. + +### Use checksum to keep your image secure + +We recommend using the image checksum in your job definition in your `.gitlab-ci.yml` file to verify the integrity of the image. A failed image integrity verification will prevent you from using a modified container. + +To use the image checksum you have to append the checksum at the end: + +```yaml +image: ruby:2.6.8@sha256:d1dbaf9665fe8b2175198e49438092fdbcf4d8934200942b94425301b17853c7 +``` + +To get the image checksum, on the image `TAG` tab, view the `DIGEST` column. +For example, view the [Ruby image](https://hub.docker.com/_/ruby?tab=tags). +The checksum is a random string, like `6155f0235e95`. + +You can also get the checksum of any image on your system with the command `docker images --digests`: + +```shell +❯ docker images --digests +REPOSITORY TAG DIGEST (...) +gitlab/gitlab-ee latest sha256:723aa6edd8f122d50cae490b1743a616d54d4a910db892314d68470cc39dfb24 (...) +gitlab/gitlab-runner latest sha256:4a18a80f5be5df44cb7575f6b89d1fdda343297c6fd666c015c0e778b276e726 (...) +``` diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index d60e5704877..e320118c191 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -7,7 +7,8 @@ description: Require approvals prior to deploying to a Protected Environment # Deployment approvals **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343864) in GitLab 14.7 with a flag named `deployment_approvals`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/343864) in GitLab 14.7 with a flag named `deployment_approvals`. Disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/347342) in GitLab 14.8. WARNING: This feature is in an alpha stage and subject to change without prior notice. @@ -49,12 +50,19 @@ Example: name: ${CI_JOB_NAME} ``` -### Require approvals for a protected environment +### Require approvals for a protected environment NOTE: -At this time, only API-based configuration is available. UI-based configuration is planned for the near future. See [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/344675). +At this time, it is not possible to require approvals for an existing protected environment. The workaround is to unprotect the environment and configure approvals when re-protecting the environment. -Use the [Protected Environments API](../../api/protected_environments.md#protect-repository-environments) to create an environment with `required_approval_count` > 0. After this is set, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. +There are two ways to configure approvals for a protected environment: + +1. Using the [UI](protected_environments.md#protecting-environments) + 1. Set the **Required approvals** field to 1 or more. +1. Using the [REST API](../../api/protected_environments.md#protect-repository-environments) + 2. Set the `required_approval_count` field to 1 or more. + +After this is configured, all jobs deploying to this environment automatically go into a blocked state and wait for approvals before running. Ensure that the number of required approvals is less than the number of users allowed to deploy. Example: @@ -65,8 +73,9 @@ curl --header 'Content-Type: application/json' --request POST \ "https://gitlab.example.com/api/v4/projects/22034114/protected_environments" ``` +NOTE: To protect, update, or unprotect an environment, you must have at least the -[Maintainer role](../../user/permissions.md). +Maintainer role. ## Approve or reject a deployment @@ -75,7 +84,7 @@ This functionality is currently only available through the API. UI is planned fo A blocked deployment is enqueued as soon as it receives the required number of approvals. A single rejection causes the deployment to fail. The creator of a deployment cannot approve it, even if they have permission to deploy. -Using the [Deployments API](../../api/deployments.md#approve-or-reject-a-blocked-deployment), users who are allowed to deploy to the protected environment can approve or reject a blocked deployment. +Using the [Deployments API](../../api/deployments.md#approve-or-reject-a-blocked-deployment), users who are allowed to deploy to the protected environment can approve or reject a blocked deployment. Example: @@ -95,7 +104,11 @@ curl --data "status=approved" \ #### Using the API -Use the [Deployments API](../../api/deployments.md) to see deployments. The `status` field indicates if a deployment is blocked. +Use the [Deployments API](../../api/deployments.md) to see deployments. + +- The `status` field indicates if a deployment is blocked. +- The `pending_approval_count` field indicates how many approvals are remaining to run a deployment. +- The `approvals` field contains the deployment's approvals. ## Related features diff --git a/doc/ci/environments/deployment_safety.md b/doc/ci/environments/deployment_safety.md index 55c3c83338d..0e73dc4f7cd 100644 --- a/doc/ci/environments/deployment_safety.md +++ b/doc/ci/environments/deployment_safety.md @@ -26,7 +26,7 @@ If you are using a continuous deployment workflow and want to ensure that concur ## Restrict write access to a critical environment By default, environments can be modified by any team member that has at least the -[Developer role](../../user/permissions.md#project-members-permissions). +Developer role. If you want to restrict write access to a critical environment (for example a `production` environment), you can set up [protected environments](protected_environments.md). @@ -93,7 +93,7 @@ If you want to prevent deployments for a particular period, for example during a vacation period when most employees are out, you can set up a [Deploy Freeze](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). During a deploy freeze period, no deployment can be executed. This is helpful to ensure that deployments do not happen unexpectedly. - + ## Setting appropriate roles to your project GitLab supports several different roles that can be assigned to your project members. See @@ -119,7 +119,7 @@ The other pipelines don't get the protected variable. You can also We recommend that you use protected variables on protected environments to make sure that the secrets aren't exposed unintentionally. You can also define production secrets on the [runner side](../runners/configure_runners.md#prevent-runners-from-revealing-sensitive-information). -This prevents other users with the [Maintainer role](../../user/permissions.md) from reading the secrets and makes sure +This prevents other users with the Maintainer role from reading the secrets and makes sure that the runner only runs on protected branches. For more information, see [pipeline security](../pipelines/index.md#pipeline-security-on-protected-branches). diff --git a/doc/ci/environments/img/environments_open_live_environment_v14_8.png b/doc/ci/environments/img/environments_open_live_environment_v14_8.png Binary files differnew file mode 100644 index 00000000000..daf531c83c4 --- /dev/null +++ b/doc/ci/environments/img/environments_open_live_environment_v14_8.png diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 794e4320fc6..63bdd279927 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -27,7 +27,7 @@ You can even access a [web terminal](#web-terminals-deprecated) for your environ Prerequisites: -- You must have at least the Reporter [role](../../user/permissions.md#project-members-permissions). +- You must have at least the Reporter role. To view a list of environments and deployments: @@ -308,10 +308,18 @@ Note the following: - If the environment URL isn't valid (for example, the URL is malformed), the system doesn't update the environment URL. - If the script that runs in `stop_review` exists only in your repository and therefore can't use - `GIT_STRATEGY: none`, configure [pipelines for merge requests](../../ci/pipelines/merge_request_pipelines.md) + `GIT_STRATEGY: none`, configure [merge request pipelines](../../ci/pipelines/merge_request_pipelines.md) for these jobs. This ensures that runners can fetch the repository even after a feature branch is deleted. For more information, see [Ref Specs for Runners](../pipelines/index.md#ref-specs-for-runners). +NOTE: +For Windows runners, using `echo` to write to `.env` files may fail. Using the PowerShell `Add-Content`command +will help in such cases. For example: + +```powershell +Add-Content -Path deploy.env -Value "DYNAMIC_ENVIRONMENT_URL=$DYNAMIC_ENVIRONMENT_URL" +``` + ## Track newly included merge requests per deployment GitLab can track newly included merge requests per deployment. @@ -372,7 +380,7 @@ places in GitLab: - In a merge request as a link:  - In the Environments view as a button: -  +  - In the Deployments view as a button:  @@ -440,7 +448,7 @@ the `stop_review` job might not be included in all pipelines that include the The job with [`action: stop` might not run](#the-job-with-action-stop-doesnt-run) if it's in a later stage than the job that started the environment. -If you can't use [pipelines for merge requests](../pipelines/merge_request_pipelines.md), +If you can't use [merge request pipelines](../pipelines/merge_request_pipelines.md), set the [`GIT_STRATEGY`](../runners/configure_runners.md#git-strategy) to `none` in the `stop_review` job. Then the [runner](https://docs.gitlab.com/runner/) doesn't try to check out the code after the branch is deleted. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index 78db2345de4..c7d1653aace 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -20,7 +20,7 @@ NOTE: GitLab administrators can use all environments, including protected environments. To protect, update, or unprotect an environment, you need to have at least the -[Maintainer role](../../user/permissions.md). +Maintainer role. ## Protecting environments @@ -38,6 +38,9 @@ To protect an environment: - You can select groups that are already associated with the project only. - Users must have at least the Developer role to appear in the **Allowed to deploy** list. +1. In the **Required approvals** list, select the number of approvals required to deploy to this environment. + - Ensure that this number is less than the number of users allowed to deploy. + - See [Deployment Approvals](deployment_approvals.md) for more information about this feature. 1. Select **Protect**. The protected environment now appears in the list of protected environments. @@ -94,7 +97,7 @@ Alternatively, you can use the API to protect an environment: 1. Use the API to add the group with protected environment access: ```shell - curl --header 'Content-Type: application/json' --request POST --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}]}' \ + curl --header 'Content-Type: application/json' --request POST --data '{"name": "production", "deploy_access_levels": [{"group_id": 9899826}], "required_approval_count": 0}' \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/22034114/protected_environments" ``` @@ -103,12 +106,12 @@ The group now has access and can be seen in the UI. ## Environment access by group membership A user may be granted access to protected environments as part of [group membership](../../user/group/index.md). Users -with the Reporter [role](../../user/permissions.md) can only be granted access to protected environments with this +with the Reporter role can only be granted access to protected environments with this method. ## Deployment branch access -Users with the [Developer role](../../user/permissions.md) can be granted +Users with the Developer role can be granted access to a protected environment through any of these methods: - As an individual contributor, through a role. @@ -125,7 +128,7 @@ they have the following privileges: Users granted access to a protected environment, but not push or merge access to the branch deployed to it, are only granted access to deploy the environment. An individual in a -group with the Reporter [role](../../user/permissions.md), or in groups added to the project with the Reporter +group with the Reporter role, or in groups added to the project with the Reporter role, appears in the dropdown menu for deployment-only access. To add deployment-only access: @@ -136,7 +139,7 @@ To add deployment-only access: 1. Follow the steps in [Protecting Environments](#protecting-environments). Note that deployment-only access is the only possible access level for groups with the Reporter -[role](../../user/permissions.md). +role. ## Modifying and unprotecting environments @@ -194,14 +197,14 @@ and are protected at the same time. In an enterprise organization, with thousands of projects under a single group, ensuring that all of the [project-level protected environments](#protecting-environments) are properly configured is not a scalable solution. For example, a developer -might gain privileged access to a higher environment when they are given the [Maintainer role](../../user/permissions.md) +might gain privileged access to a higher environment when they are given the Maintainer role for a new project. Group-level protected environments can be a solution in this situation. To maximize the effectiveness of group-level protected environments, [group-level memberships](../../user/group/index.md) must be correctly configured: -- Operators should be given at least the [Maintainer role](../../user/permissions.md) +- Operators should be given at least the Maintainer role for the top-level group. They can maintain CI/CD configurations for the higher environments (such as production) in the group-level settings page, which includes group-level protected environments, @@ -210,8 +213,8 @@ configured: configurations are inherited to the child projects as read-only entries. This ensures that only operators can configure the organization-wide deployment ruleset. -- Developers should be given no more than the [Developer role](../../user/permissions.md) - for the top-level group, or explicitly given the [Maintainer role](../../user/permissions.md) for a child project +- Developers should be given no more than the Developer role + for the top-level group, or explicitly given the Maintainer role for a child project They do *NOT* have access to the CI/CD configurations in the top-level group, so operators can ensure that the critical configuration won't be accidentally changed by the developers. @@ -224,7 +227,7 @@ configured: environment configurations exist, to run a deployment job, the user must be allowed in **both** rulesets. - In a project or a subgroup of the top-level group, developers can be - safely assigned the [Maintainer role](../../user/permissions.md) to tune their lower environments (such + safely assigned the Maintainer role to tune their lower environments (such as `testing`). Having this configuration in place: diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index aed45951239..edc58684057 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -1,6 +1,6 @@ --- -stage: Release -group: Release +stage: Verify +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: tutorial --- diff --git a/doc/ci/examples/end_to_end_testing_webdriverio/index.md b/doc/ci/examples/end_to_end_testing_webdriverio/index.md index 9881c9657bc..4d247a4ff74 100644 --- a/doc/ci/examples/end_to_end_testing_webdriverio/index.md +++ b/doc/ci/examples/end_to_end_testing_webdriverio/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments author: Vincent Tunru author_gitlab: Vinnl diff --git a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md index be33e62b75c..6dbec0dfc8b 100644 --- a/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md +++ b/doc/ci/examples/laravel_with_gitlab_and_envoy/index.md @@ -378,7 +378,7 @@ Now, we would need to deploy our app by running `envoy run deploy`, but it won't Now it's time to commit [Envoy.blade.php](https://gitlab.com/mehranrasulian/laravel-sample/blob/master/Envoy.blade.php) and push it to the `main` branch. To keep things simple, we commit directly to `main`, without using [feature-branches](../../../topics/gitlab_flow.md#github-flow-as-a-simpler-alternative) since collaboration is beyond the scope of this tutorial. -In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [Merge Requests](../../../user/project/merge_requests/index.md) to move their code across branches: +In a real world project, teams may use [Issue Tracker](../../../user/project/issues/index.md) and [merge requests](../../../user/project/merge_requests/index.md) to move their code across branches: ```shell git add Envoy.blade.php diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index cdc75fd2bec..23055514839 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -60,6 +60,15 @@ To make submodules work correctly in CI/CD jobs: GIT_SUBMODULE_STRATEGY: recursive ``` +1. You can provide additional flags to control advanced checkout behavior using + [`GIT_SUBMODULE_UPDATE_FLAGS`](runners/configure_runners.md#git-submodule-update-flags). + + ```yaml + variables: + GIT_SUBMODULE_STRATEGY: recursive + GIT_SUBMODULE_UPDATE_FLAGS: --jobs 4 + ``` + If you use the [`CI_JOB_TOKEN`](jobs/ci_job_token.md) to clone a submodule in a pipeline job, the user executing the job must be assigned to a role that has [permission](../user/permissions.md#gitlab-cicd-permissions) to trigger a pipeline diff --git a/doc/ci/img/ci_lint.png b/doc/ci/img/ci_lint.png Binary files differdeleted file mode 100644 index fdc3868cdce..00000000000 --- a/doc/ci/img/ci_lint.png +++ /dev/null diff --git a/doc/ci/img/ci_lint_dry_run.png b/doc/ci/img/ci_lint_dry_run.png Binary files differdeleted file mode 100644 index 61d6379f70e..00000000000 --- a/doc/ci/img/ci_lint_dry_run.png +++ /dev/null diff --git a/doc/ci/img/environments_available_13_10.png b/doc/ci/img/environments_available_13_10.png Binary files differdeleted file mode 100644 index 94ffb0032fa..00000000000 --- a/doc/ci/img/environments_available_13_10.png +++ /dev/null diff --git a/doc/ci/index.md b/doc/ci/index.md index c557e9e6f57..d0c2e1986b2 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -64,7 +64,7 @@ GitLab CI/CD supports numerous configuration options: | [Git submodules for CI/CD](git_submodules.md) | Configure jobs for using Git submodules. | | [SSH keys for CI/CD](ssh_keys/index.md) | Using SSH keys in your CI pipelines. | | [Pipeline triggers](triggers/index.md) | Trigger pipelines through the API. | -| [Pipelines for Merge Requests](pipelines/merge_request_pipelines.md) | Design a pipeline structure for running a pipeline in merge requests. | +| [Merge request pipelines](pipelines/merge_request_pipelines.md) | Design a pipeline structure for running a pipeline in merge requests. | | [Integrate with Kubernetes clusters](../user/infrastructure/clusters/index.md) | Connect your project to Google Kubernetes Engine (GKE) or an existing Kubernetes cluster. | | [Optimize GitLab and GitLab Runner for large repositories](large_repositories/index.md) | Recommended strategies for handling large repositories. | | [`.gitlab-ci.yml` full reference](yaml/index.md) | All the attributes you can use with GitLab CI/CD. | diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 1906d9cdb6c..a62f670cb12 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -23,7 +23,6 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Releases](../../api/releases/index.md). - [Terraform plan](../../user/infrastructure/index.md). -The token has the same permissions to access the API as the user that executes the The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to @@ -95,7 +94,7 @@ The job token scope is only for controlling access to private projects. 1. Expand **Token Access**. 1. Toggle **Limit CI_JOB_TOKEN access** to enabled. 1. Optional. Add existing projects to the token's access scope. The user adding a - project must have the [maintainer role](../../user/permissions.md) in both projects. + project must have the Maintainer role in both projects. There is [a proposal](https://gitlab.com/groups/gitlab-org/-/epics/3559) to improve the feature with more strategic control of the access permissions. diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 104badb782c..39e14d0d20a 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -95,6 +95,9 @@ You can't use these keywords as job names: - `variables` - `cache` - `include` +- `true` +- `false` +- `nil` Job names must be 255 characters or less. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342800) in GitLab 14.5, [with a feature flag](../../administration/feature_flags.md) named `ci_validate_job_length`. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index a2406a68bb2..6523de0ed1e 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -220,7 +220,7 @@ check the value of the `$CI_PIPELINE_SOURCE` variable: | `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | | `external` | When you use CI services other than GitLab. | | `external_pull_request_event` | When an external pull request on GitHub is created or updated. See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). | -| `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/pipelines_for_merged_results.md), and [merge trains](../pipelines/merge_trains.md). | +| `merge_request_event` | For pipelines created when a merge request is created or updated. Required to enable [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | | `parent_pipeline` | For pipelines triggered by a [parent/child pipeline](../pipelines/parent_child_pipelines.md) with `rules`. Use this pipeline source in the child pipeline configuration so that it can be triggered by the parent pipeline. | | `pipeline` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](../yaml/index.md#trigger) keyword. | | `push` | For pipelines triggered by a `git push` event, including for branches and tags. | @@ -263,6 +263,9 @@ Other commonly used variables for `if` clauses: branch. Use when you want to have the same configuration in multiple projects with different default branches. - `if: '$CI_COMMIT_BRANCH =~ /regex-expression/'`: If the commit branch matches a regular expression. +- `if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_COMMIT_TITLE =~ /Merge branch.*/`: + If the commit branch is the default branch and the commit message title matches a regular expression. + For example, the default commit message for a merge commit starts with `Merge branch`. - `if: '$CUSTOM_VARIABLE !~ /regex-expression/'`: If the [custom variable](../variables/index.md#custom-cicd-variables) `CUSTOM_VARIABLE` does **not** match a regular expression. - `if: '$CUSTOM_VARIABLE == "value1"'`: If the custom variable `CUSTOM_VARIABLE` is @@ -406,9 +409,9 @@ the `build` job is still skipped. The job does not run for any of the files. With some configurations that use `changes`, [jobs or pipelines might run unexpectedly](#jobs-or-pipelines-run-unexpectedly-when-using-changes) -#### Use `only:changes` with pipelines for merge requests +#### Use `only:changes` with merge request pipelines -With [pipelines for merge requests](../pipelines/merge_request_pipelines.md), +With [merge request pipelines](../pipelines/merge_request_pipelines.md), it's possible to define a job to be created based on files modified in a merge request. @@ -704,9 +707,12 @@ deploystacks: [gcp, data] deploystacks: [vultr, data] ``` -In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/239737), you can -use the variables defined in `parallel: matrix` with the [`tags`](../yaml/index.md#tags) keyword for -dynamic runner selection. +### Select different runner tags for each parallel matrix job + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/239737) in GitLab 14.1. + +You can use variables defined in `parallel: matrix` with the [`tags`](../yaml/index.md#tags) +keyword for dynamic runner selection: ```yaml deploystacks: @@ -931,7 +937,7 @@ For example: You might have jobs or pipelines that run unexpectedly when using [`rules: changes`](../yaml/index.md#ruleschanges) or [`only: changes`](../yaml/index.md#onlychanges--exceptchanges) without -[pipelines for merge requests](../pipelines/merge_request_pipelines.md). +[merge request pipelines](../pipelines/merge_request_pipelines.md). Pipelines on branches or tags that don't have an explicit association with a merge request use a previous SHA to calculate the diff. This calculation is equivalent to `git diff HEAD~` @@ -944,3 +950,19 @@ and can cause unexpected behavior, including: Additionally, rules with `changes` always evaluate as true in [scheduled pipelines](../pipelines/schedules.md). All files are considered to have changed when a scheduled pipeline runs, so jobs might always be added to scheduled pipelines that use `changes`. + +### `You are not allowed to download code from this project.` error message + +You might see pipelines fail when a GitLab administrator runs a protected manual job +in a private project. + +CI/CD jobs usually clone the project when the job starts, and this uses [the permissions](../../user/permissions.md#job-permissions) +of the user that runs the job. All users, including administrators, must be direct members +of a private project to clone the source of that project. [An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/23130) +to change this behavior. + +To run protected manual jobs: + +- Add the administrator as a direct member of the private project (any role) +- [Impersonate a user](../../user/admin_area/index.md#user-impersonation) who is a + direct member of the project. diff --git a/doc/ci/lint.md b/doc/ci/lint.md index e58907ee0bd..c0df0b2a439 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -4,48 +4,54 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Validate `.gitlab-ci.yml` syntax with the CI Lint tool **(FREE)** +# Validate GitLab CI/CD configuration **(FREE)** -If you want to test the validity of your GitLab CI/CD configuration before committing -the changes, you can use the CI Lint tool. This tool checks for syntax and logical -errors by default, and can simulate pipeline creation to try to find more complicated -issues as well. +Use the CI Lint tool to check the validity of GitLab CI/CD configuration. +You can validate the syntax from a `.gitlab-ci.yml` file or any other sample CI/CD configuration. +This tool checks for syntax and logic errors, and can simulate pipeline +creation to try to find more complicated configuration problems. -To access the CI Lint tool, navigate to **CI/CD > Pipelines** or **CI/CD > Jobs** -in your project and click **CI lint**. +If you use the [pipeline editor](pipeline_editor/index.md), it verifies configuration +syntax automatically. -If you use VS Code, you can also validate your CI/CD configuration with the +If you use VS Code, you can validate your CI/CD configuration with the [GitLab Workflow VS Code extension](../user/project/repository/vscode.md). -## Validate basic logic and syntax +## Check CI/CD syntax -By default, the CI lint checks the syntax of your CI YAML configuration and also runs -some basic logical validations. Configuration added with the [`includes` keyword](yaml/index.md#include), -is also validated. +The CI lint tool checks the syntax of GitLab CI/CD configuration, including +configuration added with the [`includes` keyword](yaml/index.md#include). -To use the CI lint, paste a complete CI configuration (`.gitlab-ci.yml` for example) -into the text box and click **Validate**: +To check CI/CD configuration with the CI lint tool: - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select one of: + - **CI/CD > Pipelines** + - **CI/CD > Jobs** +1. In the top right, select **CI lint**. +1. Paste a copy of the CI/CD configuration you want to check into the text box. +1. Select **Validate**. -## Pipeline simulation +## Simulate a pipeline > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229794) in GitLab 13.3. -Not all pipeline configuration issues can be found by the [basic CI lint validation](#validate-basic-logic-and-syntax). -You can simulate the creation of a pipeline for deeper validation that can discover -more complicated issues. - -To validate the configuration by running a pipeline simulation: +You can simulate the creation of a GitLab CI/CD pipeline to find more complicated issues, +including problems with [`needs`](yaml/index.md#needs) and [`rules`](yaml/index.md#rules) +configuration. A simulation runs as a Git `push` event on the default branch. -1. Paste the GitLab CI configuration to verify into the text box. -1. Select the **Simulate pipeline creation for the default branch** checkbox. -1. Select **Validate**. +Prerequisites: - +- You must have [permissions](../user/permissions.md#project-members-permissions) + to create pipelines on this branch to validate with a simulation. -### Pipeline simulation limitations +To simulate a pipeline: -Simulations run as `git push` events against the default branch. You must have -[permissions](../user/permissions.md#project-members-permissions) to create pipelines -on this branch to validate with a simulation. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select one of: + - **CI/CD > Pipelines** + - **CI/CD > Jobs** +1. In the top right, select **CI lint**. +1. Paste a copy of the CI/CD configuration you want to check into the text box. +1. Select **Simulate pipeline creation for the default branch**. +1. Select **Validate**. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index eb302b9ed7f..5b472eec7b5 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -1,8 +1,7 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Metrics Reports **(PREMIUM)** diff --git a/doc/ci/migration/jenkins.md b/doc/ci/migration/jenkins.md index ef6f28e36e5..bfa2eb54806 100644 --- a/doc/ci/migration/jenkins.md +++ b/doc/ci/migration/jenkins.md @@ -196,7 +196,7 @@ can leverage. You can see the complete list of packaging features in the Where you may have used plugins to get things like code quality, unit tests, security scanning, and so on working in Jenkins, GitLab takes advantage of our connected ecosystem to automatically pull these kinds of results into -your Merge Requests, pipeline details pages, and other locations. You may find that you actually don't +your merge requests, pipeline details pages, and other locations. You may find that you actually don't need to configure anything to have these appear. If they aren't working as expected, or if you'd like to see what's available, our [CI feature index](../index.md#gitlab-cicd-features) has the full list diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index e0fb5b45986..8b10a74bd78 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -149,7 +149,7 @@ namespace. ## How CI/CD minutes are calculated -CI/CD minutes are calculated based on: +CI/CD minutes for individual jobs are calculated based on: - The duration the job runs. - The visibility of the projects where the job runs. @@ -174,6 +174,10 @@ For example: - If a pipeline runs for one of the personal projects for `alice`, the CI/CD minutes are added to the overall consumption for the `alice` namespace. +The CI/CD minutes used by one pipeline is the total CI/CD minutes used by all the jobs +that ran in the pipeline. The CI/CD minute usage for a pipeline can be higher than +the duration of the pipeline if many jobs ran at the same time. + ### Cost factor The cost factor for a job running on a shared runner is: diff --git a/doc/ci/pipelines/img/merged_result_pipeline.png b/doc/ci/pipelines/img/merged_result_pipeline.png Binary files differdeleted file mode 100644 index 2584cd4d38d..00000000000 --- a/doc/ci/pipelines/img/merged_result_pipeline.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index b873b2ae30f..5a5fd2b5540 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -50,14 +50,14 @@ Pipelines can be configured in many different ways: followed by the next stage. - [Directed Acyclic Graph Pipeline (DAG) pipelines](../directed_acyclic_graph/index.md) are based on relationships between jobs and can run more quickly than basic pipelines. -- [Pipelines for Merge Requests](../pipelines/merge_request_pipelines.md) run for merge +- [Merge request pipelines](../pipelines/merge_request_pipelines.md) run for merge requests only (rather than for every commit). -- [Pipelines for Merged Results](../pipelines/pipelines_for_merged_results.md) +- [Merged results pipelines](../pipelines/merged_results_pipelines.md) are merge request pipelines that act as though the changes from the source branch have already been merged into the target branch. -- [Merge Trains](../pipelines/merge_trains.md) - use pipelines for merged results to queue merges one after the other. -- [Parent-Child pipelines](parent_child_pipelines.md) break down complex pipelines +- [Merge trains](../pipelines/merge_trains.md) + use merged results pipelines to queue merges one after the other. +- [Parent-child pipelines](parent_child_pipelines.md) break down complex pipelines into one parent pipeline that can trigger multiple child sub-pipelines, which all run in the same project and with the same SHA. This pipeline architecture is commonly used for mono-repos. - [Multi-project pipelines](multi_project_pipelines.md) combine pipelines for different projects together. @@ -123,7 +123,7 @@ This table lists the refspecs injected for each pipeline type: |--------------- |---------------------------------------- | | pipeline for branches | `+<sha>:refs/pipelines/<id>` and `+refs/heads/<name>:refs/remotes/origin/<name>` | | pipeline for tags | `+<sha>:refs/pipelines/<id>` and `+refs/tags/<name>:refs/tags/<name>` | -| [pipeline for merge requests](../pipelines/merge_request_pipelines.md) | `+<sha>:refs/pipelines/<id>` | +| [merge request pipeline](../pipelines/merge_request_pipelines.md) | `+<sha>:refs/pipelines/<id>` | The refs `refs/heads/<name>` and `refs/tags/<name>` exist in your project repository. GitLab generates the special ref `refs/pipelines/<id>` during a @@ -281,7 +281,7 @@ pipelines. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24851) in GitLab 12.7. -Users with the [Owner role](../../user/permissions.md) in a project can delete a pipeline +Users with the Owner role for a project can delete a pipeline by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** page, then using the **Delete** button. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 1710c57b36b..fa8041671dc 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -1,9 +1,8 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/job_artifacts.html' -type: reference, howto --- # Job artifacts **(FREE)** @@ -194,7 +193,8 @@ artifacts: ### Add untracked files to artifacts Use [`artifacts:untracked`](../yaml/index.md#artifactsuntracked) to add all Git untracked -files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). +files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). Untracked +files are those that haven't been added to the repository but exist in the repository checkout. Save all Git untracked files and files in `binaries`: @@ -259,7 +259,7 @@ You can delete a single job, which also removes the job's artifacts and log. You must be: - The owner of the job. -- A [maintainer](../../user/permissions.md#gitlab-cicd-permissions) of the project. +- A user with at least the Maintainer role for the project. To delete a job: @@ -381,7 +381,7 @@ to the `expire_in` specification. If a new pipeline for the same ref completes successfully, the previous pipeline's artifacts are deleted according to the `expire_in` configuration. The artifacts -of the new pipeline are kept automatically. +of the new pipeline are kept automatically. Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index 4d7ebc09e6f..dcc3e7e6919 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -5,13 +5,15 @@ info: To determine the technical writer assigned to the Stage/Group associated w --- -# Pipelines for merge requests **(FREE)** +# Merge request pipelines **(FREE)** + +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merge requests` to `merge request pipelines` in GitLab 14.8. You can configure your [pipeline](index.md) to run every time you commit changes to a branch. This type of pipeline is called a *branch pipeline*. Alternatively, you can configure your pipeline to run every time you make changes to the -source branch for a merge request. This type of pipeline is called a *pipeline for merge requests*. +source branch for a merge request. This type of pipeline is called a *merge request pipeline*. Branch pipelines: @@ -20,28 +22,28 @@ Branch pipelines: - Have access to [some predefined variables](../variables/predefined_variables.md). - Have access to [protected variables](../variables/index.md#protect-a-cicd-variable). -Pipelines for merge requests: +Merge request pipelines: - Run when you: - Create a new merge request. - Push a new commit to the source branch for a merge request. - Select **Run pipeline** from the **Pipelines** tab in a merge request. This option - is only available when pipelines for merge requests are configured for the pipeline. + is only available when merge request pipelines are configured for the pipeline. - Do not run by default. The jobs in the CI/CD configuration file [must be configured](#prerequisites) - to run in pipelines for merge request. + to run in merge request pipelines. - Have access to [more predefined variables](#available-predefined-variables). - Do not have access to [protected variables](../variables/index.md#protect-a-cicd-variable). Both of these types of pipelines can appear on the **Pipelines** tab of a merge request. -## Types of pipelines for merge requests +## Types of merge request pipelines -The three types of pipelines for merge requests are: +The three types of merge request pipelines are: -- Pipelines for merge requests, which run on the changes in the merge request's +- Merge request pipelines, which run on the changes in the merge request's source branch. These pipelines display a `detached` label to indicate that the pipeline ran only on the contents of the source branch, ignoring the target branch. -- [Pipelines for merged results](pipelines_for_merged_results.md), which run on +- [Merged results pipelines](merged_results_pipelines.md), which run on the result of combining the source branch's changes with the target branch. - [Merge trains](merge_trains.md), which run when merging multiple merge requests at the same time. The changes from each merge request are combined into the @@ -50,31 +52,31 @@ The three types of pipelines for merge requests are: ## Prerequisites -To use pipelines for merge requests: +To use merge request pipelines: - Your project's [CI/CD configuration file](../yaml/index.md) must be configured with - jobs that run in pipelines for merge requests. To do this, you can use: + jobs that run in merge request pipelines. To do this, you can use: - [`rules`](#use-rules-to-add-jobs). - [`only/except`](#use-only-to-add-jobs). -- You must have at least the Developer [role](../../user/permissions.md) in the - source project to run a pipeline for merge requests. +- You must have at least the Developer role in the + source project to run a merge request pipeline. - Your repository must be a GitLab repository, not an [external repository](../ci_cd_for_external_repos/index.md). ## Use `rules` to add jobs You can use the [`rules`](../yaml/index.md#rules) keyword to configure jobs to run in -pipelines for merge requests. For example: +merge request pipelines. For example: ```yaml job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" rules: - if: $CI_PIPELINE_SOURCE == 'merge_request_event' ``` You can also use the [`workflow: rules`](../yaml/index.md#workflowrules) keyword -to configure the entire pipeline to run in pipelines for merge requests. For example: +to configure the entire pipeline to run in merge request pipelines. For example: ```yaml workflow: @@ -83,22 +85,22 @@ workflow: job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" job2: script: - - echo "This job also runs in pipelines for merge requests" + - echo "This job also runs in merge request pipelines" ``` ## Use `only` to add jobs You can use the [`only`](../yaml/index.md#onlyrefs--exceptrefs) keyword with `merge_requests` -to configure jobs to run in pipelines for merge requests. +to configure jobs to run in merge request pipelines. ```yaml job1: script: - - echo "This job runs in pipelines for merge requests" + - echo "This job runs in merge request pipelines" only: - merge_requests ``` @@ -122,7 +124,7 @@ Pipelines for forks display with the **fork** badge in the parent project: ### Run pipelines in the parent project **(PREMIUM)** -Project members in the parent project can choose to run a pipeline for merge requests +Project members in the parent project can choose to run a merge request pipeline for a merge request submitted from a fork project. This pipeline: - Is created and runs in the parent (target) project, not the fork (source) project. @@ -140,20 +142,26 @@ parent project when the pipeline runs, even before merge. As a reviewer, careful check the changes in the merge request before triggering the pipeline. GitLab shows a warning that you must accept before you can trigger the pipeline. -Parent project members with at least the [Developer role](../../user/permissions.md) -can create pipelines in the parent project for merge requests from a forked project: +Prerequisites: + +- You must be a member of the parent project and have at least the [Developer role](../../user/permissions.md). +- The fork project must be [visible](../../public_access/public_access.md) to the + user running the pipeline. Otherwise, the **Pipelines** tab does not display + in the merge request. + +To run a pipeline in the parent project for a merge request from a fork project: 1. In the merge request, go to the **Pipelines** tab. 1. Select **Run pipeline**. You must accept the warning, or the pipeline does not run. ## Available predefined variables -When you use pipelines for merge requests, you can use: +When you use merge request pipelines, you can use: - All the same [predefined variables](../variables/predefined_variables.md) that are available in branch pipelines. - [Additional predefined variables](../variables/predefined_variables.md#predefined-variables-for-merge-request-pipelines) - available only to jobs in pipelines for merge requests. These variables contain + available only to jobs in merge request pipelines. These variables contain information from the associated merge request, which can be when calling the [GitLab Merge Request API endpoint](../../api/merge_requests.md) from a job. @@ -166,14 +174,14 @@ to run for both branches and merge requests at the same time. Adjust your pipeli configuration to [avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). In [GitLab 13.7 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201845), -you can add `workflow:rules` to [switch from branch pipelines to pipelines for merge requests](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). +you can add `workflow:rules` to [switch from branch pipelines to merge request pipelines](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines). After a merge request is open on the branch, the pipeline switches to a merge request pipeline. ### Two pipelines when pushing an invalid CI/CD configuration file If you push an invalid CI/CD configuration to a merge request's branch, two failed pipelines appear in the pipelines tab. One pipeline is a failed branch pipeline, -the other is a failed pipeline for merge requests. +the other is a failed merge request pipeline. When the configuration syntax is fixed, no further failed pipelines should appear. To find and fix the configuration problem, you can use: @@ -183,7 +191,7 @@ To find and fix the configuration problem, you can use: ### The merge request's pipeline is marked as failed but the latest pipeline succeeded -It's possible to have both branch pipelines and pipelines for merge requests in the +It's possible to have both branch pipelines and merge request pipelines in the **Pipelines** tab of a single merge request. This might be [by configuration](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines), or [by accident](#two-pipelines-when-pushing-to-a-branch). @@ -191,8 +199,8 @@ If both types of pipelines are in one merge request, the merge request's pipelin is not considered successful if: - The branch pipeline succeeds. -- The pipeline for merge request fails. +- The merge request pipeline fails. When using the [merge when pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md) -feature and both pipelines types are present, the pipelines for merge requests are checked, +feature and both pipelines types are present, the merge request pipelines are checked, not the branch pipelines. diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index d47cbf5f47c..ffcce06cfbd 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). -When [pipelines for merged results](pipelines_for_merged_results.md) are +When [merged results pipelines](merged_results_pipelines.md) are enabled, the pipeline jobs run as if the changes from your source branch have already been merged into the target branch. @@ -67,7 +67,7 @@ branch](https://www.youtube.com/watch?v=D4qCqXgZkHQ). To enable merge trains: -- You must have the [Maintainer role](../../user/permissions.md). +- You must have the Maintainer role. - You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later. - In GitLab 13.0 and later, you need [Redis](https://redis.io/) 5.0 or later. - Your repository must be a GitLab repository, not an @@ -93,7 +93,7 @@ In GitLab 13.5 and earlier, there is only one checkbox, named WARNING: If you select the check box but don't configure your CI/CD to use -pipelines for merge requests, your merge requests may become stuck in an +merge request pipelines, your merge requests may become stuck in an unresolved state or your pipelines may be dropped. ## Start a merge train @@ -145,8 +145,22 @@ This is the fastest option to get the change merged into the target branch.  WARNING: -Each time you merge a merge request immediately, the current merge train -is recreated and all pipelines restart. +Each time you merge a merge request immediately, the current merge train is recreated, +all pipelines restart, and [redundant pipelines are cancelled](#automatic-pipeline-cancellation). + +### Automatic pipeline cancellation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12996) in GitLab 12.3. + +GitLab CI/CD can detect the presence of redundant pipelines, and cancels them +to conserve CI resources. + +When a user merges a merge request immediately in an ongoing merge +train, the train is reconstructed, because it recreates the expected +post-merge commit and pipeline. In this case, the merge train may already +have pipelines running against the previous expected post-merge commit. +These pipelines are considered redundant and are automatically +canceled. ## Troubleshooting @@ -179,8 +193,8 @@ for more information. ### Merge train pipeline cannot be retried -When a pipeline for merge trains fails the merge request is dropped from the train and the pipeline can't be retried. -Pipelines for merge trains run on the merged result of the changes in the merge request and +When a merge train pipeline fails, the merge request is dropped from the train and the pipeline can't be retried. +Merge train pipelines run on the merged result of the changes in the merge request and the changes from other merge requests already on the train. If the merge request is dropped from the train, the merged result is out of date and the pipeline can't be retried. @@ -197,11 +211,16 @@ is enabled in **Settings > General > Merge requests**. This option requires that run a new successful pipeline before you can re-add a merge request to a merge train. Merge trains ensure that each pipeline has succeeded before a merge happens, so -you can clear the **Pipelines must succeed** checkbox and keep -**Enable merge trains and pipelines for merged results** (merge trains) selected. +you can: + +- Clear the **Pipelines must succeed** checkbox. +- Select the **Enable merged results pipelines** and **Enable merge trains** checkboxes. + + In GitLab 13.5 and earlier, there is only one checkbox, named + **Enable merge trains and pipelines for merged results**. If you want to keep the **Pipelines must succeed** option selected along with merge -trains, create a new pipeline for merged results when this error occurs: +trains, create a new merged results pipeline when this error occurs: 1. On the **Pipelines** tab, select **Run pipeline**. 1. Select **Start/Add to merge train when pipeline succeeds**. @@ -215,8 +234,8 @@ In [GitLab 13.6 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/244831) you can [enable or disable merge trains in the project settings](#enable-merge-trains). In GitLab 13.5 and earlier, merge trains are automatically enabled when -[pipelines for merged results](pipelines_for_merged_results.md) are enabled. -To use pipelines for merged results without using merge trains, you can enable a +[merged results pipelines](merged_results_pipelines.md) are enabled. +To use merged results pipelines without using merge trains, you can enable a [feature flag](../../user/feature_flags.md) that blocks the merge trains feature. [GitLab administrators with access to the GitLab Rails console](../../administration/feature_flags.md) diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md new file mode 100644 index 00000000000..4794107cc87 --- /dev/null +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -0,0 +1,78 @@ +--- +stage: Verify +group: Pipeline Execution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Merged results pipelines **(PREMIUM)** + +> [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/351192) from `pipelines for merged results` to `merged results pipelines` in GitLab 14.8. + +A *merged results pipeline* is a type of [merge request pipeline](merge_request_pipelines.md). It is a pipeline that runs against the results of the source and target branches merged together. + +GitLab creates an internal commit with the merged results, so the pipeline can run +against it. This commit does not exist in either branch, +but you can view it in the pipeline details. + +The pipeline runs against the target branch as it exists at the moment you run the pipeline. +Over time, while you're working in the source branch, the target branch might change. +Any time you want to be sure the merged results are accurate, you should re-run the pipeline. + +Merged results pipelines can't run when: + +- The target branch has changes that conflict with the changes in the source branch. +- The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). + +In these cases, the pipeline runs as a [merge request pipeline](merge_request_pipelines.md) +and is labeled as `detached`. + +## Prerequisites + +To use merged results pipelines: + +- Your project's [CI/CD configuration file](../yaml/index.md) must be configured to + [run jobs in merge request pipelines](merge_request_pipelines.md#prerequisites). +- Your repository must be a GitLab repository, not an + [external repository](../ci_cd_for_external_repos/index.md). +- You must not be using [fast forward merges](../../user/project/merge_requests/fast_forward_merge.md). + [An issue exits](https://gitlab.com/gitlab-org/gitlab/-/issues/26996) to change this behavior. + +## Enable merged results pipelines + +To enable merged results pipelines in a project, you must have at least the +Maintainer role: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Merge requests**. +1. Select **Enable merged results pipelines**. +1. Select **Save changes**. + +WARNING: +If you select the checkbox but don't configure your pipeline to use +merge request pipelines, your merge requests may become stuck in an +unresolved state or your pipelines may be dropped. + +## Troubleshooting + +### Merged results pipelines are not created + +In GitLab 13.7 and earlier, merged results pipelines might not be created due +to a disabled [feature flag](../../user/feature_flags.md). This feature flag +[was removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299115) in GitLab 13.8. +Upgrade to 13.8 or later, or make sure the `:merge_ref_auto_sync` +[feature flag is enabled](../../administration/feature_flags.md#check-if-a-feature-flag-is-enabled) +on your GitLab instance. + +### Pipelines fail intermittently with a `fatal: reference is not a tree:` error + +Merged results pipelines run on a merge ref for a merge request +(`refs/merge-requests/<iid>/merge`), so the Git reference could be overwritten at an +unexpected time. + +For example, when a source or target branch is advanced, the pipeline fails with +the `fatal: reference is not a tree:` error, which indicates that the checkout-SHA +is not found in the merge ref. + +This behavior was improved in GitLab 12.4 by introducing [persistent pipeline refs](../troubleshooting.md#fatal-reference-is-not-a-tree-error). +Upgrade to GitLab 12.4 or later to resolve the problem. diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index 8a83e7e31f4..2163527e3ca 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -283,7 +283,7 @@ tag in a different project. Prerequisites: - The upstream project must be [public](../../public_access/public_access.md). -- The user must have the [Developer role](../../user/permissions.md#project-members-permissions) +- The user must have the Developer role in the upstream project. To trigger the pipeline when the upstream project is rebuilt: diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 5e4b707a38c..a3ded24e8c9 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -100,7 +100,7 @@ microservice_a: ## Merge request child pipelines -To trigger a child pipeline as a [Merge Request Pipeline](merge_request_pipelines.md) we need to: +To trigger a child pipeline as a [merge request pipeline](merge_request_pipelines.md) we need to: - Set the trigger job to run on merge requests: diff --git a/doc/ci/pipelines/pipeline_artifacts.md b/doc/ci/pipelines/pipeline_artifacts.md index b174b6af9f9..e9dd1b2a942 100644 --- a/doc/ci/pipelines/pipeline_artifacts.md +++ b/doc/ci/pipelines/pipeline_artifacts.md @@ -1,8 +1,7 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, howto --- # Pipeline artifacts **(FREE)** diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 91a49a48882..457f5ce9c30 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -1,133 +1,9 @@ --- -stage: Verify -group: Pipeline Execution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'merged_results_pipelines.md' +remove_date: '2022-04-27' --- -# Pipelines for merged results **(PREMIUM)** +This document was moved to [another location](merged_results_pipelines.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7380) in GitLab 11.10. - -When you submit a merge request, you are requesting to merge changes from a -source branch into a target branch. By default, the CI pipeline runs jobs -against the source branch. - -With *pipelines for merged results*, the pipeline runs as if the changes from -the source branch have already been merged into the target branch. The commit shown for the pipeline does not exist on the source or target branches but represents the combined target and source branches. - - - -If the pipeline fails due to a problem in the target branch, you can wait until the -target is fixed and re-run the pipeline. -This new pipeline runs as if the source is merged with the updated target, and you -don't need to rebase. - -The pipeline does not automatically run when the target branch changes. Only changes -to the source branch trigger a new pipeline. If a long time has passed since the last successful -pipeline, you may want to re-run it before merge, to ensure that the source changes -can still be successfully merged into the target. - -When the merge request can't be merged, the pipeline runs against the source branch only. For example, when: - -- The target branch has changes that conflict with the changes in the source branch. -- The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). - -In these cases, the pipeline runs as a [pipeline for merge requests](merge_request_pipelines.md) -and is labeled as `detached`. If these cases no longer exist, new pipelines -again run against the merged results. - -Any user who has developer [permissions](../../user/permissions.md) can run a -pipeline for merged results. - -## Prerequisites - -To enable pipelines for merge results: - -- You must have the [Maintainer role](../../user/permissions.md). -- You must be using [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or later. -- You must not be using - [fast forward merges](../../user/project/merge_requests/fast_forward_merge.md) yet. - To follow progress, see [#26996](https://gitlab.com/gitlab-org/gitlab/-/issues/26996). -- Your repository must be a GitLab repository, not an - [external repository](../ci_cd_for_external_repos/index.md). - -## Enable pipelines for merged results - -To enable pipelines for merged results for your project: - -1. [Configure your CI/CD configuration file](merge_request_pipelines.md#prerequisites) - so that the pipeline or individual jobs run for merge requests. -1. Visit your project's **Settings > General** and expand **Merge requests**. -1. Check **Enable merged results pipelines**. -1. Click **Save changes**. - -WARNING: -If you select the checkbox but don't configure your CI/CD to use -pipelines for merge requests, your merge requests may become stuck in an -unresolved state or your pipelines may be dropped. - -## Using Merge Trains - -When you enable [Pipelines for merged results](#pipelines-for-merged-results), -GitLab [automatically displays](merge_trains.md#add-a-merge-request-to-a-merge-train) -a **Start/Add Merge Train button**. - -Generally, this is a safer option than merging merge requests immediately, because your -merge request is evaluated with an expected post-merge result before the actual -merge happens. - -For more information, read the [documentation on Merge Trains](merge_trains.md). - -## Automatic pipeline cancellation - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12996) in GitLab 12.3. - -GitLab CI/CD can detect the presence of redundant pipelines, and cancels them -to conserve CI resources. - -When a user merges a merge request immediately in an ongoing merge -train, the train is reconstructed, because it recreates the expected -post-merge commit and pipeline. In this case, the merge train may already -have pipelines running against the previous expected post-merge commit. -These pipelines are considered redundant and are automatically -canceled. - -## Troubleshooting - -### Pipelines for merged results not created even with new change pushed to merge request - -Can be caused by some disabled feature flags. Please make sure that -the following feature flags are enabled on your GitLab instance: - -- `:merge_ref_auto_sync` - -To check and set these feature flag values, please ask an administrator to: - -1. Log into the Rails console of the GitLab instance: - - ```shell - sudo gitlab-rails console - ``` - -1. Check if the flags are enabled or not: - - ```ruby - Feature.enabled?(:merge_ref_auto_sync) - ``` - -1. If needed, enable the feature flags: - - ```ruby - Feature.enable(:merge_ref_auto_sync) - ``` - -### Intermittently pipelines fail by `fatal: reference is not a tree:` error - -Since pipelines for merged results are a run on a merge ref of a merge request -(`refs/merge-requests/<iid>/merge`), the Git reference could be overwritten at an -unexpected timing. For example, when a source or target branch is advanced. -In this case, the pipeline fails because of `fatal: reference is not a tree:` error, -which indicates that the checkout-SHA is not found in the merge ref. - -This behavior was improved at GitLab 12.4 by introducing [Persistent pipeline refs](../troubleshooting.md#fatal-reference-is-not-a-tree-error). -You should be able to create pipelines at any timings without concerning the error. +<!-- This redirect file can be deleted after 2022-04-27. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index 692460120fe..fe9db3306cd 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -17,7 +17,9 @@ Pipeline schedules can be used to also run [pipelines](index.md) at specific int - Every other Sunday at 0900 hours (cron example: `0 9 * * sun%2`). - Once every day (cron example: `0 0 * * *`). -Schedule timing is configured with cron notation, parsed by [Fugit](https://github.com/floraison/fugit). +Schedule timing is configured with [cron notation](../../topics/cron/index.md). +You can use any cron value, but scheduled pipelines cannot run more frequently +than the instance's [maximum frequency for scheduled pipelines](#advanced-configuration). In addition to using the GitLab UI, pipeline schedules can be maintained using the [Pipeline schedules API](../../api/pipeline_schedules.md). @@ -82,20 +84,24 @@ job: ### Advanced configuration **(FREE SELF)** -The pipelines are not executed exactly on schedule because schedules are handled by -Sidekiq, which runs according to its interval. +Scheduled pipelines can be configured with any [cron value](../../topics/cron/index.md), +but they do not always run exactly when scheduled. An internal process, called the +_pipeline schedule worker_, queues all the scheduled pipelines, but does not +run continuously. The worker runs on its own schedule, and scheduled pipelines that +are ready to start are only queued the next time the worker runs. Scheduled pipelines +can't run more frequently than the worker. -For example, only two pipelines are created per day if: +The default frequency of the pipeline schedule worker is `3-59/10 * * * *` (every ten minutes, +starting with `0:03`, `0:13`, `0:23`, and so on). The default frequency for GitLab.com +is listed in the [GitLab.com settings](../../user/gitlab_com/index.md#gitlab-cicd). -- You set a schedule to create a pipeline every minute (`* * * * *`). -- The Sidekiq worker runs on 00:00 and 12:00 every day (`0 */12 * * *`). - -To change the Sidekiq worker's frequency: +To change the frequency of the pipeline schedule worker: 1. Edit the `gitlab_rails['pipeline_schedule_worker_cron']` value in your instance's `gitlab.rb` file. 1. [Reconfigure GitLab](../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -For GitLab.com, refer to the [dedicated settings page](../../user/gitlab_com/index.md#gitlab-cicd). +For example, to set the maximum frequency of pipelines to twice a day, set `pipeline_schedule_worker_cron` +to a cron value of `0 */12 * * *` (`00:00` and `12:00` every day). ## Working with scheduled pipelines diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 85824dfb7c7..e22746dbfa0 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -108,6 +108,10 @@ To customize the path: - Is on an external site, enter the full URL. 1. Select **Save changes**. +NOTE: +You cannot use your project's [pipeline editor](../pipeline_editor/index.md) to +edit CI/CD configuration files in other projects or on an external site. + ### Custom CI/CD configuration file examples If the CI/CD configuration file is not in the root directory, the path must be relative to it. @@ -191,21 +195,18 @@ Jobs that exceed the timeout are marked as failed. You can override this value [for individual runners](../runners/configure_runners.md#set-maximum-job-timeout-for-a-runner). -## Add test coverage results to a merge request +## Add test coverage results to a merge request (DEPRECATED) + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) +for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. If you use test coverage in your code, you can use a regular expression to find coverage results in the job log. You can then include these results in the merge request in GitLab. -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Settings > CI/CD**. -1. Expand **General pipelines**. -1. In the **Test coverage parsing** field, enter a regular expression. - Leave blank to disable this feature. - -You can use <https://rubular.com> to test your regex. The regex returns the **last** -match found in the output. - If the pipeline succeeds, the coverage is shown in the merge request widget and in the jobs table. If multiple jobs in the pipeline have coverage reports, they are averaged. @@ -214,6 +215,31 @@ averaged.  +To define a coverage-parsing regular expression: + +- Using the project's `.gitlab-ci.yml`, provide a regular expression using the [`coverage`](../yaml/index.md#coverage) + keyword. Setting the regular expression this way takes precedence over the project's CI/CD settings. + +- Using the Project's CI/CD settings: + - Set using the GitLab UI: + + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Settings > CI/CD**. + 1. Expand **General pipelines**. + 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. + + - Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) + using the GitLab API with the `build_coverage_regex` attribute: + + ```shell + curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ + --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ + --data "build_coverage_regex=<your-regular-expression>" + ``` + +You can use <https://rubular.com> to test your regular expression. The regular expression returns the **last** +match found in the output. + ### Test coverage examples Use this regex for commonly used test tools. @@ -338,7 +364,7 @@ https://gitlab.example.com/<namespace>/<project>/badges/<branch>/pipeline.svg?ig ### Test coverage report badge -You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request) +You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request-deprecated) that each job log is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index 77a666c0cca..fbdf226181b 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -12,7 +12,7 @@ Use this document to get started with [GitLab CI/CD](../index.md). Before you start, make sure you have: - A project in GitLab that you would like to use CI/CD for. -- The [Maintainer or Owner role](../../user/permissions.md) for the project. +- The Maintainer or Owner role for the project. If you are migrating from another CI/CD tool, view this documentation: diff --git a/doc/ci/resource_groups/index.md b/doc/ci/resource_groups/index.md index d31fb5561e9..9312f4a8850 100644 --- a/doc/ci/resource_groups/index.md +++ b/doc/ci/resource_groups/index.md @@ -58,7 +58,7 @@ can still run `build` jobs concurrently for maximizing the pipeline efficiency. - The basic knowledge of the [GitLab CI/CD pipelines](../pipelines/index.md) - The basic knowledge of the [GitLab Environments and Deployments](../environments/index.md) -- [Developer role](../../user/permissions.md) (or above) in the project to configure CI/CD pipelines. +- At least the Developer role for the project to configure CI/CD pipelines. ### Limitations diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 37005939eb7..ed29b33bc3a 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -1,8 +1,7 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Review Apps **(FREE)** @@ -73,7 +72,7 @@ you can copy and paste into `.gitlab-ci.yml` as a starting point. Prerequisite: -- You need at least the Developer [role](../../user/permissions.md) for the project. +- You need at least the Developer role for the project. To use the Review Apps template: diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index d826b28acce..60e21653a45 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -160,7 +160,7 @@ the GitLab instance. To determine this: ### Determine the IP address of a specific runner To can find the IP address of a runner for a specific project, -you must have the [Owner role](../../user/permissions.md#project-members-permissions) for the +you must have the Owner role for the project. 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. @@ -187,7 +187,7 @@ the appropriate dependencies to run Rails test suites. When you [register a runner](https://docs.gitlab.com/runner/register/), its default behavior is to **only pick** [tagged jobs](../yaml/index.md#tags). -To change this, you must have the [Owner role](../../user/permissions.md#project-members-permissions) for the project. +To change this, you must have the Owner role for the project. To make a runner pick untagged jobs: @@ -291,6 +291,7 @@ globally or for individual jobs: - [`GIT_CHECKOUT`](#git-checkout) - [`GIT_CLEAN_FLAGS`](#git-clean-flags) - [`GIT_FETCH_EXTRA_FLAGS`](#git-fetch-extra-flags) +- [`GIT_SUBMODULE_UPDATE_FLAGS`](#git-submodule-update-flags) - [`GIT_DEPTH`](#shallow-cloning) (shallow cloning) - [`GIT_CLONE_PATH`](#custom-build-directories) (custom build directories) - [`TRANSFER_METER_FREQUENCY`](#artifact-and-cache-settings) (artifact/cache meter update frequency) @@ -331,10 +332,6 @@ try to preserve worktrees and try to re-use them by default. This has limitations when using the [Docker Machine executor](https://docs.gitlab.com/runner/executors/docker_machine.html). -It does not work for [the `kubernetes` executor](https://docs.gitlab.com/runner/executors/kubernetes.html), -but a [feature proposal](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3847) exists. -The `kubernetes` executor always clones into an temporary directory. - A Git strategy of `none` also re-uses the local working copy, but skips all Git operations normally done by GitLab. GitLab Runner pre-clone scripts are also skipped, if present. This strategy could mean you need to add `fetch` and `checkout` commands @@ -382,6 +379,8 @@ For this feature to work correctly, the submodules must be configured - a relative path to another repository on the same GitLab server. See the [Git submodules](../git_submodules.md) documentation. +You can provide additional flags to control advanced behavior using [`GIT_SUBMODULE_UPDATE_FLAGS`](#git-submodule-update-flags). + ### Git checkout > Introduced in GitLab Runner 9.3. @@ -442,14 +441,14 @@ script: > [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4142) in GitLab Runner 13.1. -The `GIT_FETCH_EXTRA_FLAGS` variable is used to control the behavior of +Use the `GIT_FETCH_EXTRA_FLAGS` variable to control the behavior of `git fetch`. You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. `GIT_FETCH_EXTRA_FLAGS` accepts all options of the [`git fetch`](https://git-scm.com/docs/git-fetch) command. However, `GIT_FETCH_EXTRA_FLAGS` flags are appended after the default flags that can't be modified. The default flags are: -- [GIT_DEPTH](#shallow-cloning). +- [`GIT_DEPTH`](#shallow-cloning). - The list of [refspecs](https://git-scm.com/book/en/v2/Git-Internals-The-Refspec). - A remote called `origin`. @@ -475,6 +474,47 @@ git fetch origin $REFSPECS --depth 50 --prune Where `$REFSPECS` is a value provided to the runner internally by GitLab. +### Git submodule update flags + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3192) in GitLab Runner 14.8. + +Use the `GIT_SUBMODULE_UPDATE_FLAGS` variable to control the behavior of `git submodule update` +when [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy) is set to either `normal` or `recursive`. +You can set it globally or per-job in the [`variables`](../yaml/index.md#variables) section. + +`GIT_SUBMODULE_UPDATE_FLAGS` accepts all options of the +[`git submodule update`](https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-update--init--remote-N--no-fetch--no-recommend-shallow-f--force--checkout--rebase--merge--referenceltrepositorygt--depthltdepthgt--recursive--jobsltngt--no-single-branch--ltpathgt82308203) +subcommand. However, note that `GIT_SUBMODULE_UPDATE_FLAGS` flags are appended after a few default flags: + +- `--init`, if [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy) was set to `normal` or `recursive`. +- `--recursive`, if [`GIT_SUBMODULE_STRATEGY`](#git-submodule-strategy) was set to `recursive`. +- [`GIT_DEPTH`](#shallow-cloning). See the default value below. + +Git honors the last occurrence of a flag in the list of arguments, so manually +providing them in `GIT_SUBMODULE_UPDATE_FLAGS` will also override these default flags. + +You can use this variable to fetch the latest remote `HEAD` instead of the commit tracked, +in the repository, or to speed up the checkout by fetching submodules in multiple parallel jobs: + +```yaml +variables: + GIT_SUBMODULE_STRATEGY: recursive + GIT_SUBMODULE_UPDATE_FLAGS: --remote --jobs 4 +script: + - ls -al .git/modules/ +``` + +The configuration above results in `git submodule update` being called this way: + +```shell +git submodule update --init --depth 50 --recursive --remote --jobs 4 +``` + +WARNING: +You should be aware of the implications for the security, stability, and reproducibility of +your builds when using the `--remote` flag. In most cases, it is better to explicitly track +submodule commits as designed, and update them using an auto-remediation/dependency bot. + ### Shallow cloning > Introduced in GitLab 8.9 as an experimental feature. diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index f76a767abec..7cfd8e50f6c 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -55,7 +55,7 @@ To enable shared runners: ### Disable shared runners You can disable shared runners for individual projects or for groups. -You must have the [Owner role](../../user/permissions.md#group-members-permissions) for the project +You must have the Owner role for the project or group. To disable shared runners for a project: @@ -144,7 +144,7 @@ Group runners process jobs by using a first in, first out ([FIFO](https://en.wik ### Create a group runner You can create a group runner for your self-managed GitLab instance or for GitLab.com. -You must have the [Owner role](../../user/permissions.md#group-members-permissions) for the group. +You must have the Owner role for the group. To create a group runner: @@ -160,7 +160,7 @@ To create a group runner: You can view and manage all runners for a group, its subgroups, and projects. You can do this for your self-managed GitLab instance or for GitLab.com. -You must have the [Owner role](../../user/permissions.md#group-members-permissions) for the group. +You must have the Owner role for the group. 1. Go to the group where you want to view the runners. 1. Go to **Settings > CI/CD** and expand the **Runners** section. @@ -183,7 +183,7 @@ From this page, you can edit, pause, and remove runners from the group, its subg ### Pause or remove a group runner You can pause or remove a group runner for your self-managed GitLab instance or for GitLab.com. -You must have the [Owner role](../../user/permissions.md#group-members-permissions) for the group. +You must have the Owner role for the group. 1. Go to the group you want to remove or pause the runner for. 1. Go to **Settings > CI/CD** and expand the **Runners** section. @@ -213,7 +213,7 @@ A fork *does* copy the CI/CD settings of the cloned repository. ### Create a specific runner You can create a specific runner for your self-managed GitLab instance or for GitLab.com. -You must have the [Owner role](../../user/permissions.md#project-members-permissions) for the project. +You must have the Owner role for the project. To create a specific runner: @@ -227,7 +227,7 @@ To create a specific runner: A specific runner is available in the project it was created for. An administrator can enable a specific runner to apply to additional projects. -- You must have the [Owner role](../../user/permissions.md#group-members-permissions) for the +- You must have the Owner role for the project. - The specific runner must not be [locked](#prevent-a-specific-runner-from-being-enabled-for-other-projects). diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 40c4deb51aa..bfe408f4485 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -12,7 +12,7 @@ Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a build environment. -SaaS runners on macOS are in [Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) and shouldn't be relied upon for mission-critical production jobs. ## Quickstart diff --git a/doc/ci/runners/saas/windows_saas_runner.md b/doc/ci/runners/saas/windows_saas_runner.md index b08be14dbc3..dddb3afee7c 100644 --- a/doc/ci/runners/saas/windows_saas_runner.md +++ b/doc/ci/runners/saas/windows_saas_runner.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # SaaS runners on Windows (beta) **(FREE SAAS)** -SaaS runners on Windows are in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta) +SaaS runners on Windows are in [beta](../../../policy/alpha-beta-support.md#beta-features) and shouldn't be used for production workloads. During this beta period, the [shared runner quota for CI/CD minutes](../../pipelines/cicd_minutes.md) @@ -15,14 +15,14 @@ change when the beta period ends, as discussed in this [related issue](https://g Windows runners on GitLab.com autoscale by launching virtual machines on the Google Cloud Platform. This solution uses an -[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/tree/master/docs/readme.md) +[autoscaling driver](https://gitlab.com/gitlab-org/ci-cd/custom-executor-drivers/autoscaler/-/blob/main/docs/README.md) developed by GitLab for the [custom executor](https://docs.gitlab.com/runner/executors/custom.html). Windows runners execute your CI/CD jobs on `n1-standard-2` instances with 2 vCPUs and 7.5 GB RAM. You can find a full list of available Windows packages in the [package documentation](https://gitlab.com/gitlab-org/ci-cd/shared-runners/images/gcp/windows-containers/blob/main/cookbooks/preinstalled-software/README.md). We want to keep iterating to get Windows runners in a stable state and -[generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). +[generally available](../../../policy/alpha-beta-support.md#generally-available-ga). You can follow our work towards this goal in the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/2162). @@ -127,7 +127,7 @@ test: ## Limitations and known issues - All the limitations mentioned in our [beta - definition](https://about.gitlab.com/handbook/product/#beta). + definition](../../../policy/alpha-beta-support.md#beta-features). - The average provisioning time for a new Windows VM is 5 minutes. This means that you may notice slower build start times on the Windows runner fleet during the beta. In a future diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index c0a763c80f0..ea0c0d9cc84 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -1,6 +1,6 @@ --- -stage: Configure -group: Configure +stage: Verify +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: concepts, howto --- diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index 51df8c102f7..d14027cb1ef 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -260,10 +260,12 @@ test: | `entrypoint` | no | 9.4 |Command or script to execute as the container's entrypoint. It's translated to Docker's `--entrypoint` option while creating the container. The syntax is similar to [`Dockerfile`'s `ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#entrypoint) directive, where each shell token is a separate string in the array. | | `command` | no | 9.4 |Command or script that should be used as the container's command. It's translated to arguments passed to Docker after the image's name. The syntax is similar to [`Dockerfile`'s `CMD`](https://docs.docker.com/engine/reference/builder/#cmd) directive, where each shell token is a separate string in the array. | | `alias` (1) | no | 9.4 | Additional alias that can be used to access the service from the job's container. Read [Accessing the services](#accessing-the-services) for more information. | -| `variables` | no | 14.5 | Additional environment variables that are passed exclusively to the service. The syntax is the same as [Job Variables](../variables/index.md). | +| `variables` (2) | no | 14.5 | Additional environment variables that are passed exclusively to the service. The syntax is the same as [Job Variables](../variables/index.md). | (1) Alias support for the Kubernetes executor was [introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2229) in GitLab Runner 12.8, and is only available for Kubernetes version 1.7 or later. +(2) Service variables support for the Docker and the Kubernetes executor was [introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3158) in GitLab Runner 14.8. + ## Starting multiple services from the same image > Introduced in GitLab and GitLab Runner 9.4. Read more about the [extended configuration options](../docker/using_docker_images.md#extended-docker-configuration-options). @@ -387,7 +389,7 @@ time. ## Debug a job locally The following commands are run without root privileges. You should be -able to run Docker with your regular user account. +able to run Docker with your user account. First start with creating a file named `build_script`: diff --git a/doc/ci/test_cases/index.md b/doc/ci/test_cases/index.md index 384bfc10779..3ef25a4924b 100644 --- a/doc/ci/test_cases/index.md +++ b/doc/ci/test_cases/index.md @@ -20,7 +20,7 @@ use external test planning tools, which require additional overhead, context swi Prerequisite: -- You must have at least the Reporter [role](../../user/permissions.md). +- You must have at least the Reporter role. To create a test case in a GitLab project: @@ -36,7 +36,7 @@ issue list with a search query, including labels or the test case's title. Prerequisite: -- You must have at least the Guest [role](../../user/permissions.md). +- You must have at least the Guest role. To view a test case: @@ -51,7 +51,7 @@ You can edit a test case's title and description. Prerequisite: -- You must have at least the Reporter [role](../../user/permissions.md). +- You must have at least the Reporter role. - Users demoted to the Guest role can continue to edit the test cases they created when they were in the higher role. @@ -68,7 +68,7 @@ When you want to stop using a test case, you can archive it. You can [reopen an Prerequisite: -- You must have at least the Reporter [role](../../user/permissions.md). +- You must have at least the Reporter role. To archive a test case, on the test case's page, select the **Archive test case** button. @@ -81,6 +81,6 @@ To view archived test cases: If you decide to start using an archived test case again, you can reopen it. -You must have at least the Reporter [role](../../user/permissions.md). +You must have at least the Reporter role. To reopen an archived test case, on the test case's page, select **Reopen test case**. diff --git a/doc/ci/triggers/index.md b/doc/ci/triggers/index.md index 1b648a486ef..2c916ba092c 100644 --- a/doc/ci/triggers/index.md +++ b/doc/ci/triggers/index.md @@ -22,7 +22,7 @@ to authenticate an API call. The token impersonates a user's project access and Prerequisite: -- You must have at least the [Maintainer role](../../user/permissions.md) for the project. +- You must have at least the Maintainer role for the project. To create a trigger token: diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 4d550f6da13..81cb924532c 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -36,7 +36,7 @@ file is correct. Paste in full `.gitlab-ci.yml` files or individual jobs configu to verify the basic syntax. When a `.gitlab-ci.yml` file is present in a project, you can also use the CI Lint -tool to [simulate the creation of a full pipeline](lint.md#pipeline-simulation). +tool to [simulate the creation of a full pipeline](lint.md#simulate-a-pipeline). It does deeper verification of the configuration syntax. ## Verify variables @@ -69,12 +69,12 @@ if you are using that type: and run separate pipelines in the same project. You can also [dynamically generate the child pipeline's configuration](pipelines/parent_child_pipelines.md#dynamic-child-pipelines) at runtime. -- [Pipelines for Merge Requests](pipelines/merge_request_pipelines.md): Run a pipeline +- [Merge request pipelines](pipelines/merge_request_pipelines.md): Run a pipeline in the context of a merge request. - - [Pipelines for Merge Results](pipelines/pipelines_for_merged_results.md): - Pipelines for merge requests that run on the combined source and target branch - - [Merge Trains](pipelines/merge_trains.md): - Multiple pipelines for merged results that queue and run automatically before + - [Merged results pipelines](pipelines/merged_results_pipelines.md): + Merge request pipelines that run on the combined source and target branch + - [Merge trains](pipelines/merge_trains.md): + Multiple merged results pipelines that queue and run automatically before changes are merged. ### Troubleshooting Guides for CI/CD features @@ -173,8 +173,8 @@ a branch to its remote repository. To illustrate the problem, suppose you've had This occurs because the previous pipeline cannot find a checkout-SHA (which is associated with the pipeline record) from the `example` branch that the commit history has already been overwritten by the force-push. -Similarly, [Pipelines for merged results](pipelines/pipelines_for_merged_results.md) -might have failed intermittently due to [the same reason](pipelines/pipelines_for_merged_results.md#intermittently-pipelines-fail-by-fatal-reference-is-not-a-tree-error). +Similarly, [Merged results pipelines](pipelines/merged_results_pipelines.md) +might have failed intermittently due to [the same reason](pipelines/merged_results_pipelines.md#pipelines-fail-intermittently-with-a-fatal-reference-is-not-a-tree-error). As of GitLab 12.4, we've improved this behavior by persisting pipeline refs exclusively. To illustrate its life cycle: @@ -192,6 +192,21 @@ To illustrate its life cycle: The merge request pipeline widget shows information about the pipeline status in a merge request. It's displayed above the [ability to merge status widget](#merge-request-status-messages). +#### "Checking ability to merge automatically" message + +There is a [known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/229352) +where a merge request can be stuck with the `Checking ability to merge automatically` +message. + +If your merge request has this message and it does not disappear after a few minutes, +you can try one of these workarounds: + +- Refresh the merge request page. +- Close & Re-open the merge request. +- Rebase the merge request with the `/rebase` [quick action](../user/project/quick_actions.md). +- If you have already confirmed the merge request is ready to be merged, you can merge + it with the `/merge` quick action. + #### "Checking pipeline status" message This message is shown when the merge request has no pipeline associated with the @@ -226,10 +241,10 @@ This also applies if the pipeline has not been created yet, or if you are waitin for an external CI service. If you don't use pipelines for your project, then you should disable **Pipelines must succeed** so you can accept merge requests. -### "The pipeline for this merge request did not complete. Push a new commit to fix the failure or check the troubleshooting documentation to see other possible actions." message +#### "Merge blocked: pipeline must succeed. Push a new commit that fixes the failure" message This message is shown if the [merge request pipeline](pipelines/merge_request_pipelines.md), -[merged results pipeline](pipelines/pipelines_for_merged_results.md), +[merged results pipeline](pipelines/merged_results_pipelines.md), or [merge train pipeline](pipelines/merge_trains.md) has failed or been canceled. diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index 55fd8c1eb49..e2de47c6c62 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -1,8 +1,7 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Unit test reports **(FREE)** @@ -21,7 +20,7 @@ report on the merge request so that it's easier and faster to identify the failure without having to check the entire log. Unit test reports currently only support test reports in the JUnit report format. -If you don't use Merge Requests but still want to see the unit test report +If you don't use merge requests but still want to see the unit test report output without searching through job logs, the full [Unit test reports](#viewing-unit-test-reports-on-gitlab) are available in the pipeline detail view. @@ -67,7 +66,7 @@ execution time and the error output. ### Number of recent failures -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in Merge Requests in GitLab 13.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in merge requests in GitLab 13.7. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268249) in GitLab 13.8. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235525) in Test Reports in GitLab 13.9. @@ -76,7 +75,7 @@ If a test failed in the project's default branch in the last 14 days, a message ## How to set it up -To enable the Unit test reports in merge requests, you need to add +To enable the Unit test reports in merge requests, you must add [`artifacts:reports:junit`](yaml/artifacts_reports.md#artifactsreportsjunit) in `.gitlab-ci.yml`, and specify the path(s) of the generated test reports. The reports must be `.xml` files, otherwise [GitLab returns an Error 500](https://gitlab.com/gitlab-org/gitlab/-/issues/216575). diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 7ce58b015ca..ba8451110eb 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -154,7 +154,7 @@ job: ### Add a CI/CD variable to a project You can add CI/CD variables to a project's settings. Only project members with the -[Maintainer role](../../user/permissions.md#project-members-permissions) +Maintainer role can add or update project CI/CD variables. To keep a CI/CD variable secret, put it in the project settings, not in the `.gitlab-ci.yml` file. @@ -233,7 +233,7 @@ inherited. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/299879) in GitLab 13.11. To make a CI/CD variable available to all projects and groups in a GitLab instance, -add an instance CI/CD variable. You must have the [Administrator role](../../user/permissions.md). +add an instance CI/CD variable. You must have administrator access. You can define instance variables via the UI or [API](../../api/instance_level_ci_variables.md). @@ -372,7 +372,7 @@ You can protect a project, group or instance CI/CD variable so it is only passed to pipelines running on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). -[Pipelines for merge requests](../pipelines/merge_request_pipelines.md) do not have access to protected variables. +[Merge request pipelines](../pipelines/merge_request_pipelines.md) do not have access to protected variables. An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/28002) regarding this limitation. To protect a variable: @@ -651,9 +651,11 @@ which variables take precedence. The order of precedence for variables is (from highest to lowest): -1. [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call), - [scheduled pipeline variables](../pipelines/schedules.md#using-variables), - and [manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). +1. These all have the same (highest) precedence: + - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). + - [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). + - [Manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). + - Variables added when [creating a pipeline with the API](../../api/pipelines.md#create-a-new-pipeline). 1. Project [variables](#custom-cicd-variables). 1. Group [variables](#add-a-cicd-variable-to-a-group). 1. Instance [variables](#add-a-cicd-variable-to-an-instance). @@ -900,7 +902,7 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/292661) in GitLab 13.8. You can restrict access to debug logging. When restricted, only users with -[developer or higher permissions](../../user/permissions.md#project-members-permissions) +at least the Developer role can view job logs when debug logging is enabled with a variable in: - The [`.gitlab-ci.yml` file](#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file). diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 60888cd9b97..0f3461b3674 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -27,7 +27,7 @@ There are also a number of [variables you can use to configure runner behavior]( | `CI_API_V4_URL` | 11.7 | all | The GitLab API v4 root URL. | | `CI_BUILDS_DIR` | all | 11.10 | The top-level directory where builds are executed. | | `CI_COMMIT_AUTHOR` | 13.11 | all | The author of the commit in `Name <email>` format. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in pipelines for merge requests. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch. Is always `0000000000000000000000000000000000000000` in merge request pipelines. | | `CI_COMMIT_BRANCH` | 12.6 | 0.5 | The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines. | | `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit. If the title is shorter than 100 characters, the message without the first line. | | `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | @@ -63,7 +63,7 @@ There are also a number of [variables you can use to configure runner behavior]( | `CI_JOB_IMAGE` | 12.9 | 12.9 | The name of the Docker image running the job. | | `CI_JOB_JWT` | 12.10 | all | A RS256 JSON web token to authenticate with third party systems that support JWT authentication, for example [HashiCorp's Vault](../secrets/index.md). | | `CI_JOB_JWT_V1` | 14.6 | all | The same value as `CI_JOB_JWT`. | -| `CI_JOB_JWT_V2` | 14.6 | all | [**alpha:**](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). | +| `CI_JOB_JWT_V2` | 14.6 | all | [**alpha:**](../../policy/alpha-beta-support.md#alpha-features) A newly formatted RS256 JSON web token to increase compatibility. Similar to `CI_JOB_JWT`, except the issuer (`iss`) claim is changed from `gitlab.com` to `https://gitlab.com`, `sub` has changed from `job_id` to a string that contains the project path, and an `aud` claim is added. Format is subject to change. Be aware, the `aud` field is a constant value. Trusting JWTs in multiple relying parties can lead to [one RP sending a JWT to another one and acting maliciously as a job](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/72555#note_769112331). | | `CI_JOB_MANUAL` | 8.12 | all | `true` if a job was started manually. | | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job. | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the job's stage. | @@ -148,12 +148,12 @@ These variables are available when: | `CI_MERGE_REQUEST_PROJECT_URL` | 11.6 | all | The URL of the project of the merge request. For example, `http://192.168.10.15:3000/namespace/awesome-project`. | | `CI_MERGE_REQUEST_REF_PATH` | 11.6 | all | The ref path of the merge request. For example, `refs/merge-requests/1/head`. | | `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME` | 11.6 | all | The source branch name of the merge request. | -| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../pipelines/pipelines_for_merged_results.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the source branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../pipelines/merged_results_pipelines.md). | | `CI_MERGE_REQUEST_SOURCE_PROJECT_ID` | 11.6 | all | The ID of the source project of the merge request. | | `CI_MERGE_REQUEST_SOURCE_PROJECT_PATH` | 11.6 | all | The path of the source project of the merge request. | | `CI_MERGE_REQUEST_SOURCE_PROJECT_URL` | 11.6 | all | The URL of the source project of the merge request. | | `CI_MERGE_REQUEST_TARGET_BRANCH_NAME` | 11.6 | all | The target branch name of the merge request. | -| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../pipelines/pipelines_for_merged_results.md). **(PREMIUM)** | +| `CI_MERGE_REQUEST_TARGET_BRANCH_SHA` | 11.9 | all | The HEAD SHA of the target branch of the merge request. The variable is empty in merge request pipelines. The SHA is present only in [merged results pipelines](../pipelines/merged_results_pipelines.md). | | `CI_MERGE_REQUEST_TITLE` | 11.9 | all | The title of the merge request. | | `CI_MERGE_REQUEST_EVENT_TYPE` | 12.3 | all | The event type of the merge request. Can be `detached`, `merged_result` or `merge_train`. | | `CI_MERGE_REQUEST_DIFF_ID` | 13.7 | all | The version of the merge request diff. | diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 25fb861cfb7..e010dd21b9e 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 3c94ddb3c14..34db6c61d0b 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -217,6 +217,69 @@ default: - echo "Job complete." ``` +### Use nested includes with duplicate `includes` entries + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28987) in GitLab 14.8 + +Nested includes can include the same configuration file. The duplicate configuration +file is included multiple times, but the effect is the same as if it was only +included once. + +For example, with the following nested includes, where `defaults.gitlab-ci.yml` +is included multiple times: + +- Contents of the `.gitlab-ci.yml` file: + + ```yaml + include: + - template: defaults.gitlab-ci.yml + - local: unit-tests.gitlab-ci.yml + - local: smoke-tests.gitlab-ci.yml + ``` + +- Contents of the `defaults.gitlab-ci.yml` file: + + ```yaml + default: + before_script: default-before-script.sh + retry: 2 + ``` + +- Contents of the `unit-tests.gitlab-ci.yml` file: + + ```yaml + include: + - template: defaults.gitlab-ci.yml + + unit-test-job: + script: unit-test.sh + retry: 0 + ``` + +- Contents of the `smoke-tests.gitlab-ci.yml` file: + + ```yaml + include: + - template: defaults.gitlab-ci.yml + + smoke-test-job: + script: smoke-test.sh + ``` + +The final configuration would be: + +```yaml +unit-test-job: + before_script: default-before-script.sh + script: unit-test.sh + retry: 0 + +smoke-test-job: + before_script: default-before-script.sh + script: smoke-test.sh + retry: 2 +``` + ## Use variables with `include` > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/284883) in GitLab 13.8. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index 4530e2675c4..d9ea5498b63 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -415,7 +415,7 @@ and the pipeline is for either: - You can use the [`workflow:rules` templates](workflow.md#workflowrules-templates) to import a preconfigured `workflow: rules` entry. - [Common `if` clauses for `workflow:rules`](workflow.md#common-if-clauses-for-workflowrules). -- [Use `rules` to run pipelines for merge requests](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). +- [Use `rules` to run merge request pipelines](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). #### `workflow:rules:variables` @@ -939,7 +939,7 @@ rspec: Use `artifacts:untracked` to add all Git untracked files as artifacts (along with the paths defined in `artifacts:paths`). `artifacts:untracked` ignores configuration -in the repository's `.gitignore` file. +in the repository's `.gitignore`, so matching artifacts in `.gitignore` are included. **Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). @@ -974,7 +974,7 @@ failure. - `on_success` (default): Upload artifacts only when the job succeeds. - `on_failure`: Upload artifacts only when the job fails. -- `always`: Always upload artifacts. For example, when +- `always`: Always upload artifacts (except when jobs time out). For example, when [uploading artifacts](../unit_test_reports.md#viewing-junit-screenshots-on-gitlab) required to troubleshoot failing tests. @@ -1322,12 +1322,14 @@ Use `coverage` with a custom regular expression to configure how code coverage is extracted from the job output. The coverage is shown in the UI if at least one line in the job output matches the regular expression. -To extract the code coverage value in the matching line, GitLab uses this -regular expression: `\d+(\.\d+)?`. +To extract the code coverage value from the match, GitLab uses +this smaller regular expression: `\d+(\.\d+)?`. **Possible inputs**: -- A regular expression. Must start and end with `/`. +- A regular expression. Must start and end with `/`. Must match the coverage number. + May match surrounding text as well, so you don't need to use a regular expression character group + to capture the exact number. **Example of `coverage`**: @@ -1339,14 +1341,20 @@ job1: In this example: -1. GitLab checks the job log for a line that matches the regular expression. A line - like `Code coverage: 67.89` would match. -1. GitLab then checks the line to find a match to `\d+(\.\d+)?`. The sample matching - line above gives a code coverage of `67.89`. +1. GitLab checks the job log for a match with the regular expression. A line + like `Code coverage: 67.89% of lines covered` would match. +1. GitLab then checks the matched fragment to find a match to `\d+(\.\d+)?`. + The sample matching line above gives a code coverage of `67.89`. **Additional details**: -- If there is more than one matched line in the job output, the last line is used. +- Coverage regular expressions set in `gitlab-ci.yml` take precedence over coverage regular expression set in the + [GitLab UI](../pipelines/settings.md#add-test-coverage-results-to-a-merge-request-deprecated). +- If there is more than one matched line in the job output, the last line is used + (the first result of reverse search). +- If there are multiple matches in a single line, the last match is searched + for the coverage number. +- If there are multiple coverage numbers found in the matched fragment, the first number is used. - Leading zeros are removed. - Coverage output from [child pipelines](../pipelines/parent_child_pipelines.md) is not recorded or displayed. Check [the related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/280818) @@ -1467,7 +1475,7 @@ Use `environment` to define the [environment](../environments/index.md) that a j formats: - Plain text, including letters, digits, spaces, and these characters: `-`, `_`, `/`, `$`, `{`, `}`. -- CI/CD variables, including predefined, secure, or variables defined in the +- CI/CD variables, including predefined, project, group, instance, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. **Example of `environment`**: @@ -1496,7 +1504,7 @@ Common environment names are `qa`, `staging`, and `production`, but you can use formats: - Plain text, including letters, digits, spaces, and these characters: `-`, `_`, `/`, `$`, `{`, `}`. -- CI/CD variables, including predefined, secure, or variables defined in the +- CI/CD variables, including predefined, project, group, instance, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. **Example of `environment:name`**: @@ -1518,7 +1526,7 @@ Set a URL for an [environment](../environments/index.md). **Possible inputs**: A single URL, in one of these formats: - Plain text, like `https://prod.example.com`. -- CI/CD variables, including predefined, secure, or variables defined in the +- CI/CD variables, including predefined, project, group, instance, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. **Example of `environment:url`**: @@ -2366,7 +2374,7 @@ pipeline based on branch names or pipeline types. | `chat` | For pipelines created by using a [GitLab ChatOps](../chatops/index.md) command. | | `external` | When you use CI services other than GitLab. | | `external_pull_requests` | When an external pull request on GitHub is created or updated (See [Pipelines for external pull requests](../ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests)). | - | `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/pipelines_for_merged_results.md), and [merge trains](../pipelines/merge_trains.md). | + | `merge_requests` | For pipelines created when a merge request is created or updated. Enables [merge request pipelines](../pipelines/merge_request_pipelines.md), [merged results pipelines](../pipelines/merged_results_pipelines.md), and [merge trains](../pipelines/merge_trains.md). | | `pipelines` | For [multi-project pipelines](../pipelines/multi_project_pipelines.md) created by [using the API with `CI_JOB_TOKEN`](../pipelines/multi_project_pipelines.md#create-multi-project-pipelines-by-using-the-api), or the [`trigger`](#trigger) keyword. | | `pushes` | For pipelines triggered by a `git push` event, including for branches and tags. | | `schedules` | For [scheduled pipelines](../pipelines/schedules.md). | @@ -2465,7 +2473,7 @@ Use `changes` in pipelines with the following refs: - `branches` - `external_pull_requests` -- `merge_requests` (see additional details about [using `only:changes` with pipelines for merge requests](../jobs/job_control.md#use-onlychanges-with-pipelines-for-merge-requests)) +- `merge_requests` (see additional details about [using `only:changes` with merge request pipelines](../jobs/job_control.md#use-onlychanges-with-merge-request-pipelines)) **Keyword type**: Job keyword. You can use it only as part of a job. @@ -2507,7 +2515,7 @@ docker build: - [`only: changes` and `except: changes` examples](../jobs/job_control.md#onlychanges--exceptchanges-examples). - If you use `changes` with [only allow merge requests to be merged if the pipeline succeeds](../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds), - you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-pipelines-for-merge-requests). + you should [also use `only:merge_requests`](../jobs/job_control.md#use-onlychanges-with-merge-request-pipelines). - [Jobs or pipelines can run unexpectedly when using `only: changes`](../jobs/job_control.md#jobs-or-pipelines-run-unexpectedly-when-using-changes). #### `only:kubernetes` / `except:kubernetes` @@ -2654,6 +2662,7 @@ deploystacks: [vultr, processing] - [Run a one-dimensional matrix of parallel jobs](../jobs/job_control.md#run-a-one-dimensional-matrix-of-parallel-jobs). - [Run a matrix of triggered parallel jobs](../jobs/job_control.md#run-a-matrix-of-parallel-trigger-jobs). +- [Select different runner tags for each parallel matrix job](../jobs/job_control.md#select-different-runner-tags-for-each-parallel-matrix-job). ### `release` @@ -3079,7 +3088,7 @@ job: - [Common `if` expressions for `rules`](../jobs/job_control.md#common-if-clauses-for-rules). - [Avoid duplicate pipelines](../jobs/job_control.md#avoid-duplicate-pipelines). -- [Use `rules` to run pipelines for merge requests](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). +- [Use `rules` to run merge request pipelines](../pipelines/merge_request_pipelines.md#use-rules-to-add-jobs). #### `rules:changes` @@ -3569,6 +3578,7 @@ In this example, only runners with *both* the `ruby` and `postgres` tags can run **Related topics**: - [Use tags to control which jobs a runner can run](../runners/configure_runners.md#use-tags-to-control-which-jobs-a-runner-can-run). +- [Select different runner tags for each parallel matrix job](../jobs/job_control.md#select-different-runner-tags-for-each-parallel-matrix-job). ### `timeout` @@ -3642,7 +3652,7 @@ trigger_job: - Jobs with `trigger` can only use a [limited set of keywords](../pipelines/multi_project_pipelines.md#define-multi-project-pipelines-in-your-gitlab-ciyml-file). For example, you can't run commands with [`script`](#script), [`before_script`](#before_script), - or [`after_script`](#after_script). + or [`after_script`](#after_script). Also, [`environment`](#environment) is not supported with `trigger`. - You [cannot use the API to start `when:manual` trigger jobs](https://gitlab.com/gitlab-org/gitlab/-/issues/284086). - In [GitLab 13.5 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/201938), you can use [`when:manual`](#when) in the same job as `trigger`. In GitLab 13.4 and diff --git a/doc/ci/yaml/script.md b/doc/ci/yaml/script.md index ca1e79c2395..4bffcbca1cc 100644 --- a/doc/ci/yaml/script.md +++ b/doc/ci/yaml/script.md @@ -4,7 +4,7 @@ group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# GitLab CI/CD script syntax **(FREE)** +# Format scripts and job logs **(FREE)** You can use special syntax in [`script`](index.md#script) sections to: diff --git a/doc/ci/yaml/workflow.md b/doc/ci/yaml/workflow.md index 332214638d8..b46d504edfa 100644 --- a/doc/ci/yaml/workflow.md +++ b/doc/ci/yaml/workflow.md @@ -126,7 +126,7 @@ makes your pipelines run for branches and tags. Branch pipeline status is displayed in merge requests that use the branch as a source. However, this pipeline type does not support any features offered by [merge request pipelines](../pipelines/merge_request_pipelines.md), like -[pipelines for merged results](../pipelines/pipelines_for_merged_results.md) +[merged results pipelines](../pipelines/merged_results_pipelines.md) or [merge trains](../pipelines/merge_trains.md). This template intentionally avoids those features. @@ -140,7 +140,7 @@ include: The [`MergeRequest-Pipelines` template](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Workflows/MergeRequest-Pipelines.gitlab-ci.yml) makes your pipelines run for the default branch, tags, and all types of merge request pipelines. Use this template if you use any of the -the [pipelines for merge requests features](../pipelines/merge_request_pipelines.md). +the [merge request pipelines features](../pipelines/merge_request_pipelines.md). To [include](index.md#include) it: |
