diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-06-20 14:10:13 +0300 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-06-20 14:10:13 +0300 |
| commit | 0ea3fcec397b69815975647f5e2aa5fe944a8486 (patch) | |
| tree | 7979381b89d26011bcf9bdc989a40fcc2f1ed4ff /doc/ci | |
| parent | 72123183a20411a36d607d70b12d57c484394c8e (diff) | |
Add latest changes from gitlab-org/gitlab@15-1-stable-eev15.1.0-rc42
Diffstat (limited to 'doc/ci')
56 files changed, 1537 insertions, 1195 deletions
diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index 0aa7f157602..40b29ebb6ea 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -17,9 +17,7 @@ To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab, create a project: 1. On the top menu, select **Projects > Create new project**. 1. Select **Run CI/CD for external repository**. - 1. Select **Repo by URL**. - -  + 1. Select **Repository by URL**. GitLab imports the repository and enables [Pull Mirroring](../../user/project/repository/mirror/pull.md). diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index a906f213626..a928d315c6b 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -62,7 +62,7 @@ To manually enable GitLab CI/CD for your repository: `repo` so that GitLab can access your project and update commit statuses. 1. In GitLab, create a project: 1. On the top menu, select **Projects > Create new project**. - 1. Select **Run CI/CD for external repository** and **Repo by URL**. + 1. Select **Run CI/CD for external repository** and **Repository by URL**. 1. In the **Git repository URL** field, enter the HTTPS URL for your GitHub repository. If your project is private, use the personal access token you just created for authentication. 1. Fill in all the other fields and select **Create project**. diff --git a/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png b/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png Binary files differindex ac5be3c3058..163148d53fd 100644 --- a/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png +++ b/doc/ci/ci_cd_for_external_repos/img/bitbucket_app_password.png diff --git a/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png b/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png Binary files differindex 0a3476d9035..b2a5c1c1eb8 100644 --- a/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png +++ b/doc/ci/ci_cd_for_external_repos/img/bitbucket_webhook.png diff --git a/doc/ci/ci_cd_for_external_repos/img/external_repository.png b/doc/ci/ci_cd_for_external_repos/img/external_repository.png Binary files differdeleted file mode 100644 index b850d91f56b..00000000000 --- a/doc/ci/ci_cd_for_external_repos/img/external_repository.png +++ /dev/null diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 4e0cd7609cb..e3a8141ed88 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -18,21 +18,17 @@ external repository to get the benefits of GitLab CI/CD. Connecting an external repository sets up [repository mirroring](../../user/project/repository/mirror/index.md) and creates a lightweight project with issues, merge requests, wiki, and snippets disabled. These features -[can be re-enabled later](../../user/project/settings/index.md#sharing-and-permissions). +[can be re-enabled later](../../user/project/settings/index.md#configure-project-visibility-features-and-permissions). ## Connect to an external repository To connect to an external repository: -<!-- vale gitlab.Spelling = NO --> - 1. On the top bar, select **Menu > Projects > Create new project**. 1. Select **Run CI/CD for external repository**. -1. Select **GitHub** or **Repo by URL**. +1. Select **GitHub** or **Repository by URL**. 1. Complete the fields. -<!-- vale gitlab.Spelling = YES --> - ## Pipelines for external pull requests > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/65139) in GitLab 12.3. diff --git a/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md new file mode 100644 index 00000000000..9af5218e058 --- /dev/null +++ b/doc/ci/cloud_deployment/ecs/deploy_to_aws_ecs.md @@ -0,0 +1,253 @@ +--- +stage: Release +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Deploy to Amazon Elastic Container Service **(FREE)** + +This step-by-step guide helps you deploy a project hosted on GitLab.com to +the Amazon [Elastic Container Service (ECS)](https://aws.amazon.com/ecs/). + +In this guide, you begin by creating an ECS cluster manually using the AWS console. You create and +deploy a simple application that you create from a GitLab template. + +These instructions work for both SaaS and self-managed GitLab instances. +Ensure your own [runners are configured](../../runners/index.md). + +## Prerequisites + +- An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). + Sign in with an existing AWS account or create a new one. +- In this guide, you create an infrastructure in [`us-east-2` region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). + You can use any region, but do not change it after you begin. + +## Create an infrastructure and initial deployment on AWS + +For deploying an application from GitLab, you must first create an infrastructure and initial +deployment on AWS. +This includes an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +and related components, such as +[ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html), +[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html), +and containerized application image. + +For the first step here, you create a demo application from a project template. + +### Create a new project from a template + +Use a GitLab project template to get started. As the name suggests, these projects provide a +bare-bones application built on some well-known frameworks. + +1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select + **New project**. + +1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or + NodeJS Express project. For this guide, use the Ruby on Rails template. + +  + +1. Give your project a name. In this example, it's named `ecs-demo`. Make it public so that you can + take advantage of the features available in the + [GitLab Ultimate plan](https://about.gitlab.com/pricing/). + +1. Select **Create project**. + +Now that you created a demo project, you must containerize the application and push it to the +container registry. + +### Push a containerized application image to GitLab Container Registry + +[ECS](https://aws.amazon.com/ecs/) is a container orchestration service, meaning that you must +provide a containerized application image during the infrastructure build. To do so, you can use +GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) +and [Container Registry](../../../user/packages/container_registry/index.md). + +1. Go to **ecs-demo** project on GitLab. +1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` + creation form. +1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines + a pipeline for continuous deployment to ECS. + + ```yaml + include: + - template: AWS/Deploy-ECS.gitlab-ci.yml + ``` + +1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` + job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md). + +  + +1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been + pushed. + +  + +Now you have a containerized application image that can be pulled from AWS. Next, you define the +spec of how this application image is used in AWS. + +Note that the `production_ecs` job fails because ECS Cluster is not connected yet. You'll fix this +later. + +### Create an ECS task definition + +[ECS Task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) +is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). + +1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). +1. Select **Create new Task Definition**. + +  + +1. Choose **EC2** as the launch type. Select **Next Step**. +1. Set `ecs_demo` to **Task Definition Name**. +1. Set `512` to **Task Size > Task memory** and **Task CPU**. +1. Select **Container Definitions > Add container**. This opens a container registration form. +1. Set `web` to **Container name**. +1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. + Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). + +  + +1. Add a port mapping. Set `80` to **Host Port** and `5000` to **Container port**. + +  + +1. Select **Create**. + +Now you have the initial task definition. Next, you create an actual infrastructure to run the +application image. + +### Create an ECS cluster + +An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) +is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). +It's also associated with EC2 or Fargate as the computation resource. + +1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). +1. Select **Create Cluster**. +1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. +1. Set `ecs-demo` to **Cluster Name**. +1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) + in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. +1. Set all available subnets of the VPC to **Subnets**. +1. Select **Create**. +1. Make sure that the ECS cluster has been successfully created. + +  + +Now you can register an ECS service to the ECS cluster in the next step. + +Note the following: + +- Optionally, you can set a SSH key pair in the creation form. This allows you to SSH to the EC2 + instance for debugging. +- If you don't choose an existing VPC, it creates a new VPC by default. This could cause an error if + it reaches the maximum allowed number of internet gateways on your account. +- The cluster requires an EC2 instance, meaning it costs you [according to the instance-type](https://aws.amazon.com/ec2/pricing/on-demand/). + +### Create an ECS Service + +[ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) +is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). + +1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) +1. Select **Deploy**. This opens a service creation form. +1. Select `EC2` in **Launch Type**. +1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). +1. Set `ecs_demo` to **Service name**. +1. Set `1` to **Desired tasks**. + +  + +1. Select **Deploy**. +1. Make sure that the created service is active. + +  + +Note that AWS's console UI changes from time to time. If you can't find a relevant component in the +instructions, select the closest one. + +### View the demo application + +Now, the demo application is accessible from the internet. + +1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) +1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). +1. Select the ID of the EC2 instance. This brings you to the instance detail page. +1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application + running. + +  + +In this guide, HTTPS/SSL is **not** configured. You can access to the application through HTTP only +(for example, `http://<ec2-ipv4-address>`). + +## Setup Continuous Deployment from GitLab + +Now that you have an application running on ECS, you can set up continuous deployment from GitLab. + +### Create a new IAM user as a deployer + +For GitLab to access the ECS cluster, service, and task definition that you created above, You must +create a deployer user on AWS: + +1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). +1. Select **Add user**. +1. Set `ecs_demo` to **User name**. +1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. +1. Select `Attach existing policies directly` in **Set permissions**. +1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**. + +  + +1. Select **Create user**. +1. Take note of the **Access key ID** and **Secret access key** of the created user. + +NOTE: +Do not share the secret access key in a public place. You must save it in a secure place. + +### Setup credentials in GitLab to let pipeline jobs access to ECS + +You can register the access information in [GitLab Environment Variables](../../variables/index.md#custom-cicd-variables). +These variables are injected into the pipeline jobs and can access the ECS API. + +1. Go to **ecs-demo** project on GitLab. +1. Go to **Settings > CI/CD > Variables**. +1. Select **Add Variable** and set the following key-value pairs. + + |Key|Value|Note| + |---|---|---| + |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | + |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | + |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | + |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | + |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. | + |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | + +### Make a change to the demo application + +Change a file in the project and see if it's reflected in the demo application on ECS: + +1. Go to **ecs-demo** project on GitLab. +1. Open the file at **app > views > welcome > `index.html.erb`**. +1. Select **Edit**. +1. Change the text to `You're on ECS!`. +1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. +1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see + this: + +  + +Congratulations! You successfully set up continuous deployment to ECS. + +NOTE: +ECS deploy jobs wait for the rollout to complete before exiting. To disable this behavior, +set `CI_AWS_ECS_WAIT_FOR_ROLLOUT_COMPLETE_DISABLED` to a non-empty value. + +## Further reading + +- If you're interested in more of the continuous deployments to clouds, see [cloud deployments](../index.md). +- If you want to quickly set up DevSecOps in your project, see [Auto DevOps](../../../topics/autodevops/index.md). +- If you want to quickly set up the production-grade environment, see [the 5 Minute Production App](https://gitlab.com/gitlab-org/5-minute-production-app/deploy-template/-/blob/master/README.md). diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md index 5571c51ef27..7522bc80d0d 100644 --- a/doc/ci/cloud_deployment/ecs/quick_start_guide.md +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -1,250 +1,11 @@ --- -stage: Release -group: Release -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'deploy_to_aws_ecs.md' +remove_date: '2022-09-18' --- -# Getting started with Continuous Deployment to AWS Elastic Container Service **(FREE)** +This document was moved to [another location](deploy_to_aws_ecs.md). -This step-by-step guide helps you use [Continuous Deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs) -that deploys a project hosted on GitLab.com to [Elastic Container Service](https://aws.amazon.com/ecs/) -(ECS) on AWS. - -In this guide, you begin by creating an ECS cluster manually using the AWS console. You create and -deploy a simple application that you create from a GitLab template. - -These instructions work for both SaaS and self-managed GitLab instances. -Ensure your own [runners are configured](../../runners/index.md). - -## Prerequisites - -- An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/). - Sign in with an existing AWS account or create a new one. -- In this guide, you create an infrastructure in [`us-east-2` region](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). - You can use any region, but do not change it after you begin. - -## Create an infrastructure and initial deployment on AWS - -For deploying an application from GitLab, you must first create an infrastructure and initial -deployment on AWS. -This includes an [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) -and related components, such as -[ECS task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html), -[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html), -and containerized application image. - -For the first step here, you create a demo application from a project template. - -### Create a new project from a template - -Use a GitLab project template to get started. As the name suggests, these projects provide a -bare-bones application built on some well-known frameworks. - -1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select - **New project**. - -1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or - NodeJS Express project. For this guide, use the Ruby on Rails template. - -  - -1. Give your project a name. In this example, it's named `ecs-demo`. Make it public so that you can - take advantage of the features available in the - [GitLab Ultimate plan](https://about.gitlab.com/pricing/). - -1. Select **Create project**. - -Now that you created a demo project, you must containerize the application and push it to the -container registry. - -### Push a containerized application image to GitLab Container Registry - -[ECS](https://aws.amazon.com/ecs/) is a container orchestration service, meaning that you must -provide a containerized application image during the infrastructure build. To do so, you can use -GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) -and [Container Registry](../../../user/packages/container_registry/index.md). - -1. Go to **ecs-demo** project on GitLab. -1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` - creation form. -1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines - [a pipeline for continuous deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs). - - ```yaml - include: - - template: AWS/Deploy-ECS.gitlab-ci.yml - ``` - -1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` - job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md). - -  - -1. Visit **Packages & Registries > Container Registry**. Make sure the application image has been - pushed. - -  - -Now you have a containerized application image that can be pulled from AWS. Next, you define the -spec of how this application image is used in AWS. - -Note that the `production_ecs` job fails because ECS Cluster is not connected yet. You'll fix this -later. - -### Create an ECS task definition - -[ECS Task definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definitions.html) -is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). - -1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). -1. Select **Create new Task Definition**. - -  - -1. Choose **EC2** as the launch type. Select **Next Step**. -1. Set `ecs_demo` to **Task Definition Name**. -1. Set `512` to **Task Size > Task memory** and **Task CPU**. -1. Select **Container Definitions > Add container**. This opens a container registration form. -1. Set `web` to **Container name**. -1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. - Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). - -  - -1. Add a port mapping. Set `80` to **Host Port** and `5000` to **Container port**. - -  - -1. Select **Create**. - -Now you have the initial task definition. Next, you create an actual infrastructure to run the -application image. - -### Create an ECS cluster - -An [ECS cluster](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/clusters.html) -is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). -It's also associated with EC2 or Fargate as the computation resource. - -1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). -1. Select **Create Cluster**. -1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. -1. Set `ecs-demo` to **Cluster Name**. -1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) - in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. -1. Set all available subnets of the VPC to **Subnets**. -1. Select **Create**. -1. Make sure that the ECS cluster has been successfully created. - -  - -Now you can register an ECS service to the ECS cluster in the next step. - -Note the following: - -- Optionally, you can set a SSH key pair in the creation form. This allows you to SSH to the EC2 - instance for debugging. -- If you don't choose an existing VPC, it creates a new VPC by default. This could cause an error if - it reaches the maximum allowed number of internet gateways on your account. -- The cluster requires an EC2 instance, meaning it costs you [according to the instance-type](https://aws.amazon.com/ec2/pricing/on-demand/). - -### Create an ECS Service - -[ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) -is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). - -1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) -1. Select **Deploy**. This opens a service creation form. -1. Select `EC2` in **Launch Type**. -1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). -1. Set `ecs_demo` to **Service name**. -1. Set `1` to **Desired tasks**. - -  - -1. Select **Deploy**. -1. Make sure that the created service is active. - -  - -Note that AWS's console UI changes from time to time. If you can't find a relevant component in the -instructions, select the closest one. - -### View the demo application - -Now, the demo application is accessible from the internet. - -1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) -1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). -1. Select the ID of the EC2 instance. This brings you to the instance detail page. -1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application - running. - -  - -In this guide, HTTPS/SSL is **NOT** configured. You can access to the application through HTTP only -(for example, `http://<ec2-ipv4-address>`). - -## Setup Continuous Deployment from GitLab - -Now that you have an application running on ECS, you can set up continuous deployment from GitLab. - -### Create a new IAM user as a deployer - -For GitLab to access the ECS cluster, service, and task definition that you created above, You must -create a deployer user on AWS: - -1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). -1. Select **Add user**. -1. Set `ecs_demo` to **User name**. -1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. -1. Select `Attach existing policies directly` in **Set permissions**. -1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**. - -  - -1. Select **Create user**. -1. Take note of the **Access key ID** and **Secret access key** of the created user. - -NOTE: -Do not share the secret access key in a public place. You must save it in a secure place. - -### Setup credentials in GitLab to let pipeline jobs access to ECS - -You can register the access information in [GitLab Environment Variables](../../variables/index.md#custom-cicd-variables). -These variables are injected into the pipeline jobs and can access the ECS API. - -1. Go to **ecs-demo** project on GitLab. -1. Go to **Settings > CI/CD > Variables**. -1. Select **Add Variable** and set the following key-value pairs. - - |Key|Value|Note| - |---|---|---| - |`AWS_ACCESS_KEY_ID`|`<Access key ID of the deployer>`| For authenticating `aws` CLI. | - |`AWS_SECRET_ACCESS_KEY`|`<Secret access key of the deployer>`| For authenticating `aws` CLI. | - |`AWS_DEFAULT_REGION`|`us-east-2`| For authenticating `aws` CLI. | - |`CI_AWS_ECS_CLUSTER`|`ecs-demo`| The ECS cluster is accessed by `production_ecs` job. | - |`CI_AWS_ECS_SERVICE`|`ecs_demo`| The ECS service of the cluster is updated by `production_ecs` job. | - |`CI_AWS_ECS_TASK_DEFINITION`|`ecs_demo`| The ECS task definition is updated by `production_ecs` job. | - -### Make a change to the demo application - -Change a file in the project and see if it's reflected in the demo application on ECS: - -1. Go to **ecs-demo** project on GitLab. -1. Open the file at **app > views > welcome > `index.html.erb`**. -1. Select **Edit**. -1. Change the text to `You're on ECS!`. -1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. -1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see - this: - -  - -Congratulations! You successfully set up continuous deployment to ECS. - -## Further reading - -- If you're interested in more of the continuous deployments to clouds, see [cloud deployments](../index.md). -- If you want to quickly set up DevSecOps in your project, see [Auto DevOps](../../../topics/autodevops/index.md). -- If you want to quickly set up the production-grade environment, see [the 5 Minute Production App](https://gitlab.com/gitlab-org/5-minute-production-app/deploy-template/-/blob/master/README.md). +<!-- This redirect file can be deleted after <2022-09-18>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
\ No newline at end of file diff --git a/doc/ci/cloud_deployment/index.md b/doc/ci/cloud_deployment/index.md index c89ce2675e4..c5be2328264 100644 --- a/doc/ci/cloud_deployment/index.md +++ b/doc/ci/cloud_deployment/index.md @@ -5,165 +5,101 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Cloud deployment **(FREE)** +# Deploy to AWS from GitLab CI/CD **(FREE)** -Interacting with a major cloud provider may have become a much needed task that's -part of your delivery process. With GitLab you can -[deploy your application anywhere](https://about.gitlab.com/stages-devops-lifecycle/deploy-targets/). +GitLab provides Docker images with the libraries and tools you need to deploy +to AWS. You can reference these images in your CI/CD pipeline. -For some specific deployment targets, GitLab makes this process less painful by providing Docker -images with the needed libraries and tools pre-installed. By referencing them in your -CI/CD pipeline, you can interact with your chosen cloud provider more easily. +If you're using GitLab.com and deploying to the [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (ECS), +read about [deploying to ECS](ecs/quick_start_guide.md). -## AWS +## Authenticate GitLab with AWS -GitLab provides Docker images that you can use to [run AWS commands from GitLab CI/CD](#run-aws-commands-from-gitlab-cicd), and a template to make -it easier to [deploy to AWS](#deploy-your-application-to-the-aws-elastic-container-service-ecs). +To use GitLab CI/CD to connect to AWS, you must authenticate. +After you set up authentication, you can configure CI/CD to deploy. -### Quick start +1. Sign on to your AWS account. +1. Create [an IAM user](https://console.aws.amazon.com/iam/home#/home). +1. Select your user to access its details. Go to **Security credentials > Create a new access key**. +1. Note the **Access key ID** and **Secret access key**. +1. In your GitLab project, go to **Settings > CI/CD**. Set the following + [CI/CD variables](../variables/index.md): -If you're using GitLab.com, see the [quick start guide](ecs/quick_start_guide.md) -for setting up Continuous Deployment to [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS). + | Environment variable name | Value | + |:-------------------------------|:------------------------| + | `AWS_ACCESS_KEY_ID` | Your Access key ID. | + | `AWS_SECRET_ACCESS_KEY` | Your secret access key. | + | `AWS_DEFAULT_REGION` | Your region code. You might want to confirm that the AWS service you intend to use is [available in the chosen region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). | -### Run AWS commands from GitLab CI/CD +1. Variables are [protected by default](../variables/index.md#protected-cicd-variables). + To use GitLab CI/CD with branches or tags that are not protected, + clear the **Protect variable** checkbox. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31167) in GitLab 12.6. +## Use an image to run AWS commands -The GitLab AWS Docker image provides the [AWS Command Line Interface](https://aws.amazon.com/cli/), -which enables you to run `aws` commands. As part of your deployment strategy, you can run `aws` commands directly from -`.gitlab-ci.yml` by specifying the [GitLab AWS Docker image](https://gitlab.com/gitlab-org/cloud-deploy). +If an image contains the [AWS Command Line Interface](https://aws.amazon.com/cli/), +you can reference the image in your project's `.gitlab-ci.yml` file. Then you can run +`aws` commands in your CI/CD jobs. -Some credentials are required to be able to run `aws` commands: +For example: -1. Sign up for [an AWS account](https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-set-up.html) if you don't have one yet. -1. Log in onto the console and create [a new IAM user](https://console.aws.amazon.com/iam/home#/home). -1. Select your newly created user to access its details. Navigate to **Security credentials > Create a new access key**. - - NOTE: - A new **Access key ID** and **Secret access key** are generated. Please take a note of them right away. - -1. In your GitLab project, go to **Settings > CI/CD**. Set the following as - [CI/CD variables](../variables/index.md) - (see table below): - - - Access key ID. - - Secret access key. - - Region code. You can check the [list of AWS regional endpoints](https://docs.aws.amazon.com/general/latest/gr/rande.html#regional-endpoints). - You might want to check if the AWS service you intend to use is - [available in the chosen region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). - - | Environment variable name | Value | - |:-------------------------------|:-----------------------| - | `AWS_ACCESS_KEY_ID` | Your Access key ID | - | `AWS_SECRET_ACCESS_KEY` | Your Secret access key | - | `AWS_DEFAULT_REGION` | Your region code | - - NOTE: - When you create a variable it's set to be [protected by default](../variables/index.md#protected-cicd-variables). If you want to use the `aws` commands on branches or tags that are not protected, make sure to uncheck the **Protect variable** checkbox. - -1. You can now use `aws` commands in the `.gitlab-ci.yml` file of this project: - - ```yaml - deploy: - stage: deploy - image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest # see the note below - script: - - aws s3 ... - - aws create-deployment ... - ``` +```yaml +deploy: + stage: deploy + image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest + script: + - aws s3 ... + - aws create-deployment ... +``` - NOTE: - The image used in the example above - (`registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest`) is hosted on the [GitLab - Container Registry](../../user/packages/container_registry/index.md) and is - ready to use. Alternatively, replace the image with one hosted on AWS ECR. +GitLab provides a Docker image that includes the AWS CLI: -### Use an AWS Elastic Container Registry (ECR) image in your CI/CD +- Images are hosted in the GitLab Container Registry. The latest image is + `registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest`. +- [Images are stored in a GitLab repository](https://gitlab.com/gitlab-org/cloud-deploy/-/tree/master/aws). -Instead of referencing an image hosted on the GitLab Registry, you can -reference an image hosted on any third-party registry, such as the -[Amazon Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/). +Alternately, you can use an [Amazon Elastic Container Registry (ECR)](https://aws.amazon.com/ecr/) image. +[Learn how to push an image to your ECR repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html). -To do so, [push your image into your ECR -repository](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html). -Then reference it in your `.gitlab-ci.yml` file and replace the `image` -path to point to your ECR image. +You can also use an image from any third-party registry. -### Deploy your application to the AWS Elastic Container Service (ECS) +## Deploy your application to ECS > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207962) in GitLab 12.9. > - The `Deploy-ECS.gitlab-ci.yml` template was [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/220821) to `AWS/Deploy-ECS.gitlab-ci.yml` in GitLab 13.2. -GitLab provides a series of [CI templates that you can include in your project](../yaml/index.md#include). -To automate deployments of your application to your [Amazon Elastic Container Service](https://aws.amazon.com/ecs/) (AWS ECS) -cluster, you can `include` the `AWS/Deploy-ECS.gitlab-ci.yml` template in your `.gitlab-ci.yml` file. +You can automate deployments of your application to your [Amazon ECS](https://aws.amazon.com/ecs/) +cluster. -GitLab also provides [Docker images](https://gitlab.com/gitlab-org/cloud-deploy/-/tree/master/aws) that can be used in your `.gitlab-ci.yml` file to simplify working with AWS: +Prerequisites: -- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest` to use AWS CLI commands. -- Use `registry.gitlab.com/gitlab-org/cloud-deploy/aws-ecs:latest` to deploy your application to AWS ECS. +- [Authenticate AWS with GitLab](#authenticate-gitlab-with-aws). +- Create a cluster on Amazon ECS. +- Create related components, like an ECS service, a database on Amazon RDS, and so on. +- Create an ECS task definition, where the value for the `containerDefinitions[].name` attribute is + the same as the `Container name` defined in your targeted ECS service. The task definition can be: + - An existing task definition in ECS. + - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/222618), + a JSON file in your GitLab project. Use the + [template in the AWS documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html#task-definition-template) + and save the file in your project. For example `<project-root>/ci/aws/task-definition.json`. -Before getting started with this process, you need a cluster on AWS ECS, as well as related -components, like an ECS service, ECS task definition, a database on Amazon RDS, and so on. -[Read more about AWS ECS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html). - -The ECS task definition can be: - -- An existing task definition in AWS ECS -- A JSON file containing a task definition. Create the JSON file by using the template provided in - the [AWS documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/create-task-definition.html#task-definition-template). - Copy the task definition into a new file in your project, for example `<project-root>/ci/aws/task-definition.json`. - [Available](https://gitlab.com/gitlab-org/gitlab/-/issues/222618) in GitLab 13.3 and later. - -After you have these prerequisites ready, follow these steps: - -1. Make sure your AWS credentials are set up as CI/CD variables for your - project. You can follow [the steps above](#run-aws-commands-from-gitlab-cicd) to complete this setup. -1. Add these variables to your project's `.gitlab-ci.yml` file, or in the project's - [CI/CD settings](../variables/index.md#custom-cicd-variables): - - - `CI_AWS_ECS_CLUSTER`: The name of the AWS ECS cluster that you're targeting for your deployments. - - `CI_AWS_ECS_SERVICE`: The name of the targeted service tied to your AWS ECS cluster. - - `CI_AWS_ECS_TASK_DEFINITION`: The name of an existing task definition in ECS tied - to the service mentioned above. - - ```yaml - variables: - CI_AWS_ECS_CLUSTER: my-cluster - CI_AWS_ECS_SERVICE: my-service - CI_AWS_ECS_TASK_DEFINITION: my-task-definition - ``` - - You can find these names after selecting the targeted cluster on your [AWS ECS dashboard](https://console.aws.amazon.com/ecs/home): - -  - - Alternatively, if you want to use a task definition defined in a JSON file, use - `CI_AWS_ECS_TASK_DEFINITION_FILE` instead: - - ```yaml - variables: - CI_AWS_ECS_CLUSTER: my-cluster - CI_AWS_ECS_SERVICE: my-service - CI_AWS_ECS_TASK_DEFINITION_FILE: ci/aws/my_task_definition.json - ``` +To deploy to your ECS cluster: - You can create your `CI_AWS_ECS_TASK_DEFINITION_FILE` variable as a - [file-typed CI/CD variable](../variables/index.md#cicd-variable-types) instead of a - regular CI/CD variable. If you choose to do so, set the variable value to be the full contents of - the JSON task definition. You can then remove the JSON file from your project. +1. In your GitLab project, go to **Settings > CI/CD**. Set the following + [CI/CD variables](../variables/index.md). You can find these names by + selecting the targeted cluster on your [Amazon ECS dashboard](https://console.aws.amazon.com/ecs/home). - In both cases, make sure that the value for the `containerDefinitions[].name` attribute is - the same as the `Container name` defined in your targeted ECS service. + | Environment variable name | Value | + |:-------------------------------|:------------------------| + | `CI_AWS_ECS_CLUSTER` | The name of the AWS ECS cluster that you're targeting for your deployments. | + | `CI_AWS_ECS_SERVICE` | The name of the targeted service tied to your AWS ECS cluster. | + | `CI_AWS_ECS_TASK_DEFINITION` | If the task definition is in ECS, the name of the task definition tied to the service. | + | `CI_AWS_ECS_TASK_DEFINITION_FILE` | If the task definition is a JSON file in GitLab, the filename, including the path. For example, `ci/aws/my_task_definition.json`. If the name of the task definition in your JSON file is the same name as an existing task definition in ECS, then a new revision is created when CI/CD runs. Otherwise, a brand new task definition is created, starting at revision 1. | WARNING: - `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence over `CI_AWS_ECS_TASK_DEFINITION` if both these - variables are defined within your project. - - NOTE: - If the name of the task definition you wrote in your JSON file is the same name - as an existing task definition on AWS, then a new revision is created for it. - Otherwise, a brand new task definition is created, starting at revision 1. + If you define both `CI_AWS_ECS_TASK_DEFINITION_FILE` and `CI_AWS_ECS_TASK_DEFINITION`, + `CI_AWS_ECS_TASK_DEFINITION_FILE` takes precedence. 1. Include this template in `.gitlab-ci.yml`: @@ -172,86 +108,78 @@ After you have these prerequisites ready, follow these steps: - template: AWS/Deploy-ECS.gitlab-ci.yml ``` - The `AWS/Deploy-ECS` template ships with GitLab and is available [on - GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml). + The `AWS/Deploy-ECS` template ships with GitLab and is available + [on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml). -1. Commit and push your updated `.gitlab-ci.yml` to your project's repository, and you're done! +1. Commit and push your updated `.gitlab-ci.yml` to your project's repository. - Your application Docker image is rebuilt and pushed to the GitLab registry. - If your image is located in a private registry, make sure your task definition is - [configured with a `repositoryCredentials` attribute](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html). +Your application Docker image is rebuilt and pushed to the GitLab Container Registry. +If your image is located in a private registry, make sure your task definition is +[configured with a `repositoryCredentials` attribute](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html). - Then the targeted task definition is updated with the location of the new - Docker image, and a new revision is created in ECS as result. +The targeted task definition is updated with the location of the new +Docker image, and a new revision is created in ECS as result. - Finally, your AWS ECS service is updated with the new revision of the - task definition, making the cluster pull the newest version of your - application. +Finally, your AWS ECS service is updated with the new revision of the +task definition, making the cluster pull the newest version of your +application. + +NOTE: +ECS deploy jobs wait for the rollout to complete before exiting. To disable this behavior, +set `CI_AWS_ECS_WAIT_FOR_ROLLOUT_COMPLETE_DISABLED` to a non-empty value. WARNING: The [`AWS/Deploy-ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/AWS/Deploy-ECS.gitlab-ci.yml) -template includes both the [`Jobs/Build.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) -and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml) -"sub-templates". Do not include these "sub-templates" on their own, and only include the main -`AWS/Deploy-ECS.gitlab-ci.yml` template. The "sub-templates" are designed to only be -used along with the main template. They may move or change unexpectedly causing your -pipeline to fail if you didn't include the main template. Also, the job names within -these templates may change. Do not override these jobs names in your own pipeline, -as the override stops working when the name changes. - -Alternatively, if you don't wish to use the `AWS/Deploy-ECS.gitlab-ci.yml` template -to deploy to AWS ECS, you can always use our -`aws-base` Docker image to run your own [AWS CLI commands for ECS](https://docs.aws.amazon.com/cli/latest/reference/ecs/index.html#cli-aws-ecs). +template includes two templates: [`Jobs/Build.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Build.gitlab-ci.yml) +and [`Jobs/Deploy/ECS.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml). Do not include these templates on their own. Only include the +`AWS/Deploy-ECS.gitlab-ci.yml` template. These other templates are designed to be +used only with the main template. They may move or change unexpectedly. Also, the job names within +these templates may change. Do not override these job names in your own pipeline, +because the override stops working when the name changes. -```yaml -deploy: - stage: deploy - image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest - script: - - aws ecs register-task-definition ... -``` - -### Provision and deploy to your AWS Elastic Compute Cloud (EC2) +## Deploy your application to EC2 > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/201742) in GitLab 13.5. -You can use the `AWS/CF-Provision-and-Deploy-EC2` CI template to perform the -following actions within the same pipeline: +GitLab provides a template, called `AWS/CF-Provision-and-Deploy-EC2`, +to assist you in deploying to Amazon EC2. -1. **Create stack**: Provision your own infrastructure by leveraging the [AWS CloudFormation](https://aws.amazon.com/cloudformation/) API. -1. **Push to S3**: Push your previously-built artifact to an [AWS S3](https://aws.amazon.com/s3/) bucket. -1. **Deploy to EC2**: Deploy this pushed content onto an [AWS EC2](https://aws.amazon.com/ec2/) instance. +When you configure related JSON objects and use the template, the pipeline: - +1. **Creates the stack**: Your infrastructure is provisioned by using + the [AWS CloudFormation](https://aws.amazon.com/cloudformation/) API. +1. **Pushes to an S3 bucket**: When your build runs, it creates an artifact. + The artifact is pushed to an [AWS S3](https://aws.amazon.com/s3/) bucket. +1. **Deploys to EC2**: The content is deployed on an [AWS EC2](https://aws.amazon.com/ec2/) instance. -#### Run the `AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml` template + -To run the `AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml` template, you must -pass three JSON input objects, based on existing templates: +### Configure the template and JSON -1. The AWS documentation provides templates for the _Create stack_ and _Deploy to EC2_ steps (links - below). We provide the template for the remaining step, _Push to S3_: +To deploy to EC2, complete the following steps. - - [Template for the _Create stack_ step on AWS](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html). - - Template for the _Push to S3_ step. Note that `source` is where a preceding `build` job built - your application, exporting the build through [`artifacts:paths`](../yaml/index.md#artifactspaths): +1. Create JSON for your stack. Use the [AWS template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html). +1. Create JSON to push to S3. Include the following details. - ```json - { - "applicationName": "string", - "source": "string", - "s3Location": "s3://your/bucket/project_built_file...]" - } - ``` + ```json + { + "applicationName": "string", + "source": "string", + "s3Location": "s3://your/bucket/project_built_file...]" + } + ``` - - [Template for the _Deploy to EC2_ step on AWS](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html). + The `source` is the location where a `build` job built your application. + The build is saved to [`artifacts:paths`](../yaml/index.md#artifactspaths). -1. After you have completed these three templates based on your requirements, you - have two ways to pass in these JSON objects: +1. Create JSON to deploy to EC2. Use the [AWS template](https://docs.aws.amazon.com/codedeploy/latest/APIReference/API_CreateDeployment.html). +1. Make the JSON objects accessible to your pipeline: + - If you want these JSON objects saved in your repository, save the objects as three + separate files. - - They can be three actual files located in your project. You must specify their path relative to - your project root in your `.gitlab-ci.yml` file, using the following CI/CD variables. For example, if - your files are in a `<project_root>/aws` folder: + In your `.gitlab-ci.yml` file, add [CI/CD variables](../variables/index.md) + that point to the file paths relative to the project root. For example, + if your JSON files are in a `<project_root>/aws` folder: ```yaml variables: @@ -260,71 +188,30 @@ pass three JSON input objects, based on existing templates: CI_AWS_EC2_DEPLOYMENT_FILE: 'aws/create_deployment.json' ``` - - Alternatively, you can provide these JSON objects as [file-typed CI/CD variables](../variables/index.md#cicd-variable-types). - In your project, go to **Settings > CI/CD > Variables** and add - the three variables listed above as file-typed CI/CD variables. - For each variable, set the value to its corresponding JSON object. + - If you do not want these JSON objects saved in your repository, add each object + as a separate [file type CI/CD variable](../variables/index.md#cicd-variable-types) + in the project settings. Use the same variable names as above. -1. Provide the name of the stack you're creating and/or targeting, as a CI/CD variable: +1. In your `.gitlab-ci.yml` file, create a CI/CD variable for the name of the stack. For example: ```yaml variables: CI_AWS_CF_STACK_NAME: 'YourStackName' ``` -1. Add this CI template to your `.gitlab-ci.yml`: +1. In your `.gitlab-ci.yml` file, add the CI template: ```yaml include: - template: AWS/CF-Provision-and-Deploy-EC2.gitlab-ci.yml ``` -When running your project pipeline at this point: - -- Your AWS CloudFormation stack is created based on the content of your - `CI_AWS_CF_CREATE_STACK_FILE` file/variable. - If your stack already exists, this step is skipped, but the `provision` job it belongs to still - runs. -- Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based - on the related JSON object's content. The deployment job finishes whenever the deployment to EC2 - is done or has failed. - -#### Custom build job for Auto DevOps - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216008) in GitLab 13.6. - -To leverage [Auto DevOps](../../topics/autodevops/index.md) for your project when deploying to -AWS EC2, first you must define [your AWS credentials as CI/CD variables](#run-aws-commands-from-gitlab-cicd). - -Next, define a job for the `build` stage. To do so, you must reference the -`Auto-DevOps.gitlab-ci.yml` template and include a job named `build_artifact` in your -`.gitlab-ci.yml` file. For example: - -```yaml -# .gitlab-ci.yml - -include: - - template: Auto-DevOps.gitlab-ci.yml - -variables: - AUTO_DEVOPS_PLATFORM_TARGET: EC2 - -build_artifact: - stage: build - script: - - <your build script goes here> - artifacts: - paths: - - <built artifact> -``` - -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> -For a video walkthrough of this configuration process, see [Auto Deploy to EC2](https://www.youtube.com/watch?v=4B-qSwKnacA). - -### Deploy to Amazon EKS - -- [How to deploy your application to a GitLab-managed Amazon EKS cluster with Auto DevOps](https://about.gitlab.com/blog/2020/05/05/deploying-application-eks/) - -## Deploy to Google Cloud +1. Run the pipeline. -- [Deploying with GitLab on Google Cloud](https://about.gitlab.com/partners/technology-partners/google-cloud-platform/) + - Your AWS CloudFormation stack is created based on the content of your + `CI_AWS_CF_CREATE_STACK_FILE` variable. + If your stack already exists, this step is skipped, but the `provision` + job it belongs to still runs. + - Your built application is pushed to your S3 bucket then and deployed to your EC2 instance, based + on the related JSON object's content. The deployment job finishes when the deployment to EC2 + is done or has failed. diff --git a/doc/ci/cloud_services/google_cloud/index.md b/doc/ci/cloud_services/google_cloud/index.md index 14928f91816..f4e4a2046ba 100644 --- a/doc/ci/cloud_services/google_cloud/index.md +++ b/doc/ci/cloud_services/google_cloud/index.md @@ -50,8 +50,8 @@ inside the Workload Identity Pool created in the previous step, using the follow such as `gitlab/gitlab`. - **Provider ID**: Unique ID in the pool for the Workload Identity Provider, such as `gitlab-gitlab`. This value is used to refer to the provider, and appears in URLs. -- **Issuer (URL)**: The address of your GitLab instance, such as `https://gitlab.com` or - `https://gitlab.example.com`. +- **Issuer (URL)**: The address of your GitLab instance, such as `https://gitlab.com/` or + `https://gitlab.example.com/`. - The address must use the `https://` protocol. - The address must end in a trailing slash. - **Audiences**: Manually set the allowed audiences list to the address of your diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index 5d0aaf7cc45..f63ecc57028 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -87,7 +87,7 @@ are certain use cases that you may need to work around. For more information, ch The needs visualization makes it easier to visualize the relationships between dependent jobs in a DAG. This graph displays all the jobs in a pipeline that need or are needed by other jobs. Jobs with no relationships are not displayed in this view. -To see the needs visualization, click on the **Needs** tab when viewing a pipeline that uses the `needs` keyword. +To see the needs visualization, select **Needs** when viewing a pipeline that uses the `needs` keyword.  diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 9b91cd40338..df0c7b69d46 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -92,7 +92,7 @@ the job script in context of the image in privileged mode. We recommend you use Docker-in-Docker with TLS enabled, which is supported by [GitLab.com shared runners](../runners/index.md). -You should always specify a specific version of the image, like `docker:19.03.12`. +You should always specify a specific version of the image, like `docker:20.10.16`. If you use a tag like `docker:stable`, you have no control over which version is used. Unpredictable behavior can result, especially when new versions are released. @@ -126,12 +126,12 @@ To use Docker-in-Docker with TLS enabled: --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ + --docker-image "docker:20.10.16" \ --docker-privileged \ --docker-volumes "/certs/client" ``` - - This command registers a new runner to use the `docker:19.03.12` image. + - This command registers a new runner to use the `docker:20.10.16` image. To start the build and service containers, it uses the `privileged` mode. If you want to use [Docker-in-Docker](https://www.docker.com/blog/docker-can-now-run-within-docker/), you must always use `privileged = true` in your Docker containers. @@ -149,7 +149,7 @@ To use Docker-in-Docker with TLS enabled: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.12" + image = "docker:20.10.16" privileged = true disable_cache = false volumes = ["/certs/client", "/cache"] @@ -159,10 +159,10 @@ To use Docker-in-Docker with TLS enabled: ``` 1. You can now use `docker` in the job script. Note the inclusion of the - `docker:19.03.12-dind` service: + `docker:20.10.16-dind` service: ```yaml - image: docker:19.03.12 + image: docker:20.10.16 variables: # When you use the dind service, you must instruct Docker to talk with @@ -182,7 +182,7 @@ To use Docker-in-Docker with TLS enabled: DOCKER_TLS_CERTDIR: "/certs" services: - - docker:19.03.12-dind + - docker:20.10.16-dind before_script: - docker info @@ -209,7 +209,7 @@ Assuming that the runner's `config.toml` is similar to: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.12" + image = "docker:20.10.16" privileged = true disable_cache = false volumes = ["/cache"] @@ -219,10 +219,10 @@ Assuming that the runner's `config.toml` is similar to: ``` You can now use `docker` in the job script. Note the inclusion of the -`docker:19.03.12-dind` service: +`docker:20.10.16-dind` service: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 variables: # When using dind service, you must instruct docker to talk with the @@ -243,7 +243,7 @@ variables: DOCKER_TLS_CERTDIR: "" services: - - docker:19.03.12-dind + - docker:20.10.16-dind before_script: - docker info @@ -284,10 +284,10 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: ``` 1. You can now use `docker` in the job script. Note the inclusion of the - `docker:19.03.13-dind` service: + `docker:20.10.16-dind` service: ```yaml - image: docker:19.03.13 + image: docker:20.10.16 variables: # When using dind service, you must instruct Docker to talk with @@ -315,7 +315,7 @@ To use Docker-in-Docker with TLS enabled in Kubernetes: DOCKER_CERT_PATH: "$DOCKER_TLS_CERTDIR/client" services: - - docker:19.03.13-dind + - docker:20.10.16-dind before_script: - docker info @@ -341,7 +341,7 @@ not without its own challenges: - **Storage drivers**: By default, earlier versions of Docker use the `vfs` storage driver, which copies the file system for each job. Docker 17.09 and later use `--storage-driver overlay2`, which is the recommended storage driver. See [Using the OverlayFS driver](#use-the-overlayfs-driver) for details. -- **Root file system**: Because the `docker:19.03.12-dind` container and the runner container don't share their +- **Root file system**: Because the `docker:20.10.16-dind` container and the runner container don't share their root file system, you can use the job's working directory as a mount point for child containers. For example, if you have files you want to share with a child container, you might create a subdirectory under `/builds/$CI_PROJECT_PATH` @@ -364,7 +364,7 @@ container. Docker is then available in the context of the image. NOTE: If you bind the Docker socket and you are [using GitLab Runner 11.11 or later](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/1261), -you can no longer use `docker:19.03.12-dind` as a service. Volume bindings +you can no longer use `docker:20.10.16-dind` as a service. Volume bindings are done to the services as well, making these incompatible. #### Use the Docker executor with Docker socket binding @@ -383,7 +383,7 @@ Your configuration should look something like this: executor = "docker" [runners.docker] tls_verify = false - image = "docker:19.03.12" + image = "docker:20.10.16" privileged = false disable_cache = false volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"] @@ -399,7 +399,7 @@ sudo gitlab-runner register -n \ --registration-token REGISTRATION_TOKEN \ --executor docker \ --description "My Docker Runner" \ - --docker-image "docker:19.03.12" \ + --docker-image "docker:20.10.16" \ --docker-volumes /var/run/docker.sock:/var/run/docker.sock ``` @@ -417,7 +417,7 @@ mirror: ```yaml services: - - name: docker:19.03.13-dind + - name: docker:20.10.16-dind command: ["--registry-mirror", "https://registry-mirror.example.com"] # Specify the registry mirror to use ``` @@ -440,7 +440,7 @@ Docker: ... privileged = true [[runners.docker.services]] - name = "docker:19.03.13-dind" + name = "docker:20.10.16-dind" command = ["--registry-mirror", "https://registry-mirror.example.com"] ``` @@ -454,7 +454,7 @@ Kubernetes: ... privileged = true [[runners.kubernetes.services]] - name = "docker:19.03.13-dind" + name = "docker:20.10.16-dind" command = ["--registry-mirror", "https://registry-mirror.example.com"] ``` @@ -563,11 +563,11 @@ the implications of this method are: docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` -You don't need to include the `docker:19.03.12-dind` service, like you do when +You don't need to include the `docker:20.10.16-dind` service, like you do when you're using the Docker-in-Docker executor: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 before_script: - docker info @@ -591,13 +591,13 @@ In [`before_script`](../yaml/index.md#before_script), run `docker login`: ```yaml -image: docker:19.03.13 +image: docker:20.10.16 variables: DOCKER_TLS_CERTDIR: "/certs" services: - - docker:19.03.13-dind + - docker:20.10.16-dind build: stage: build @@ -616,7 +616,7 @@ empty or remove it. If you are an administrator for GitLab Runner, you can mount a file with the authentication configuration to `~/.docker/config.json`. Then every job that the runner picks up is authenticated already. If you -are using the official `docker:19.03.13` image, the home directory is +are using the official `docker:20.10.16` image, the home directory is under `/root`. If you mount the configuration file, any `docker` command @@ -699,13 +699,13 @@ The following example shows [`before_script`](../yaml/index.md#before_script). The same commands apply for any solution you implement. ```yaml -image: docker:19.03.13 +image: docker:20.10.16 variables: DOCKER_TLS_CERTDIR: "/certs" services: - - docker:19.03.13-dind + - docker:20.10.16-dind build: stage: build @@ -741,10 +741,10 @@ with the `--cache-from` argument must first be pulled Here's a `.gitlab-ci.yml` file that shows how to use Docker caching: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 services: - - docker:19.03.12-dind + - docker:20.10.16-dind variables: # Use TLS https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#tls-enabled @@ -862,10 +862,10 @@ This issue can occur when the service's image name [includes a registry hostname](../../ci/services/index.md#available-settings-for-services). For example: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 services: - - registry.hub.docker.com/library/docker:19.03.12-dind + - registry.hub.docker.com/library/docker:20.10.16-dind ``` A service's hostname is [derived from the full image name](../../ci/services/index.md#accessing-the-services). @@ -873,9 +873,9 @@ However, the shorter service hostname `docker` is expected. To allow service resolution and access, add an explicit alias for the service name `docker`: ```yaml -image: docker:19.03.12 +image: docker:20.10.16 services: - - name: registry.hub.docker.com/library/docker:19.03.12-dind + - name: registry.hub.docker.com/library/docker:20.10.16-dind alias: docker ``` diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 7edff334134..fdd8b6d38b8 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -112,7 +112,7 @@ For example, the following two definitions are equal: image: "registry.example.com/my/image:latest" services: - - postgresql:9.4 + - postgresql:14.3 - redis:latest ``` @@ -124,7 +124,7 @@ For example, the following two definitions are equal: name: "registry.example.com/my/image:latest" services: - - name: postgresql:9.4 + - name: postgresql:14.3 - name: redis:latest ``` diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 2e514fd0f0a..dac52a4540e 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -5,50 +5,46 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# How to enable or disable GitLab CI/CD **(FREE)** - -To use GitLab CI/CD, you need: - -- A valid [`.gitlab-ci.yml`](yaml/index.md) file present at the root directory - of your project. -- A [runner](runners/index.md) ready to run jobs. - -You can read our [quick start guide](quick_start/index.md) to get you started. +# Disabling GitLab CI/CD **(FREE)** +GitLab CI/CD is enabled by default on all new projects. If you use an external CI/CD server like Jenkins or Drone CI, you can disable GitLab CI/CD to avoid conflicts with the commits status API. -GitLab CI/CD is enabled by default on all new projects. You can: +You can disable GitLab CI/CD: -- Disable GitLab CI/CD [under each project's settings](#enable-cicd-in-a-project). -- Set GitLab CI/CD to be [disabled in all new projects on an instance](../administration/cicd.md). +- [For each project](#disable-cicd-in-a-project). +- [For all new projects on an instance](../administration/cicd.md). -If you disable GitLab CI/CD in a project: +These changes do not apply to projects in an +[external integration](../user/project/integrations/index.md#available-integrations). -- The **CI/CD** item in the left sidebar is removed. -- The `/pipelines` and `/jobs` pages are no longer available. -- Existing jobs and pipelines are not deleted. Re-enable CI/CD to access them again. +## Disable CI/CD in a project -The project or instance settings do not enable or disable pipelines run in an -[external integration](../user/project/integrations/index.md#available-integrations). +When you disable GitLab CI/CD: -## Enable CI/CD in a project +- The **CI/CD** item in the left sidebar is removed. +- The `/pipelines` and `/jobs` pages are no longer available. +- Existing jobs and pipelines are hidden, not removed. -To enable or disable GitLab CI/CD pipelines in your project: +To disable GitLab CI/CD in your project: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility, project features, permissions**. -1. In the **Repository** section, turn on or off **CI/CD** as required. +1. In the **Repository** section, turn off **CI/CD**. +1. Select **Save changes**. -**Project visibility** also affects pipeline visibility. If set to: +## Enable CI/CD in a project -- **Private**: Only project members can access pipelines. -- **Internal** or **Public**: Pipelines can be set to either **Only Project Members** - or **Everyone With Access** by using the dropdown box. +To enable GitLab CI/CD in your project: -Press **Save changes** for the settings to take effect. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility, project features, permissions**. +1. In the **Repository** section, turn on **CI/CD**. +1. Select **Save changes**. <!-- ## Troubleshooting diff --git a/doc/ci/environments/environments_dashboard.md b/doc/ci/environments/environments_dashboard.md index 8ff31827ae7..c3c1e7868fd 100644 --- a/doc/ci/environments/environments_dashboard.md +++ b/doc/ci/environments/environments_dashboard.md @@ -36,9 +36,9 @@ environments are not displayed. To add a project to the dashboard: -1. Click the **Add projects** button in the home screen of the dashboard. +1. Select **Add projects** in the home screen of the dashboard. 1. Search and add one or more projects using the **Search your projects** field. -1. Click the **Add projects** button. +1. Select **Add projects**. Once added, you can see a summary of each project's environment operational health, including the latest commit, pipeline status, and deployment time. diff --git a/doc/ci/environments/incremental_rollouts.md b/doc/ci/environments/incremental_rollouts.md index d359ed078ff..ee2e708f822 100644 --- a/doc/ci/environments/incremental_rollouts.md +++ b/doc/ci/environments/incremental_rollouts.md @@ -63,7 +63,7 @@ rollout 10%: ROLLOUT_PERCENTAGE: 10 ``` -When the jobs are built, a **play** button appears next to the job's name. Click the **play** button +When the jobs are built, a **play** button appears next to the job's name. Select **play** to release each stage of pods. You can also rollback by running a lower percentage job. Once 100% is reached, you cannot roll back using this method. It is still possible to roll back by redeploying the old version using the **Rollback** button on the environment page. diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 7e1ef5efaa5..b13bd041d46 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -646,7 +646,7 @@ To delete a stopped environment in the GitLab UI: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208655) in GitLab 13.2. -You can define a job that accesses an environment for various purposes, such as verification or preparation. This +You can define a job that accesses an environment for various purposes, such as verification or preparation. This effectively bypasses deployment creation, so that you can adjust your CD workflow more accurately. To do so, add either `action: prepare`, `action: verify`, or `action: access` to the `environment` section of your job: diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index adc215c7aa1..35ee4f9dd33 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -127,20 +127,16 @@ they have the following privileges: ## Deployment-only access to protected environments Users granted access to a protected environment, but not push or merge access -to the branch deployed to it, are only granted access to deploy the environment. An individual in a -group with the Reporter role, or in groups added to the project with the Reporter -role, appears in the dropdown menu for deployment-only access. +to the branch deployed to it, are only granted access to deploy the environment. +[Invited groups](../../user/project/members/share_project_with_groups.md#share-a-project-with-a-group-of-users) added +to the project with [Reporter role](../../user/permissions.md#project-members-permissions), appear in the dropdown menu for deployment-only access. To add deployment-only access: -1. Add a group with the Reporter role. -1. Add users to the group. -1. Invite the group to be a project member. +1. Create a group with members who are granted to access to the protected environment, if it doesn't exist yet. +1. [Invite the group](../../user/project/members/share_project_with_groups.md#share-a-project-with-a-group-of-users) to the project with the Reporter role. 1. Follow the steps in [Protecting Environments](#protecting-environments). -Note that deployment-only access is the only possible access level for groups with the Reporter -role. - ## Modifying and unprotecting environments Maintainers can: @@ -208,7 +204,7 @@ configured: deployment ruleset. - Developers should be given no more than the Developer role for the top-level group, or explicitly given the Maintainer role for a child project - They do *NOT* have access to the CI/CD configurations in the + They do *not* have access to the CI/CD configurations in the top-level group, so operators can ensure that the critical configuration won't be accidentally changed by the developers. - For sub-groups and child projects: @@ -232,17 +228,23 @@ Having this configuration in place: ### Protect critical environments under a group -To protect a group-level environment: +To protect a group-level environment, make sure your environments have the correct +[`deployment_tier`](index.md#deployment-tier-of-environments) defined in `.gitlab-ci.yml`. -1. Make sure your environments have the correct - [`deployment_tier`](index.md#deployment-tier-of-environments) defined in - `.gitlab-ci.yml`. -1. Configure the group-level protected environments by using the - [REST API](../../api/group_protected_environments.md). +#### Using the UI -NOTE: -Configuration [with the UI](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) -is scheduled for a later release. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/325249) in GitLab 15.1. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Protected environments**. +1. From the **Environment** list, select the [deployment tier of environments](index.md#deployment-tier-of-environments) you want to protect. +1. In the **Allowed to deploy** list, select the [subgroups](../../user/group/subgroups/index.md) you want to give deploy access to. +1. Select **Protect**. + +#### Using the API + +Configure the group-level protected environments by using the [REST API](../../api/group_protected_environments.md). ## Deployment approvals diff --git a/doc/ci/img/ecs_dashboard_v12_9.png b/doc/ci/img/ecs_dashboard_v12_9.png Binary files differdeleted file mode 100644 index 62b85c75ec2..00000000000 --- a/doc/ci/img/ecs_dashboard_v12_9.png +++ /dev/null diff --git a/doc/ci/index.md b/doc/ci/index.md index 05e97613384..4fe2dee32bf 100644 --- a/doc/ci/index.md +++ b/doc/ci/index.md @@ -89,7 +89,7 @@ GitLab CI/CD features, grouped by DevOps stage, include: | [GitLab CI/CD for external repositories](ci_cd_for_external_repos/index.md) | Get the benefits of GitLab CI/CD combined with repositories in GitHub and Bitbucket Cloud. | | [Interactive Web Terminals](interactive_web_terminal/index.md) | Open an interactive web terminal to debug the running jobs. | | [Review Apps](review_apps/index.md) | Configure GitLab CI/CD to preview code changes. | -| [Unit test reports](unit_test_reports.md) | Identify test failures directly on merge requests. | +| [Unit test reports](testing/unit_test_reports.md) | Identify test failures directly on merge requests. | | [Using Docker images](docker/using_docker_images.md) | Use GitLab and GitLab Runner with Docker to build and test applications. | | **Release** | | | [Auto Deploy](../topics/autodevops/stages.md#auto-deploy) | Deploy your application to a production environment in a Kubernetes cluster. | diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 8e5c8c8ab45..9567bc9cd65 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -20,7 +20,7 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Get job artifacts](../../api/job_artifacts.md#get-job-artifacts). - [Get job token's job](../../api/jobs.md#get-job-tokens-job). - [Pipeline triggers](../../api/pipeline_triggers.md), using the `token=` parameter. -- [Releases](../../api/releases/index.md). +- [Releases](../../api/releases/index.md) and [Release links](../../api/releases/links.md). - [Terraform plan](../../user/infrastructure/index.md). NOTE: @@ -28,7 +28,7 @@ There's an open issue, [GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), which prevents you from using a job token with internal projects. This bug only impacts self-managed GitLab instances. - + The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to @@ -104,13 +104,12 @@ The job token scope is only for controlling access to private projects. There is [a proposal](https://gitlab.com/groups/gitlab-org/-/epics/3559) to improve the feature with more strategic control of the access permissions. -## Trigger a multi-project pipeline by using a CI job token +## Trigger a multi-project pipeline by using a CI/CD job token > `CI_JOB_TOKEN` for multi-project pipelines was [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) from GitLab Premium to GitLab Free in 12.4. -You can use the `CI_JOB_TOKEN` to trigger [multi-project pipelines](../pipelines/multi_project_pipelines.md) -from a CI/CD job. A pipeline triggered this way creates a dependent pipeline relation -that is visible on the [pipeline graph](../pipelines/multi_project_pipelines.md#multi-project-pipeline-visualization). +You can use the `CI_JOB_TOKEN` to [trigger multi-project pipelines](../../api/pipeline_triggers.md#trigger-a-pipeline-with-a-token) +from a CI/CD job. For example: diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 42827dc2d48..1f7a7436c0a 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -129,7 +129,7 @@ they are collapsed into a single group in regular pipeline graphs (not the mini You can recognize when a pipeline has grouped jobs if you don't see the retry or cancel button inside them. Hovering over them shows the number of grouped -jobs. Click to expand them. +jobs. Select to expand them.  @@ -266,7 +266,7 @@ In this example: When running manual jobs you can supply additional job specific variables. You can do this from the job page of the manual job you want to run with -additional variables. To access this page, click on the **name** of the manual job in +additional variables. To access this page, select the **name** of the manual job in the pipeline view, *not* the play (**{play}**) button. This is useful when you want to alter the execution of a job that uses @@ -328,7 +328,7 @@ job1: - echo -e "\e[0Ksection_end:`date +%s`:my_first_section\r\e[0K" ``` -Depending on the shell that your runner uses, for example if it is using ZSH, you may need to +Depending on the shell that your runner uses, for example if it is using Zsh, you may need to escape the special characters like so: `\\e` and `\\r`. In the example above: @@ -337,7 +337,7 @@ In the example above: - `my_first_section`: The name given to the section. - `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored) job log, but they are displayed in the raw job log. To see them, in the top right - of the job log, click **{doc-text}** (**Show complete raw**). + of the job log, select **{doc-text}** (**Show complete raw**). - `\r`: carriage return. - `\e[0K`: clear line ANSI escape code. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 542d55665c7..03869a639f1 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9788) in GitLab 11.10. Requires GitLab Runner 11.10 and above. -GitLab provides a lot of great reporting tools for things like [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), and performance tests. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. +GitLab provides a lot of great reporting tools for things like [merge requests](../user/project/merge_requests/index.md) - [Unit test reports](testing/unit_test_reports.md), [code quality](../user/project/merge_requests/code_quality.md), and performance tests. While JUnit is a great open framework for tests that "pass" or "fail", it is also important to see other types of metrics from a given change. You can configure your job to use custom Metrics Reports, and GitLab displays a report on the merge request so that it's easier and faster to identify changes without having to check the entire log. diff --git a/doc/ci/pipeline_editor/index.md b/doc/ci/pipeline_editor/index.md index 57a9d7a9358..d87b336224c 100644 --- a/doc/ci/pipeline_editor/index.md +++ b/doc/ci/pipeline_editor/index.md @@ -53,12 +53,8 @@ reflected in the CI lint. It displays the same results as the existing [CI Lint ## View included CI/CD configuration -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7064) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `pipeline_editor_file_tree`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) -named `pipeline_editor_file_tree`. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7064) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `pipeline_editor_file_tree`. Disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357219) in GitLab 15.1. You can review configuration added with the [`include`](../yaml/index.md#include) keyword in the pipeline editor. In the top right, select the file tree (**{file-tree}**) diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 2b18b1d353b..ff30d5701cc 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -74,6 +74,8 @@ If you set a quota for a subgroup, it is not used. ## View CI/CD minutes used by a group +> Displaying shared runners duration per project [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/355666) in GitLab 15.0. + You can view the number of CI/CD minutes being used by a group. Prerequisite: @@ -88,6 +90,10 @@ To view CI/CD minutes being used for your group:  +The projects list shows projects with CI/CD minute usage or shared runners usage +in the current month only. The list includes all projects in the namespace and its +subgroups, sorted in descending order of CI/CD minute usage. + ## View CI/CD minutes used by a personal namespace > Displaying shared runners duration [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345795) in GitLab 15.0. @@ -98,6 +104,10 @@ You can view the number of CI/CD minutes being used by a personal namespace: 1. Select **Edit profile**. 1. On the left sidebar, select **Usage Quotas**. +The projects list shows [personal projects](../../user/project/working_with_projects.md#view-personal-projects) +with CI/CD minutes usage or shared runners usage in the current month only. The list +is sorted in descending order of CI/CD minute usage. + ## Purchase additional CI/CD minutes **(FREE SAAS)** If you're using GitLab SaaS, you can purchase additional packs of CI/CD minutes. @@ -129,6 +139,7 @@ so be sure to select the correct group. 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Usage Quotas**. +1. Select **Pipelines**. 1. Select **Buy additional minutes**. 1. Complete the details of the transaction. @@ -185,9 +196,9 @@ the duration of the pipeline if many jobs ran at the same time. The cost factor for a job running on a shared runner is: - `0.008` for public projects on GitLab SaaS, if [created 2021-07-17 or later](https://gitlab.com/gitlab-org/gitlab/-/issues/332708). - (For every 125 minutes of job time, you accrue 1 CD/CD minute.) + (For every 125 minutes of job time, you accrue 1 CI/CD minute.) - `0.008` for projects members of GitLab [Open Source program](../../subscriptions/index.md#gitlab-for-open-source). - (For every 125 minutes of job time, you accrue 1 CD/CD minute.) + (For every 125 minutes of job time, you accrue 1 CI/CD minute.) - `0` for public projects on GitLab self-managed instances, and for GitLab SaaS public projects created before 2021-07-17. - `1` for internal and private projects. @@ -198,7 +209,7 @@ GitLab SaaS shared runners have different cost factors, depending on the runner | GitLab SaaS runner type | Virtual machine configuration | CI/CD minutes cost factor | | :--------- | :------------------- | :--------- | | Linux OS + Docker executor| 1 vCPU, 3.75 GB RAM |1| -| macOS + shell executor | 4 vCPU, 10 GB RAM| 6 | +| macOS + shell executor | 4 vCPU, 10 GB RAM| 6 | ### Monthly reset of CI/CD minutes diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index c6142ebefc5..76419e61661 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -287,9 +287,15 @@ page, then selecting **Delete**.  +Deleting a pipeline does not automatically delete its +[child pipelines](parent_child_pipelines.md). +See the [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/39503) +for details. + WARNING: -Deleting a pipeline expires all pipeline caches, and deletes all related objects, -such as builds, logs, artifacts, and triggers. **This action cannot be undone.** +Deleting a pipeline expires all pipeline caches, and deletes all immediately +related objects, such as builds, logs, artifacts, and triggers. +**This action cannot be undone.** ### Pipeline security on protected branches @@ -451,12 +457,15 @@ For information on adding pipeline badges to projects, see [Pipeline badges](set ### Downstream pipelines -> Cancel or retry downstream pipelines from the graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. - In the pipeline graph view, downstream pipelines ([Multi-project pipelines](multi_project_pipelines.md) and [Parent-child pipelines](parent_child_pipelines.md)) display as a list of cards on the right of the graph. +#### Cancel or retry downstream pipelines from the graph view + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - [Generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**) on the pipeline's card. diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index fa8041671dc..b0336d0499f 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -375,13 +375,14 @@ job artifacts are deleted. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. > - [Made optional with a CI/CD setting](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8. -By default artifacts are always kept for the most recent successful pipeline for +By default artifacts are always kept for successful pipelines for the most recent commit on each ref. This means that the latest artifacts do not immediately expire according to the `expire_in` specification. -If a new pipeline for the same ref completes successfully, the previous pipeline's +If a pipeline for a new commit on the same ref completes successfully, the previous pipeline's artifacts are deleted according to the `expire_in` configuration. The artifacts -of the new pipeline are kept automatically. +of the new pipeline are kept automatically. If multiple pipelines run for the most +recent commit on the ref, all artifacts are kept. Keeping the latest artifacts can use a large amount of storage space in projects with a lot of jobs or large artifacts. If the latest artifacts are not needed in diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index d200dde143c..e0fe59d1ab8 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9186) in GitLab 12.0. > - [Squash and merge](../../user/project/merge_requests/squash_and_merge.md) support [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13001) in GitLab 12.6. -For more information about why you might want to use merge trains, read [How merge trains keep your master green](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). +For more information about why you might want to use merge trains, read [How starting merge trains improve efficiency for DevOps](https://about.gitlab.com/blog/2020/01/30/all-aboard-merge-trains/). When [merged results pipelines](merged_results_pipelines.md) are enabled, the pipeline jobs run as if the changes from your source branch have already diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 3206345d757..3fd739087ec 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -48,6 +48,9 @@ include the child pipeline configuration. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For an overview, see [Parent-Child Pipelines feature demo](https://youtu.be/n8KpBSqZNbk). +NOTE: +The artifact containing the generated YAML file must not be larger than 5MB. + ## Examples The simplest case is [triggering a child pipeline](../yaml/index.md#trigger) using a diff --git a/doc/ci/pipelines/pipeline_efficiency.md b/doc/ci/pipelines/pipeline_efficiency.md index 94cfeff3acc..991b3aef76c 100644 --- a/doc/ci/pipelines/pipeline_efficiency.md +++ b/doc/ci/pipelines/pipeline_efficiency.md @@ -74,7 +74,7 @@ hosted with a paid cloud service may be provisioned with: The [Pipeline success and duration charts](index.md#pipeline-success-and-duration-charts) give information about pipeline runtime and failed job counts. -Tests like [unit tests](../unit_test_reports.md), integration tests, end-to-end tests, +Tests like [unit tests](../testing/unit_test_reports.md), integration tests, end-to-end tests, [code quality](../../user/project/merge_requests/code_quality.md) tests, and others ensure that problems are automatically found by the CI/CD pipeline. There could be many pipeline stages involved causing long runtimes. diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index 2a95a4e1421..40daf154a5f 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -31,19 +31,43 @@ To change the visibility of your pipelines and related features: 1. Select or clear the **Public pipelines** checkbox. When it is selected, pipelines and related features are visible: - - For **public** projects, to everyone. - - For **internal** projects, to all logged-in users except [external users](../../user/permissions.md#external-users). - - For **private** projects, to all project members (Guest or higher). + - For [**Public**](../../user/public_access.md) projects, to everyone. + - For **Internal** projects, to all logged-in users except [external users](../../user/permissions.md#external-users). + - For **Private** projects, to all project members (Guest or higher). When it is cleared: - - For **public** projects, job logs, job artifacts, the pipeline security dashboard, + - For **Public** projects, job logs, job artifacts, the pipeline security dashboard, and the **CI/CD** menu items are visible only to project members (Reporter or higher). Other users, including guest users, can only view the status of pipelines and jobs, and only when viewing merge requests or commits. - - For **internal** projects, pipelines are visible to all logged in users except [external users](../../user/permissions.md#external-users). + - For **Internal** projects, pipelines are visible to all logged in users except [external users](../../user/permissions.md#external-users). Related features are visible only to project members (Reporter or higher). - - For **private** projects, pipelines and related features are visible to project members (Reporter or higher) only. + - For **Private** projects, pipelines and related features are visible to project members (Reporter or higher) only. + +### Change pipeline visibility for non-project members in public projects + +You can control the visibility of pipelines for non-project members in [public projects](../../user/public_access.md). + +This setting has no effect when: + +- Project visibility is set to [**Internal** or **Private**](../../user/public_access.md), + because non-project members cannot access internal or private projects. +- The [**Public pipelines**](#change-which-users-can-view-your-pipelines) setting is disabled. + +To change the pipeline visibility for non-project members: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > General**. +1. Expand **Visibility, project features, permissions**. +1. For **CI/CD**, choose: + - **Only project members**: Only project members can view pipelines. + - **Everyone With Access**: Non-project members can also view pipelines. +1. Select **Save changes**. + +The [CI/CD permissions table](../../user/permissions.md#gitlab-cicd-permissions) +lists the pipeline features non-project members can access when **Everyone With Access** +is selected. ## Auto-cancel redundant pipelines @@ -92,7 +116,7 @@ For more information, see [Deployment safety](../environments/deployment_safety. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211339) in GitLab 13.6. A deployment job can fail because a newer one has run. If you retry the failed deployment job, the -environment could be overwritten with older source code. If you click **Retry**, a modal warns you +environment could be overwritten with older source code. If you select **Retry**, a modal warns you about this and asks for confirmation. For more information, see [Deployment safety](../environments/deployment_safety.md). @@ -273,7 +297,7 @@ Use this regex for commonly used test tools. <!-- vale gitlab.Spelling = NO --> - Simplecov (Ruby). Example: `\(\d+.\d+\%\) covered`. -- pytest-cov (Python). Example: `^TOTAL.+?(\d+\%)$`. +- pytest-cov (Python). Example: `(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$`. - Scoverage (Scala). Example: `Statement coverage[A-Za-z\.*]\s*:\s*([^%]+)`. - `phpunit --coverage-text --colors=never` (PHP). Example: `^\s*Lines:\s*\d+.\d+\%`. - gcovr (C/C++). Example: `^TOTAL.*\s+(\d+\%)$`. @@ -287,6 +311,7 @@ Use this regex for commonly used test tools. - .NET (OpenCover). Example: `(Visited Points).*\((.*)\)`. - .NET (`dotnet test` line coverage). Example: `Total\s*\|\s*(\d+(?:\.\d+)?)`. - tarpaulin (Rust). Example: `^\d+.\d+% coverage`. +- Pester (PowerShell). Example: `Covered (\d+\.\d+%)`. <!-- vale gitlab.Spelling = YES --> diff --git a/doc/ci/quick_start/index.md b/doc/ci/quick_start/index.md index e35a042a320..0369824c92e 100644 --- a/doc/ci/quick_start/index.md +++ b/doc/ci/quick_start/index.md @@ -79,7 +79,7 @@ To create a `.gitlab-ci.yml` file: 1. On the left sidebar, select **Project information > Details**. 1. Above the file list, select the branch you want to commit to, - click the plus icon, then select **New file**: + select the plus icon, then select **New file**:  @@ -115,7 +115,7 @@ To create a `.gitlab-ci.yml` file: [predefined variables](../variables/predefined_variables.md) that populate when the job runs. -1. Click **Commit changes**. +1. Select **Commit changes**. The pipeline starts when the commit is committed. @@ -172,11 +172,11 @@ To view your pipeline:  -- To view a visual representation of your pipeline, click the pipeline ID. +- To view a visual representation of your pipeline, select the pipeline ID.  -- To view details of a job, click the job name, for example, `deploy-prod`. +- To view details of a job, select the job name, for example, `deploy-prod`.  diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 56e8b96d11f..3ebd38df8db 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -58,7 +58,7 @@ The process of configuring Review Apps is as follows: 1. [Install](https://docs.gitlab.com/runner/install/) and [configure](https://docs.gitlab.com/runner/commands/) a runner to do deployment. 1. Set up a job in `.gitlab-ci.yml` that uses the [predefined CI/CD variable](../variables/index.md) `${CI_COMMIT_REF_SLUG}` to create dynamic environments and restrict it to run only on branches. - Alternatively, you can get a YML template for this job by [enabling review apps](#enable-review-apps-button) for your project. + Alternatively, you can get a YAML template for this job by [enabling review apps](#enable-review-apps-button) for your project. 1. Optionally, set a job that [manually stops](../environments/index.md#stop-an-environment) the Review Apps. ### Enable Review Apps button diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index e7165339ea0..efd78fac2c6 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -85,9 +85,9 @@ To protect or unprotect a runner: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. 1. Find the runner you want to protect or unprotect. Make sure it's enabled. -1. Click the pencil button. +1. Select the pencil button. 1. Check the **Protected** option. -1. Click **Save changes**. +1. Select **Save changes**.  @@ -113,7 +113,7 @@ To reset the registration token: 1. Go to the project's **Settings > CI/CD**. 1. Expand the **General pipelines settings** section. -1. Find the **Runner token** form field and click the **Reveal value** button. +1. Find the **Runner token** form field and select **Reveal value**. 1. Delete the value and save the form. 1. After the page is refreshed, expand the **Runners settings** section and check the registration token - it should be changed. @@ -193,16 +193,16 @@ To make a runner pick untagged jobs: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. 1. Find the runner you want to pick untagged jobs and make sure it's enabled. -1. Click the pencil button. +1. Select the pencil button. 1. Check the **Run untagged jobs** option. -1. Click the **Save changes** button for the changes to take effect. +1. Select **Save changes** for the changes to take effect. NOTE: The runner tags list can not be empty when it's not allowed to pick untagged jobs. Below are some example scenarios of different variations. -### runner runs only tagged jobs +### Runner runs only tagged jobs The following examples illustrate the potential impact of the runner being set to run only tagged jobs. @@ -222,7 +222,7 @@ Example 3: 1. The runner is configured to run only tagged jobs and has the `docker` tag. 1. A job that has no tags defined is executed and stuck. -### runner is allowed to run untagged jobs +### Runner is allowed to run untagged jobs The following examples illustrate the potential impact of the runner being set to run tagged and untagged jobs. @@ -281,6 +281,17 @@ variables: - echo "Hello runner selector feature" ``` +## Runner statuses + +A runner can have one of the following statuses. + +| Status | Description | +|--------|-------------| +| **online** | The runner has contacted GitLab within the last 2 hours and is available to run jobs. | +| **offline** | The runner has not contacted GitLab in more than 2 hours and is not available to run jobs. Check the runner to see if you can bring it online. | +| **stale** | The runner has not contacted GitLab in more than 3 months. If the runner was created more than 3 months ago, but it never contacted the instance, it is also considered **stale**. | +| **never_contacted** | The runner has never contacted GitLab. To make the runner contact GitLab, run `gitlab-runner run`. | + ## Configure runner behavior with variables You can use [CI/CD variables](../variables/index.md) to configure runner Git behavior @@ -706,3 +717,175 @@ variables: | `ARTIFACT_COMPRESSION_LEVEL` | To adjust compression ratio, set to `fastest`, `fast`, `default`, `slow`, or `slowest`. This setting works with the Fastzip archiver only, so the GitLab Runner feature flag [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) must also be enabled. | | `CACHE_COMPRESSION_LEVEL` | To adjust compression ratio, set to `fastest`, `fast`, `default`, `slow`, or `slowest`. This setting works with the Fastzip archiver only, so the GitLab Runner feature flag [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) must also be enabled. | | `CACHE_REQUEST_TIMEOUT` | Configure the maximum duration of cache upload and download operations for a single job in minutes. Default is `10` minutes. | + +## Artifact attestation + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28940) in GitLab Runner 15.1. + +GitLab Runner can generate and produce attestation metadata for all build artifacts. To enable this feature, you must set the `RUNNER_GENERATE_ARTIFACTS_METADATA` environment variable to `true`. This variable can either be set globally or it can be set for individual jobs. The metadata is in rendered in a plain text `.json` file that's stored with the artifact. The file name is as follows: `{JOB_ID}-artifacts-metadata.json`. + +### Attestation format + +The attestation metadata is generated in the [in-toto attestation format](https://github.com/in-toto/attestation) for spec version [v0.1](https://in-toto.io/Statement/v0.1). The following fields are populated by default: + +| Field | Value | +| ------ | ------ | +| `_type` | `https://in-toto.io/Statement/v0.1` | +| `subject.name` | The filename of the artifact. | +| `subject.digest.sha256` | The artifact's `sha256` checksum. | +| `predicateType` | `https://slsa.dev/provenance/v0.2` | +| `predicate.buildType` | `https://gitlab.com/gitlab-org/gitlab-runner/-/blob/{GITLAB_RUNNER_VERSION}/PROVENANCE.md`. For example v15.0.0 | +| `predicate.builder.id` | A URI pointing to the runner details page, for example `https://gitlab.com/gitlab-com/www-gitlab-com/-/runners/3785264`. | +| `predicate.invocation.configSource.uri` | ``https://gitlab.example.com/.../{PROJECT_NAME}`` | +| `predicate.invocation.configSource.digest.sha256` | The repository's `sha256` checksum. | +| `predicate.invocation.configSource.entryPoint` | The name of the CI job that triggered the build. | +| `predicate.invocation.environment.name` | The name of the runner. | +| `predicate.invocation.environment.executor` | The runner executor. | +| `predicate.invocation.environment.architecture` | The architecture on which the CI job is run. | +| `predicate.invocation.parameters` | The names of any CI/CD or environment variables that were present when the build command was run. The value is always represented as an empty string to avoid leaking any secrets. | +| `metadata.buildStartedOn` | The time when the build was started. `RFC3339` formatted. | +| `metadata.buildEndedOn` | The time when the build ended. Since metadata generation happens during the build this moment in time will be slightly earlier than the one reported in GitLab. `RFC3339` formatted. | +| `metadata.reproducible` | Whether the build is reproducible by gathering all the generated metadata. Always `false`. | +| `metadata.completeness.parameters` | Whether the parameters are supplied. Always `true`. | +| `metadata.completeness.environment` | Whether the builder's environment is reported. Always `true`. | +| `metadata.completeness.materials` | Whether the build materials are reported. Always `false`. | + +An example of an attestation that the GitLab Runner might generate is as follows: + +```yaml +{ + "_type": "https://gitlab.com/gitlab-org/gitlab-runner/-/blob/v15.1.0/PROVENANCE.md", + "subject": [ + { + "name": "script.sh", + "digest": { + "sha256": "f5ae5ced234922eebe6461d32228ba8ab9c3d0c0f3983a3bef707e6e1a1ab52a" + } + } + ], + "predicateType": "https://slsa.dev/provenance/v0.2", + "predicate": { + "buildType": "https://gitlab.com/gitlab-org/gitlab-runner/-/blob/v15.1.0/PROVENANCE.md", + "builder": { + "id": "https://gitlab.com/ggeorgiev_gitlab/playground/-/runners/14811533" + }, + "invocation": { + "configSource": { + "uri": "https://gitlab.com/ggeorgiev_gitlab/playground", + "digest": { + "sha256": "f0582e2c9a16b5cc2cde90e8be8f1b50fd67c631" + }, + "entryPoint": "whoami shell" + }, + "environment": { + "name": "local", + "executor": "shell", + "architecture": "amd64" + }, + "parameters": { + "CI_PIPELINE_ID": "", + "CI_PIPELINE_URL": "", + // All other CI variable names are listed here. Values are always represented as empty strings to avoid leaking secrets. + } + }, + "metadata": { + "buildStartedOn": "2022-06-17T00:47:27+03:00", + "buildFinishedOn": "2022-06-17T00:47:28+03:00", + "completeness": { + "parameters": true, + "environment": true, + "materials": false + }, + "reproducible": false + }, + "materials": [] + } +} +``` + +### Staging directory + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3403) in GitLab Runner 15.0. + +If you do not want to archive cache and artifacts in the system's default temporary directory, you can specify a different directory. + +You might need to change the directory if your system's default temporary path has constraints. +If you use a fast disk for the directory location, it can also improve performance. + +To change the directory, set `ARCHIVER_STAGING_DIR` as a variable in your CI job, or use a runner variable when you register the runner (`gitlab register --env ARCHIVER_STAGING_DIR=<dir>`). + +The directory you specify is used as the location for downloading artifacts prior to extraction. If the `fastzip` archiver is +used, this location is also used as scratch space when archiving. + +### Configure `fastzip` to improve performance + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3130) in GitLab Runner 15.0. + +To tune `fastzip`, ensure the [`FF_USE_FASTZIP`](https://docs.gitlab.com/runner/configuration/feature-flags.html#available-feature-flags) flag is enabled. +Then use any of the following environment variables. + +| Variable | Description | +|---------------------------------|--------------------------------------------------------| +| `FASTZIP_ARCHIVER_CONCURRENCY` | The number of files to be concurrently compressed. Default is the number of CPUs available. | +| `FASTZIP_ARCHIVER_BUFFER_SIZE` | The buffer size allocated per concurrency for each file. Data exceeding this number moves to scratch space. Default is 2 MiB. | +| `FASTZIP_EXTRACTOR_CONCURRENCY` | The number of files to be concurrency decompressed. Default is the number of CPUs available. | + +Files in a zip archive are appended sequentially. This makes concurrent compression challenging. `fastzip` works around +this limitation by compressing files concurrently to disk first, and then copying the result back to zip archive +sequentially. + +To avoid writing to disk and reading the contents back for smaller files, a small buffer per concurrency is used. This setting +can be controlled with `FASTZIP_ARCHIVER_BUFFER_SIZE`. The default size for this buffer is 2 MiB, therefore, a +concurrency of 16 will allocate 32 MiB. Data that exceeds the buffer size will be written to and read back from disk. +Therefore, using no buffer, `FASTZIP_ARCHIVER_BUFFER_SIZE: 0`, and only scratch space is a valid option. + +`FASTZIP_ARCHIVER_CONCURRENCY` controls how many files are compressed concurrency. As mentioned above, this setting +therefore can increase how much memory is being used, but also how much temporary data is written to the scratch space. +The default is the number of CPUs available, but given the memory ramifications, this may not always be the best +setting. + +`FASTZIP_EXTRACTOR_CONCURRENCY` controls how many files are decompressed at once. Files from a zip archive can natively +be read from concurrency, so no additional memory is allocated in additional to what the decompressor requires. This +defaults to the number of CPUs available. + +## Clean up stale runners **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363012) in GitLab 15.1. + +You can clean up group runners that have been inactive for more than three months. + +Group runners are those that were created at the group level. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Runners**. +1. Turn on the **Enable stale runner cleanup** toggle. + +### View stale runner cleanup logs + +You can check the [Sidekiq logs](../../administration/logs.md#sidekiq-logs) to see the cleanup result. In Kibana you can use the following query: + +```json +{ + "query": { + "match_phrase": { + "json.class.keyword": "Ci::Runners::StaleGroupRunnersPruneCronWorker" + } + } +} +``` + +Filter entries where stale runners were removed: + +```json +{ + "query": { + "range": { + "json.extra.ci_runners_stale_group_runners_prune_cron_worker.total_pruned": { + "gte": 1, + "lt": null + } + } + } +} +``` diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 257dc02805b..729de4d99d3 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -7,16 +7,16 @@ type: reference # Runner SaaS **(FREE SAAS)** -If you are using self-managed GitLab or you use GitLab.com but want to use your own runners, you can -[install and configure your own runners](https://docs.gitlab.com/runner/install/). - -If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. +If you use GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. No configuration is required. Your jobs can run on: - [Linux runners](saas/linux_saas_runner.md). - [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). -- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). +- [macOS runners](saas/macos_saas_runner.md) ([Limited Availability](../../policy/alpha-beta-support.md#limited-availability-la)). The number of minutes you can use on these runners depends on the [maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) in your [subscription plan](https://about.gitlab.com/pricing/). + +If you use self-managed GitLab or you use GitLab.com but want to use your own runners, you can +[install and configure your own runners](https://docs.gitlab.com/runner/install/). diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 6082a17d001..8b2753d26f1 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -84,7 +84,7 @@ To disable shared runners for a group: 1. Go to the group's **Settings > CI/CD** and expand the **Runners** section. 1. In the **Shared runners** area, turn off the **Enable shared runners for this group** toggle. 1. Optionally, to allow shared runners to be enabled for individual projects or subgroups, - click **Allow projects and subgroups to override the group setting**. + select **Allow projects and subgroups to override the group setting**. NOTE: To re-enable the shared runners for a group, turn on the @@ -200,11 +200,11 @@ You must have the Owner role for the group. 1. Go to the group you want to remove or pause the runner for. 1. On the left sidebar, select **CI/CD > Runners**. -1. Click **Pause** or **Remove runner**. +1. Select **Pause** or **Remove runner**. - If you pause a group runner that is used by multiple projects, the runner pauses for all projects. - From the group view, you cannot remove a runner that is assigned to more than one project. You must remove it from each project first. -1. On the confirmation dialog, click **OK**. +1. On the confirmation dialog, select **OK**. ## Specific runners @@ -226,33 +226,47 @@ A fork *does* copy the CI/CD settings of the cloned repository. ### Create a specific runner You can create a specific runner for your self-managed GitLab instance or for GitLab.com. -You must have the Owner role for the project. + +Prerequisite: + +- You must have at least the Maintainer role for the project. To create a specific runner: -1. [Install runner](https://docs.gitlab.com/runner/install/). -1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. Note the URL and token. +1. [Install GitLab Runner](https://docs.gitlab.com/runner/install/). +1. On the top bar, select **Menu > Projects** and find the project where you want to use the runner. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Runners**. +1. In the **Specific runners** section, note the URL and token. 1. [Register the runner](https://docs.gitlab.com/runner/register/). -### Enable a specific runner for a specific project +The runner is now enabled for the project. -A specific runner is available in the project it was created for. An administrator can -enable a specific runner to apply to additional projects. +### Enable a specific runner for a different project -- You must have the Owner role for the - project. +After a specific runner is created, you can enable it for other projects. + +Prerequisites: +You must have at least the Maintainer role for: + +- The project where the runner is already enabled. +- The project where you want to enable the runner. - The specific runner must not be [locked](#prevent-a-specific-runner-from-being-enabled-for-other-projects). -To enable or disable a specific runner for a project: +To enable a specific runner for a project: -1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. -1. Click **Enable for this project** or **Disable for this project**. +1. On the top bar, select **Menu > Projects** and find the project where you want to enable the runner. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **General pipelines**. +1. Expand **Runners**. +1. By the runner you want, select **Enable for this project**. You can edit a specific runner from any of the projects it's enabled for. -The modifications, which include unlocking, editing tags and the description, +The modifications, which include unlocking and editing tags and the description, affect all projects that use the runner. +An administrator can [enable the runner for multiple projects](../../user/admin_area/settings/continuous_integration.md#enable-a-specific-runner-for-multiple-projects). + ### Prevent a specific runner from being enabled for other projects You can configure a specific runner so it is "locked" and cannot be enabled for other projects. @@ -261,8 +275,9 @@ but can also be changed later. To lock or unlock a specific runner: -1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. +1. Go to the project's **Settings > CI/CD**. +1. Expand the **Runners** section. 1. Find the specific runner you want to lock or unlock. Make sure it's enabled. You cannot lock shared or group runners. -1. Click the pencil button. +1. Select **Edit** (**{pencil}**). 1. Check the **Lock to current projects** option. -1. Click **Save changes**. +1. Select **Save changes**. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index fd3fedbc3f5..f1e28f7e74d 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -18,7 +18,7 @@ region of the VMs is US East1. Each instance is used only for one job. This ensures that any sensitive data left on the system can't be accessed by other people's CI/CD jobs. NOTE: -The final disk space your jobs can use will be less than 25GB. Some disk space allocated to the instance will be occupied by the operating system, the Docker image, and a copy of your cloned repository. +The final disk space your jobs can use will be less than 25GB. Some disk space allocated to the instance will be occupied by the operating system, the Docker image, and a copy of your cloned repository. The `gitlab-shared-runners-manager-X.gitlab.com` fleet of runners are dedicated for GitLab projects as well as community forks of them. They use a slightly larger machine type (n1-standard-2) and have a bigger SSD disk size. They don't run untagged jobs and unlike the general fleet of shared runners, the instances are re-used up to 40 times. @@ -55,7 +55,7 @@ To use this feature, define a [CI/CD variable](../../../ci/variables/index.md#cu `CI_PRE_CLONE_SCRIPT` that contains a bash script. NOTE: -The `CI_PRE_CLONE_SCRIPT` variable does not work on Windows runners. +The `CI_PRE_CLONE_SCRIPT` variable does not work on GitLab SaaS Windows or macOS Runners. ### Pre-clone script example diff --git a/doc/ci/runners/saas/macos/codesigning.md b/doc/ci/runners/saas/macos/codesigning.md index 4f8316faf17..71fdc61f0b6 100644 --- a/doc/ci/runners/saas/macos/codesigning.md +++ b/doc/ci/runners/saas/macos/codesigning.md @@ -6,13 +6,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Code signing for SaaS runners on macOS -> Introduced in GitLab 15.0. - Before you can integrate GitLab with Apple services, install to a device, or deploy to the Apple App Store, you must [code sign](https://developer.apple.com/support/code-signing/) your application. To code sign an iOS project, you need the following files: -- A certifcate issued by Apple. +- A certificate issued by Apple. - A provisioning profile. ## Code signing iOS Projects with fastlane @@ -61,7 +59,7 @@ To use fastlane to code sign your application: get_provisioning_profile # match(type: "appstore",read_only: true) gym - upload_to_testflight + upload_to_testflight end end ``` diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index 0ab2de36f10..65404c89f9a 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # SaaS runners on macOS (Limited Availability) **(PREMIUM SAAS)** -SaaS runners on macOS are now in [Limited Availability](../../../policy/alpha-beta-support.md#beta-features) for approved open source programs and customers in Premium and Ultimate plans. +SaaS runners on macOS are in [Limited Availability](../../../policy/alpha-beta-support.md#limited-availability-la) for approved open source programs and customers in Premium and Ultimate plans. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). @@ -18,6 +18,15 @@ CI/CD minutes used on GitLab SaaS macOS runners are included in your CI/CD minut Refer to the CI/CD minutes [cost factor](../../../ci/pipelines/cicd_minutes.md#cost-factor) for the cost factor applied to the GitLab SaaS macOS runners. +## Access request process + +While in limited availability, to run CI jobs on the macOS runners, GitLab SaaS customer namespaces must be explicitly added to the macOS `allow-list`. Customers who participated in the beta have already been added. + +After you have been added, you can use the macOS runners for any projects in your namespace. + +To request access, open a [limited availability access request](https://gitlab.com/gitlab-com/runner-saas-macos-limited-availability/-/issues/new). +The expected turnaround for activation is two business days. + ## Quickstart To start using SaaS runners on macOS, you must be an active GitLab SaaS Premium or Ultimate customer. Participants in the GitLab Open Source program are also eligible to use the service. diff --git a/doc/ci/secure_files/index.md b/doc/ci/secure_files/index.md index 2bf360e69f1..fb421ab8944 100644 --- a/doc/ci/secure_files/index.md +++ b/doc/ci/secure_files/index.md @@ -20,6 +20,8 @@ are stored securely outside of your project's repository, and are not version co It is safe to store sensitive information in these files. Secure files support both plain text and binary file types. +You can manage secure files in the project settings, or with the [secure files API](../../api/secure_files.md). + Secure files can be [downloaded and used by CI/CD jobs](#use-secure-files-in-cicd-jobs) by using the [load-secure-files](https://gitlab.com/gitlab-org/incubation-engineering/devops-for-mobile-apps/load-secure-files) tool. @@ -30,49 +32,14 @@ Additional features and capabilities are planned. ## Add a secure file to a project -To add a secure file to a project, use the [Secure Files API](../../api/secure_files.md#create-secure-file). - -Send a POST request to the secure files endpoint for your project: - -```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ - "https://gitlab.example.com/api/v4/projects/:project_id/secure_files" --form "name=myfile.jks" --form "file=@/path/to/file/myfile.jks" -``` - -The response returns all of the metadata for the file you just uploaded. For example: - -```json -{ - "id": 1, - "name": "myfile.jks", - "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", - "checksum_algorithm": "sha256", - "created_at": "2022-02-22T22:22:22.222Z" -} -``` - -## List all secure files in a project - -To list all secure files in a project, use the [Secure Files API](../../api/secure_files.md#list-project-secure-files). +To add a secure file to a project: -Send a GET request to the secure files endpoint for your project: - -```shell -curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" \ - "https://gitlab.example.com/api/v4/projects/:project_id/secure_files" -``` - -The response returns an array of all of the secure files in the project. For example: - -```json -[{ - "id": 1, - "name": "myfile.jks", - "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", - "checksum_algorithm": "sha256", - "created_at": "2022-02-22T22:22:22.222Z" -}] -``` +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. In the **Secure Files** section, select **Manage**. +1. Select **Upload File**. +1. Find the file to upload, select **Open**, and the file upload begins immediately. + The file shows up in the list when the upload is complete. ## Use secure files in CI/CD jobs diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index e876c6d7326..3ab814200fb 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -201,7 +201,7 @@ end-to-end-tests: alias: firefox - name: registry.gitlab.com/organization/private-api:latest alias: backend-api - - postgres:9.6.19 + - postgres:14.3 variables: FF_NETWORK_PER_BUILD: 1 POSTGRES_PASSWORD: supersecretpassword diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 0bd43917cd1..c2ff4c60771 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -46,7 +46,7 @@ If you're wondering why we used `postgres` for the `Host`, read more at [How services are linked to the job](../services/index.md#how-services-are-linked-to-the-job). You can also use any other Docker image available on [Docker Hub](https://hub.docker.com/_/postgres). -For example, to use PostgreSQL 9.3, the service becomes `postgres:9.3`. +For example, to use PostgreSQL 14.3, the service becomes `postgres:14.3`. The `postgres` image can accept some environment variables. For more details, see the documentation on [Docker Hub](https://hub.docker.com/_/postgres). diff --git a/doc/ci/img/junit_test_report.png b/doc/ci/testing/img/junit_test_report.png Binary files differindex a4b98c8b910..a4b98c8b910 100644 --- a/doc/ci/img/junit_test_report.png +++ b/doc/ci/testing/img/junit_test_report.png diff --git a/doc/ci/img/pipelines_junit_test_report_v13_10.png b/doc/ci/testing/img/pipelines_junit_test_report_v13_10.png Binary files differindex ef79a2547af..ef79a2547af 100644 --- a/doc/ci/img/pipelines_junit_test_report_v13_10.png +++ b/doc/ci/testing/img/pipelines_junit_test_report_v13_10.png diff --git a/doc/ci/img/pipelines_junit_test_report_with_errors_v13_10.png b/doc/ci/testing/img/pipelines_junit_test_report_with_errors_v13_10.png Binary files differindex cfcf3bec76c..cfcf3bec76c 100644 --- a/doc/ci/img/pipelines_junit_test_report_with_errors_v13_10.png +++ b/doc/ci/testing/img/pipelines_junit_test_report_with_errors_v13_10.png diff --git a/doc/ci/testing/index.md b/doc/ci/testing/index.md new file mode 100644 index 00000000000..52af329873f --- /dev/null +++ b/doc/ci/testing/index.md @@ -0,0 +1,35 @@ +--- +stage: Verify +group: Pipeline Insights +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Test with GitLab CI/CD and generate reports in merge requests **(FREE)** + +Use GitLab CI/CD to test the changes included in a feature branch. You can also +display reports or link to important information directly from [merge requests](../../user/project/merge_requests/index.md). + +| Feature | Description | +|-------------------------------------------------------------------------------------------------|-------------| +| [Accessibility Testing](../../user/project/merge_requests/accessibility_testing.md) | Automatically report A11y violations for changed pages in merge requests. | +| [Browser Performance Testing](../../user/project/merge_requests/browser_performance_testing.md) | Quickly determine the browser performance impact of pending code changes. | +| [Load Performance Testing](../../user/project/merge_requests/load_performance_testing.md) | Quickly determine the server performance impact of pending code changes. | +| [Code Quality](../../user/project/merge_requests/code_quality.md) | Analyze your source code quality using the [Code Climate](https://codeclimate.com/) analyzer and show the Code Climate report right in the merge request widget area. | +| [Display arbitrary job artifacts](../yaml/index.md#artifactsexpose_as) | Configure CI pipelines with the `artifacts:expose_as` parameter to directly link to selected [artifacts](../pipelines/job_artifacts.md) in merge requests. | +| [Unit test reports](unit_test_reports.md) | Configure your CI jobs to use Unit test reports, and let GitLab display a report on the merge request so that it's easier and faster to identify the failure without having to check the entire job log. | +| [License Compliance](../../user/compliance/license_compliance/index.md) | Manage the licenses of your dependencies. | +| [Metrics Reports](../metrics_reports.md) | Display the Metrics Report on the merge request so that it's fast and easier to identify changes to important metrics. | +| [Test Coverage visualization](../../user/project/merge_requests/test_coverage_visualization.md) | See test coverage results for merge requests, in the file diff. | +| [Fail fast testing](../../user/project/merge_requests/fail_fast_testing.md#fail-fast-testing) | Run a subset of your RSpec test suite, so failed tests stop the pipeline before the full suite of tests run, saving resources. | + +## Security Reports **(ULTIMATE)** + +In addition to the reports listed above, GitLab can do many types of [Security reports](../../user/application_security/index.md), +generated by scanning and reporting any vulnerabilities found in your project: + +| Feature | Description | +|----------------------------------------------------------------------------------------------|-------------| +| [Container Scanning](../../user/application_security/container_scanning/index.md) | Analyze your Docker images for known vulnerabilities. | +| [Dynamic Application Security Testing (DAST)](../../user/application_security/dast/index.md) | Analyze your running web applications for known vulnerabilities. | +| [Dependency Scanning](../../user/application_security/dependency_scanning/index.md) | Analyze your dependencies for known vulnerabilities. | +| [Static Application Security Testing (SAST)](../../user/application_security/sast/index.md) | Analyze your source code for known vulnerabilities. | diff --git a/doc/ci/testing/unit_test_report_examples.md b/doc/ci/testing/unit_test_report_examples.md new file mode 100644 index 00000000000..a54deb254b7 --- /dev/null +++ b/doc/ci/testing/unit_test_report_examples.md @@ -0,0 +1,266 @@ +--- +stage: Verify +group: Pipeline Insights +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Unit test report examples **(FREE)** + +[Unit test reports](unit_test_reports.md) can be generated for many languages and packages. +Use these examples as guidelines for configuring your pipeline to generate unit test reports +for the listed languages and packages. You might need to edit the examples to match +the version of the language or package you are using. + +## Ruby + +Use the following job in `.gitlab-ci.yml`. This includes the `artifacts:paths` keyword to provide a link to the Unit test report output file. + +```yaml +## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report format XML file with rspec +ruby: + stage: test + script: + - bundle install + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml + artifacts: + when: always + paths: + - rspec.xml + reports: + junit: rspec.xml +``` + +## Go + +Use the following job in `.gitlab-ci.yml`: + +```yaml +## Use https://github.com/gotestyourself/gotestsum to generate a JUnit report format XML file with go +golang: + stage: test + script: + - go get gotest.tools/gotestsum + - gotestsum --junitfile report.xml --format testname + artifacts: + when: always + reports: + junit: report.xml +``` + +## Java + +There are a few tools that can produce JUnit report format XML file in Java. + +### Gradle + +In the following example, `gradle` is used to generate the test reports. +If there are multiple test tasks defined, `gradle` generates multiple +directories under `build/test-results/`. In that case, you can leverage glob +matching by defining the following path: `build/test-results/test/**/TEST-*.xml`: + +```yaml +java: + stage: test + script: + - gradle test + artifacts: + when: always + reports: + junit: build/test-results/test/**/TEST-*.xml +``` + +In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) +and later, you can use `**`. + +### Maven + +For parsing [Surefire](https://maven.apache.org/surefire/maven-surefire-plugin/) +and [Failsafe](https://maven.apache.org/surefire/maven-failsafe-plugin/) test +reports, use the following job in `.gitlab-ci.yml`: + +```yaml +java: + stage: test + script: + - mvn verify + artifacts: + when: always + reports: + junit: + - target/surefire-reports/TEST-*.xml + - target/failsafe-reports/TEST-*.xml +``` + +## Python example + +This example uses pytest with the `--junitxml=report.xml` flag to format the output +into the JUnit report XML format: + +```yaml +pytest: + stage: test + script: + - pytest --junitxml=report.xml + artifacts: + when: always + reports: + junit: report.xml +``` + +## C/C++ + +There are a few tools that can produce JUnit report format XML files in C/C++. + +### GoogleTest + +In the following example, `gtest` is used to generate the test reports. +If there are multiple `gtest` executables created for different architectures (`x86`, `x64` or `arm`), +you are required to run each test providing a unique filename. The results +are then aggregated together. + +```yaml +cpp: + stage: test + script: + - gtest.exe --gtest_output="xml:report.xml" + artifacts: + when: always + reports: + junit: report.xml +``` + +### CUnit + +[CUnit](https://cunity.gitlab.io/cunit/) can be made to produce [JUnit report format XML files](https://cunity.gitlab.io/cunit/group__CI.html) +automatically when run using its `CUnitCI.h` macros: + +```yaml +cunit: + stage: test + script: + - ./my-cunit-test + artifacts: + when: always + reports: + junit: ./my-cunit-test.xml +``` + +## .NET + +The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) NuGet +package can generate test reports for .Net Framework and .Net Core applications. The following +example expects a solution in the root folder of the repository, with one or more +project files in sub-folders. One result file is produced per test project, and each file +is placed in the artifacts folder. This example includes optional formatting arguments, which +improve the readability of test data in the test widget. A full .Net Core +[example is available](https://gitlab.com/Siphonophora/dot-net-cicd-test-logging-demo). + +```yaml +## Source code and documentation are here: https://github.com/spekt/junit.testlogger/ + +Test: + stage: test + script: + - 'dotnet test --test-adapter-path:. --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose"' + artifacts: + when: always + paths: + - ./**/*test-result.xml + reports: + junit: + - ./**/*test-result.xml +``` + +## JavaScript + +There are a few tools that can produce JUnit report format XML files in JavaScript. + +### Jest + +The [jest-junit](https://github.com/jest-community/jest-junit) npm package can generate +test reports for JavaScript applications. In the following `.gitlab-ci.yml` example, +the `javascript` job uses Jest to generate the test reports: + +```yaml +javascript: + stage: test + script: + - 'jest --ci --reporters=default --reporters=jest-junit' + artifacts: + when: always + reports: + junit: + - junit.xml +``` + +### Karma + +The [Karma-junit-reporter](https://github.com/karma-runner/karma-junit-reporter) +npm package can generate test reports for JavaScript applications. In the following +`.gitlab-ci.yml` example, the `javascript` job uses Karma to generate the test reports: + +```yaml +javascript: + stage: test + script: + - karma start --reporters junit + artifacts: + when: always + reports: + junit: + - junit.xml +``` + +### Mocha + +The [JUnit Reporter for Mocha](https://github.com/michaelleeallen/mocha-junit-reporter) +NPM package can generate test reports for JavaScript applications. In the following +`.gitlab-ci.yml` example, the `javascript` job uses Mocha to generate the test reports: + +```yaml +javascript: + stage: test + script: + - mocha --reporter mocha-junit-reporter --reporter-options mochaFile=junit.xml + artifacts: + when: always + reports: + junit: + - junit.xml +``` + +## Flutter or Dart + +This example `.gitlab-ci.yml` file uses the [JUnit Report](https://pub.dev/packages/junitreport) +package to convert the `flutter test` output into JUnit report XML format: + +```yaml +test: + stage: test + script: + - flutter test --machine | tojunit -o report.xml + artifacts: + when: always + reports: + junit: + - report.xml +``` + +## PHP + +This example uses [PHPUnit](https://phpunit.de/) with the `--log-junit` flag. +You can also add this option using +[XML](https://phpunit.readthedocs.io/en/stable/configuration.html#the-junit-element) +in the `phpunit.xml` configuration file. + +```yaml +phpunit: + stage: test + script: + - composer install + - vendor/bin/phpunit --log-junit report.xml + artifacts: + when: always + reports: + junit: report.xml +``` diff --git a/doc/ci/testing/unit_test_reports.md b/doc/ci/testing/unit_test_reports.md new file mode 100644 index 00000000000..8aa41cd6fc0 --- /dev/null +++ b/doc/ci/testing/unit_test_reports.md @@ -0,0 +1,165 @@ +--- +stage: Verify +group: Pipeline Insights +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Unit test reports **(FREE)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45318) in GitLab 11.2. Requires GitLab Runner 11.2 and above. +> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39737) from JUnit test reports to Unit test reports in GitLab 13.4. + +It is very common that a [CI/CD pipeline](../pipelines/index.md) contains a +test job that verifies your code. +If the tests fail, the pipeline fails and users get notified. The person that +works on the merge request has to check the job logs and see where the +tests failed so that they can fix them. + +You can configure your job to use Unit test reports, and GitLab displays a +report on the merge request so that it's easier and faster to identify the +failure without having to check the entire log. Unit test reports currently +only support test reports in the JUnit report format. + +If you don't use merge requests but still want to see the unit test report +output without searching through job logs, the full +[Unit test reports](#view-unit-test-reports-on-gitlab) are available +in the pipeline detail view. + +Consider the following workflow: + +1. Your default branch is rock solid, your project is using GitLab CI/CD and + your pipelines indicate that there isn't anything broken. +1. Someone from your team submits a merge request, a test fails and the pipeline + gets the known red icon. To investigate more, you have to go through the job + logs to figure out the cause of the failed test, which usually contain + thousands of lines. +1. You configure the Unit test reports and immediately GitLab collects and + exposes them in the merge request. No more searching in the job logs. +1. Your development and debugging workflow becomes easier, faster and efficient. + +## How it works + +First, GitLab Runner uploads all [JUnit report format XML files](https://www.ibm.com/docs/en/adfz/developer-for-zos/14.1.0?topic=formats-junit-xml-format) +as [artifacts](../yaml/artifacts_reports.md#artifactsreportsjunit) to GitLab. Then, when you visit a merge request, GitLab starts +comparing the head and base branch's JUnit report format XML files, where: + +- The base branch is the target branch (usually the default branch). +- The head branch is the source branch (the latest pipeline in each merge request). + +The reports panel has a summary showing how many tests failed, how many had errors +and how many were fixed. If no comparison can be done because data for the base branch +is not available, the panel just shows the list of failed tests for head. + +There are four types of results: + +1. **Newly failed tests:** Test cases which passed on base branch and failed on head branch +1. **Newly encountered errors:** Test cases which passed on base branch and failed due to a + test error on head branch +1. **Existing failures:** Test cases which failed on base branch and failed on head branch +1. **Resolved failures:** Test cases which failed on base branch and passed on head branch + +Each entry in the panel shows the test name and its type from the list +above. Clicking on the test name opens a modal window with details of its +execution time and the error output. + + + +### Number of recent failures + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in merge requests in GitLab 13.7. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268249) in GitLab 13.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235525) in Test Reports in GitLab 13.9. + +If a test failed in the project's default branch in the last 14 days, a message like +`Failed {n} time(s) in {default_branch} in the last 14 days` is displayed for that test. + +## How to set it up + +To enable the Unit test reports in merge requests, you must add +[`artifacts:reports:junit`](../yaml/artifacts_reports.md#artifactsreportsjunit) +in `.gitlab-ci.yml`, and specify the paths of the generated test reports. +The reports must be `.xml` files, otherwise [GitLab returns an Error 500](https://gitlab.com/gitlab-org/gitlab/-/issues/216575). + +In the following example for Ruby, the job in the `test` stage runs and GitLab +collects the unit test report from the job. After the job is executed, the +XML report is stored in GitLab as an artifact, and the results are shown in the +merge request widget. + +```yaml +## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report format XML file with rspec +ruby: + stage: test + script: + - bundle install + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml + artifacts: + when: always + paths: + - rspec.xml + reports: + junit: rspec.xml +``` + +To make the Unit test report output files browsable, include them with the +[`artifacts:paths`](../yaml/index.md#artifactspaths) keyword as well, as shown in the example. +To upload the report even if the job fails (for example if the tests do not pass), +use the [`artifacts:when:always`](../yaml/index.md#artifactswhen) keyword. + +You cannot have multiple tests with the same name and class in your JUnit report format XML file. + +In GitLab 15.0 and earlier, test reports from [parallel:matrix](../yaml/index.md#parallel:matrix) +jobs are aggregated together, which can cause some report information to not be displayed. +In GitLab 15.1 and later, [this bug is fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/296814), +and all report information is displayed. + +## View Unit test reports on GitLab + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24792) in GitLab 12.5 behind a feature flag (`junit_pipeline_view`), disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/216478) in GitLab 13.3. + +If JUnit report format XML files are generated and uploaded as part of a pipeline, these reports +can be viewed inside the pipelines details page. The **Tests** tab on this page +displays a list of test suites and cases reported from the XML file. + + + +You can view all the known test suites and select each of these to see further +details, including the cases that make up the suite. + +You can also retrieve the reports via the [GitLab API](../../api/pipelines.md#get-a-pipelines-test-report). + +### Unit test reports parsing errors + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263457) in GitLab 13.10. + +If parsing JUnit report XML results in an error, an indicator is shown next to the job name. Hovering over the icon shows the parser error in a tooltip. If multiple parsing errors come from [grouped jobs](../jobs/index.md#group-jobs-in-a-pipeline), GitLab shows only the first error from the group. + + + +For test case parsing limits, see [Max test cases per unit test report](../../user/gitlab_com/#gitlab-cicd). + +GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. + +## View JUnit screenshots on GitLab + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0 behind the `:junit_pipeline_screenshots_view` feature flag, disabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/216979) in GitLab 13.12. + +Upload your screenshots as [artifacts](../yaml/artifacts_reports.md#artifactsreportsjunit) to GitLab. If JUnit +report format XML files contain an `attachment` tag, GitLab parses the attachment. Note that: + +- The `attachment` tag **must** contain the relative path to `$CI_PROJECT_DIR` of the screenshots you uploaded. For + example: + + ```xml + <testcase time="1.00" name="Test"> + <system-out>[[ATTACHMENT|/path/to/some/file]]</system-out> + </testcase> + ``` + +- You should set the job that uploads the screenshot to + [`artifacts:when: always`](../yaml/index.md#artifactswhen) so that it still uploads a screenshot + when a test fails. + +A link to the test case attachment appears in the test case details in +[the pipeline test report](#view-unit-test-reports-on-gitlab). diff --git a/doc/ci/troubleshooting.md b/doc/ci/troubleshooting.md index 1757543be5b..8d8afbffab9 100644 --- a/doc/ci/troubleshooting.md +++ b/doc/ci/troubleshooting.md @@ -26,8 +26,32 @@ experience (rather than the single file editor or the Web IDE). It includes: - The [CI/CD configuration visualization](pipeline_editor/index.md#visualize-ci-configuration), a graphical representation of your `.gitlab-ci.yml` file. -If you prefer to use another editor, you can use a schema like [the Schemastore `gitlab-ci` schema](https://json.schemastore.org/gitlab-ci) -with your editor of choice. +### Edit `.gitlab-ci.yml` locally + +If you prefer to edit your pipeline configuration locally, you can use the +GitLab CI/CD schema in your editor to verify basic syntax issues. Any +[editor with Schemastore support](https://www.schemastore.org/json/#editors) uses +the GitLab CI/CD schema by default. + +If you need to link to the schema directly, it +is at: + +```plaintext +https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json. +``` + +The schema rules for custom YAML tags like `!reference` will not work until the +custom tags are set in the editor settings. For example, in VS Code, you can set +`vscode-yaml` to parse `customTags`: + +```json +"yaml.customTags": [ + "!reference sequence" +] +``` + +To see the full list of custom tags covered by the CI/CD schema, check the +latest version of the schema, linked above. ### Verify syntax with CI Lint tool diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index 029bd20c553..7578a1e6009 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -1,395 +1,11 @@ --- -stage: Verify -group: Pipeline Insights -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'testing/unit_test_reports.md' +remove_date: '2022-08-31' --- -# Unit test reports **(FREE)** +This document was moved to [another location](testing/unit_test_reports.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/45318) in GitLab 11.2. Requires GitLab Runner 11.2 and above. -> - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/39737) from JUnit test reports to Unit test reports in GitLab 13.4. - -It is very common that a [CI/CD pipeline](pipelines/index.md) contains a -test job that verifies your code. -If the tests fail, the pipeline fails and users get notified. The person that -works on the merge request has to check the job logs and see where the -tests failed so that they can fix them. - -You can configure your job to use Unit test reports, and GitLab displays a -report on the merge request so that it's easier and faster to identify the -failure without having to check the entire log. Unit test reports currently -only support test reports in the JUnit report format. - -If you don't use merge requests but still want to see the unit test report -output without searching through job logs, the full -[Unit test reports](#viewing-unit-test-reports-on-gitlab) are available -in the pipeline detail view. - -Consider the following workflow: - -1. Your default branch is rock solid, your project is using GitLab CI/CD and - your pipelines indicate that there isn't anything broken. -1. Someone from your team submits a merge request, a test fails and the pipeline - gets the known red icon. To investigate more, you have to go through the job - logs to figure out the cause of the failed test, which usually contain - thousands of lines. -1. You configure the Unit test reports and immediately GitLab collects and - exposes them in the merge request. No more searching in the job logs. -1. Your development and debugging workflow becomes easier, faster and efficient. - -## How it works - -First, GitLab Runner uploads all [JUnit report format XML files](https://www.ibm.com/docs/en/adfz/developer-for-zos/14.1.0?topic=formats-junit-xml-format) -as [artifacts](yaml/artifacts_reports.md#artifactsreportsjunit) to GitLab. Then, when you visit a merge request, GitLab starts -comparing the head and base branch's JUnit report format XML files, where: - -- The base branch is the target branch (usually the default branch). -- The head branch is the source branch (the latest pipeline in each merge request). - -The reports panel has a summary showing how many tests failed, how many had errors -and how many were fixed. If no comparison can be done because data for the base branch -is not available, the panel just shows the list of failed tests for head. - -There are four types of results: - -1. **Newly failed tests:** Test cases which passed on base branch and failed on head branch -1. **Newly encountered errors:** Test cases which passed on base branch and failed due to a - test error on head branch -1. **Existing failures:** Test cases which failed on base branch and failed on head branch -1. **Resolved failures:** Test cases which failed on base branch and passed on head branch - -Each entry in the panel shows the test name and its type from the list -above. Clicking on the test name opens a modal window with details of its -execution time and the error output. - - - -### Number of recent failures - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241759) in merge requests in GitLab 13.7. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/268249) in GitLab 13.8. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/235525) in Test Reports in GitLab 13.9. - -If a test failed in the project's default branch in the last 14 days, a message like -`Failed {n} time(s) in {default_branch} in the last 14 days` is displayed for that test. - -## How to set it up - -To enable the Unit test reports in merge requests, you must add -[`artifacts:reports:junit`](yaml/artifacts_reports.md#artifactsreportsjunit) -in `.gitlab-ci.yml`, and specify the paths of the generated test reports. -The reports must be `.xml` files, otherwise [GitLab returns an Error 500](https://gitlab.com/gitlab-org/gitlab/-/issues/216575). - -In the following examples, the job in the `test` stage runs and GitLab -collects the Unit test report from each job. After each job is executed, the -XML reports are stored in GitLab as artifacts and their results are shown in the -merge request widget. - -To make the Unit test report output files browsable, include them with the -[`artifacts:paths`](yaml/index.md#artifactspaths) keyword as well, as shown in the [Ruby example](#ruby-example). -To upload the report even if the job fails (for example if the tests do not pass), use the [`artifacts:when:always`](yaml/index.md#artifactswhen) -keyword. - -You cannot have multiple tests with the same name and class in your JUnit report format XML file. - -### Ruby example - -Use the following job in `.gitlab-ci.yml`. This includes the `artifacts:paths` keyword to provide a link to the Unit test report output file. - -```yaml -## Use https://github.com/sj26/rspec_junit_formatter to generate a JUnit report format XML file with rspec -ruby: - stage: test - script: - - bundle install - - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml - artifacts: - when: always - paths: - - rspec.xml - reports: - junit: rspec.xml -``` - -### Go example - -Use the following job in `.gitlab-ci.yml`: - -```yaml -## Use https://github.com/gotestyourself/gotestsum to generate a JUnit report format XML file with go -golang: - stage: test - script: - - go get gotest.tools/gotestsum - - gotestsum --junitfile report.xml --format testname - artifacts: - when: always - reports: - junit: report.xml -``` - -### Java examples - -There are a few tools that can produce JUnit report format XML file in Java. - -#### Gradle - -In the following example, `gradle` is used to generate the test reports. -If there are multiple test tasks defined, `gradle` generates multiple -directories under `build/test-results/`. In that case, you can leverage glob -matching by defining the following path: `build/test-results/test/**/TEST-*.xml`: - -```yaml -java: - stage: test - script: - - gradle test - artifacts: - when: always - reports: - junit: build/test-results/test/**/TEST-*.xml -``` - -In [GitLab Runner 13.0](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/2620) -and later, you can use `**`. - -#### Maven - -For parsing [Surefire](https://maven.apache.org/surefire/maven-surefire-plugin/) -and [Failsafe](https://maven.apache.org/surefire/maven-failsafe-plugin/) test -reports, use the following job in `.gitlab-ci.yml`: - -```yaml -java: - stage: test - script: - - mvn verify - artifacts: - when: always - reports: - junit: - - target/surefire-reports/TEST-*.xml - - target/failsafe-reports/TEST-*.xml -``` - -### Python example - -This example uses pytest with the `--junitxml=report.xml` flag to format the output -into the JUnit report XML format: - -```yaml -pytest: - stage: test - script: - - pytest --junitxml=report.xml - artifacts: - when: always - reports: - junit: report.xml -``` - -### C/C++ example - -There are a few tools that can produce JUnit report format XML files in C/C++. - -#### GoogleTest - -In the following example, `gtest` is used to generate the test reports. -If there are multiple `gtest` executables created for different architectures (`x86`, `x64` or `arm`), -you are required to run each test providing a unique filename. The results -are then aggregated together. - -```yaml -cpp: - stage: test - script: - - gtest.exe --gtest_output="xml:report.xml" - artifacts: - when: always - reports: - junit: report.xml -``` - -#### CUnit - -[CUnit](https://cunity.gitlab.io/cunit/) can be made to produce [JUnit report format XML files](https://cunity.gitlab.io/cunit/group__CI.html) automatically when run using its `CUnitCI.h` macros: - -```yaml -cunit: - stage: test - script: - - ./my-cunit-test - artifacts: - when: always - reports: - junit: ./my-cunit-test.xml -``` - -### .NET example - -The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) NuGet -package can generate test reports for .Net Framework and .Net Core applications. The following -example expects a solution in the root folder of the repository, with one or more -project files in sub-folders. One result file is produced per test project, and each file -is placed in the artifacts folder. This example includes optional formatting arguments, which -improve the readability of test data in the test widget. A full .Net Core -[example is available](https://gitlab.com/Siphonophora/dot-net-cicd-test-logging-demo). - -```yaml -## Source code and documentation are here: https://github.com/spekt/junit.testlogger/ - -Test: - stage: test - script: - - 'dotnet test --test-adapter-path:. --logger:"junit;LogFilePath=..\artifacts\{assembly}-test-result.xml;MethodFormat=Class;FailureBodyFormat=Verbose"' - artifacts: - when: always - paths: - - ./**/*test-result.xml - reports: - junit: - - ./**/*test-result.xml -``` - -### JavaScript example - -There are a few tools that can produce JUnit report format XML files in JavaScript. - -#### Jest - -The [jest-junit](https://github.com/jest-community/jest-junit) npm package can generate test reports for JavaScript applications. -In the following `.gitlab-ci.yml` example, the `javascript` job uses Jest to generate the test reports: - -```yaml -javascript: - stage: test - script: - - 'jest --ci --reporters=default --reporters=jest-junit' - artifacts: - when: always - reports: - junit: - - junit.xml -``` - -#### Karma - -The [Karma-junit-reporter](https://github.com/karma-runner/karma-junit-reporter) npm package can generate test reports for JavaScript applications. -In the following `.gitlab-ci.yml` example, the `javascript` job uses Karma to generate the test reports: - -```yaml -javascript: - stage: test - script: - - karma start --reporters junit - artifacts: - when: always - reports: - junit: - - junit.xml -``` - -#### Mocha - -The [JUnit Reporter for Mocha](https://github.com/michaelleeallen/mocha-junit-reporter) NPM package can generate test reports for JavaScript -applications. -In the following `.gitlab-ci.yml` example, the `javascript` job uses Mocha to generate the test reports: - -```yaml -javascript: - stage: test - script: - - mocha --reporter mocha-junit-reporter --reporter-options mochaFile=junit.xml - artifacts: - when: always - reports: - junit: - - junit.xml -``` - -### Flutter / Dart example - -This example `.gitlab-ci.yml` file uses the [JUnit Report](https://pub.dev/packages/junitreport) package to convert the `flutter test` output into JUnit report XML format: - -```yaml -test: - stage: test - script: - - flutter test --machine | tojunit -o report.xml - artifacts: - when: always - reports: - junit: - - report.xml -``` - -### PHP example - -This example uses [PHPUnit](https://phpunit.de/) with the `--log-junit` flag. -You can also add this option using -[XML](https://phpunit.readthedocs.io/en/stable/configuration.html#the-junit-element) -in the `phpunit.xml` configuration file. - -```yaml -phpunit: - stage: test - script: - - composer install - - vendor/bin/phpunit --log-junit report.xml - artifacts: - when: always - reports: - junit: report.xml -``` - -## Viewing Unit test reports on GitLab - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/24792) in GitLab 12.5 behind a feature flag (`junit_pipeline_view`), disabled by default. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/216478) in GitLab 13.3. - -If JUnit report format XML files are generated and uploaded as part of a pipeline, these reports -can be viewed inside the pipelines details page. The **Tests** tab on this page -displays a list of test suites and cases reported from the XML file. - - - -You can view all the known test suites and select each of these to see further -details, including the cases that make up the suite. - -You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a-pipelines-test-report). - -### Unit test reports parsing errors - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/263457) in GitLab 13.10. - -If parsing JUnit report XML results in an error, an indicator is shown next to the job name. Hovering over the icon shows the parser error in a tooltip. If multiple parsing errors come from [grouped jobs](jobs/index.md#group-jobs-in-a-pipeline), GitLab shows only the first error from the group. - - - -For test case parsing limits, see [Max test cases per unit test report](../user/gitlab_com/#gitlab-cicd). - -GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. - -## Viewing JUnit screenshots on GitLab - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/202114) in GitLab 13.0 behind the `:junit_pipeline_screenshots_view` feature flag, disabled by default. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/216979) in GitLab 13.12. - -Upload your screenshots as [artifacts](yaml/artifacts_reports.md#artifactsreportsjunit) to GitLab. If JUnit -report format XML files contain an `attachment` tag, GitLab parses the attachment. Note that: - -- The `attachment` tag **must** contain the relative path to `$CI_PROJECT_DIR` of the screenshots you uploaded. For - example: - - ```xml - <testcase time="1.00" name="Test"> - <system-out>[[ATTACHMENT|/path/to/some/file]]</system-out> - </testcase> - ``` - -- You should set the job that uploads the screenshot to - [`artifacts:when: always`](yaml/index.md#artifactswhen) so that it still uploads a screenshot - when a test fails. - -A link to the test case attachment appears in the test case details in -[the pipeline test report](#viewing-unit-test-reports-on-gitlab). +<!-- This redirect file can be deleted after <2022-08-31>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 90403351b11..c53fad69376 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -373,7 +373,7 @@ You can configure a project, group, or instance CI/CD variable to be available only to pipelines that run on [protected branches](../../user/project/protected_branches.md) or [protected tags](../../user/project/protected_tags.md). -[Merged results pipelines](../pipelines/merge_request_pipelines.md#types-of-merge-request-pipelines), which run on a +[Merged results pipelines](../pipelines/merged_results_pipelines.md), which run on a temporary merge commit, not a branch or tag, do not have access to these variables. Pipelines that run directly on the merge request's source branch, with no added merge commit, can access @@ -526,6 +526,7 @@ export CI_PROJECT_ID="34" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-foss" export CI_PROJECT_NAME="gitlab-foss" export CI_PROJECT_TITLE="GitLab FOSS" +export CI_PROJECT_DESCRIPTION="GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed." export CI_PROJECT_NAMESPACE="gitlab-org" export CI_PROJECT_ROOT_NAMESPACE="gitlab-org" export CI_PROJECT_PATH="gitlab-org/gitlab-foss" @@ -697,6 +698,10 @@ You can override the value of a variable when you: The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence). +NOTE: +You should avoid overriding [predefined variables](predefined_variables.md), +as it can cause the pipeline to behave unexpectedly. + ### Override a variable when running a pipeline manually You can override the value of a CI/CD variable when you @@ -851,14 +856,16 @@ if [[ -d "/builds/gitlab-examples/ci-debug-trace/.git" ]]; then ++ CI_SERVER_PROTOCOL=https ++ export CI_SERVER_NAME=GitLab ++ CI_SERVER_NAME=GitLab -++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal -++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_project_service_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ export GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal +++ GITLAB_FEATURES=audit_events,burndown_charts,code_owners,contribution_analytics,description_diffs,elastic_search,group_bulk_edit,group_burndown_charts,group_webhooks,issuable_default_templates,issue_weights,jenkins_integration,ldap_group_sync,member_lock,merge_request_approvers,multiple_issue_assignees,multiple_ldap_servers,multiple_merge_request_assignees,protected_refs_for_users,push_rules,related_issues,repository_mirrors,repository_size_limit,scoped_issue_board,usage_quotas,visual_review_app,wip_limits,adjourned_deletion_for_projects_and_groups,admin_audit_log,auditor_user,batch_comments,blocking_merge_requests,board_assignee_lists,board_milestone_lists,ci_cd_projects,cluster_deployments,code_analytics,code_owner_approval_required,commit_committer_check,cross_project_pipelines,custom_file_templates,custom_file_templates_for_namespace,custom_project_templates,custom_prometheus_metrics,cycle_analytics_for_groups,db_load_balancing,default_project_deletion_protection,dependency_proxy,deploy_board,design_management,email_additional_text,extended_audit_events,external_authorization_service_api_management,feature_flags,file_locks,geo,github_integration,group_allowed_email_domains,group_project_templates,group_saml,issues_analytics,jira_dev_panel_integration,ldap_group_sync_filter,merge_pipelines,merge_request_performance_metrics,merge_trains,metrics_reports,multiple_approval_rules,multiple_group_issue_boards,object_storage,operations_dashboard,packages,productivity_analytics,project_aliases,protected_environments,reject_unsigned_commits,required_ci_templates,scoped_labels,service_desk,smartcard_auth,group_timelogs,type_of_work_analytics,unprotection_restrictions,ci_project_subscriptions,cluster_health,container_scanning,dast,dependency_scanning,epics,group_ip_restriction,incident_management,insights,license_management,personal_access_token_expiration_policy,pod_logs,prometheus_alerts,report_approver_rules,sast,security_dashboard,tracing,web_ide_terminal ++ export CI_PROJECT_ID=17893 ++ CI_PROJECT_ID=17893 ++ export CI_PROJECT_NAME=ci-debug-trace ++ CI_PROJECT_NAME=ci-debug-trace ++ export CI_PROJECT_TITLE='GitLab FOSS' ++ CI_PROJECT_TITLE='GitLab FOSS' +++ export CI_PROJECT_DESCRIPTION='GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed.' +++ CI_PROJECT_DESCRIPTION='GitLab FOSS is a read-only mirror of GitLab, with all proprietary code removed.' ++ export CI_PROJECT_PATH=gitlab-examples/ci-debug-trace ++ CI_PROJECT_PATH=gitlab-examples/ci-debug-trace ++ export CI_PROJECT_PATH_SLUG=gitlab-examples-ci-debug-trace diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index f8f775a6df4..e6d61ef342b 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -16,6 +16,10 @@ with a `script` command. There are also a number of [variables you can use to configure runner behavior](../runners/configure_runners.md#configure-runner-behavior-with-variables) globally or for individual jobs. +NOTE: +You should avoid [overriding](index.md#override-a-defined-cicd-variable) predefined variables, +as it can cause the pipeline to behave unexpectedly. + | Variable | GitLab | Runner | Description | |------------------------------------------|--------|--------|-------------| | `CHAT_CHANNEL` | 10.6 | all | The Source chat channel that triggered the [ChatOps](../chatops/index.md) command. | @@ -92,6 +96,7 @@ There are also a number of [variables you can use to configure runner behavior]( | `CI_PROJECT_REPOSITORY_LANGUAGES` | 12.3 | all | A comma-separated, lowercase list of the languages used in the repository. For example `ruby,javascript,html,css`. | | `CI_PROJECT_ROOT_NAMESPACE` | 13.2 | 0.5 | The root project namespace (username or group name) of the job. For example, if `CI_PROJECT_NAMESPACE` is `root-group/child-group/grandchild-group`, `CI_PROJECT_ROOT_NAMESPACE` is `root-group`. | | `CI_PROJECT_TITLE` | 12.4 | all | The human-readable project name as displayed in the GitLab web interface. | +| `CI_PROJECT_DESCRIPTION` | 15.1 | all | The project description as displayed in the GitLab web interface. | | `CI_PROJECT_URL` | 8.10 | 0.5 | The HTTP(S) address of the project. | | `CI_PROJECT_VISIBILITY` | 10.3 | all | The project visibility. Can be `internal`, `private`, or `public`. | | `CI_PROJECT_CLASSIFICATION_LABEL` | 14.2 | all | The project [external authorization classification label](../../user/admin_area/settings/external_authorization.md). | @@ -112,7 +117,6 @@ There are also a number of [variables you can use to configure runner behavior]( | `CI_SERVER_PORT` | 12.8 | all | The port of the GitLab instance URL, without host or protocol. For example `8080`. | | `CI_SERVER_PROTOCOL` | 12.8 | all | The protocol of the GitLab instance URL, without host or port. For example `https`. | | `CI_SERVER_REVISION` | all | all | GitLab revision that schedules jobs. | -| `CI_SERVER_TLS_CA_FILE` | all | all | File containing the CA certificate to verify the GitLab server. | | `CI_SERVER_URL` | 12.7 | all | The base URL of the GitLab instance, including protocol and port. For example `https://gitlab.example.com:8080`. | | `CI_SERVER_VERSION_MAJOR` | 11.4 | all | The major version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MAJOR` is `13`. | | `CI_SERVER_VERSION_MINOR` | 11.4 | all | The minor version of the GitLab instance. For example, if the GitLab version is `13.6.1`, the `CI_SERVER_VERSION_MINOR` is `6`. | diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 64d61ec9f11..7a1ef61f109 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -7,7 +7,7 @@ type: reference # Where variables can be used **(FREE)** -As it's described in the [CI/CD variables](index.md) docs, you can +As it's described in the [CI/CD variables](index.md) documentation, you can define many different variables. Some of them can be used for all GitLab CI/CD features, but some of them are more or less limited. @@ -17,30 +17,30 @@ This document describes where and how the different types of variables can be us There are two places defined variables can be used. On the: -1. GitLab side, in `.gitlab-ci.yml`. +1. GitLab side, in the [`.gitlab-ci.yml` file](../yaml/index.md). 1. The GitLab Runner side, in `config.toml`. ### `.gitlab-ci.yml` file -| Definition | Can be expanded? | Expansion place | Description | -|:----------------------|:-----------------|:-----------------------|:------------| -| `after_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | -| `artifacts:name` | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. | -| `before_script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | -| `cache:key` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| `environment:name` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | -| `environment:url` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | -| `except:variables:[]` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | -| `image` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| `include` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | -| `only:variables:[]` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | -| `resource_group` | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). | -| `rules:if` | no | n/a | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | -| `script` | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | -| `services:[]:name` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| `services:[]` | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | -| `tags` | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35742) in GitLab 14.1. | -| `variables` | yes | GitLab/Runner | The variable expansion is first made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab, and then any unrecognized or unavailable variables are expanded by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| Definition | Can be expanded? | Expansion place | Description | +|:----------------------------------------------------------------------|:-----------------|:-----------------------|:------------| +| [`after_script`](../yaml/index.md#after_script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | +| [`artifacts:name`](../yaml/index.md#artifactsname) | yes | Runner | The variable expansion is made by GitLab Runner's shell environment. | +| [`before_script`](../yaml/index.md#before_script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment) | +| [`cache:key`](../yaml/index.md#cachekey) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| [`environment:name`](../yaml/index.md#environmentname) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| [`environment:url`](../yaml/index.md#environmenturl) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab.<br/><br/>Supported are all variables defined for a job (project/group variables, variables from `.gitlab-ci.yml`, variables from triggers, variables from pipeline schedules).<br/><br/>Not supported are variables defined in the GitLab Runner `config.toml` and variables created in the job's `script`. | +| [`except:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| [`image`](../yaml/index.md#image) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| [`include`](../yaml/index.md#include) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. <br/><br/>Predefined project variables are supported: `GITLAB_FEATURES`, `CI_DEFAULT_BRANCH`, and all variables that start with `CI_PROJECT_` (for example `CI_PROJECT_NAME`). | +| [`only:variables`](../yaml/index.md#onlyvariables--exceptvariables) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| [`resource_group`](../yaml/index.md#resource_group) | yes | GitLab | Similar to `environment:url`, but the variables expansion doesn't support the following:<br/>- `CI_ENVIRONMENT_URL`<br/>- [Persisted variables](#persisted-variables). | +| [`rules:if`](../yaml/index.md#rulesif) | no | Not applicable | The variable must be in the form of `$variable`. Not supported are the following:<br/><br/>- Variables that are based on the environment's name (`CI_ENVIRONMENT_NAME`, `CI_ENVIRONMENT_SLUG`).<br/>- Any other variables related to environment (currently only `CI_ENVIRONMENT_URL`).<br/>- [Persisted variables](#persisted-variables). | +| [`script`](../yaml/index.md#script) | yes | Script execution shell | The variable expansion is made by the [execution shell environment](#execution-shell-environment). | +| [`services:name`](../yaml/index.md#services) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| [`services`](../yaml/index.md#services) | yes | Runner | The variable expansion is made by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | +| [`tags`](../yaml/index.md#tags) | yes | GitLab | The variable expansion is made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35742) in GitLab 14.1. | +| [`variables`](../yaml/index.md#variables) | yes | GitLab/Runner | The variable expansion is first made by the [internal variable expansion mechanism](#gitlab-internal-variable-expansion-mechanism) in GitLab, and then any unrecognized or unavailable variables are expanded by GitLab Runner's [internal variable expansion mechanism](#gitlab-runner-internal-variable-expansion-mechanism). | ### `config.toml` file diff --git a/doc/ci/yaml/artifacts_reports.md b/doc/ci/yaml/artifacts_reports.md index 289e8b91104..e4324ab06e1 100644 --- a/doc/ci/yaml/artifacts_reports.md +++ b/doc/ci/yaml/artifacts_reports.md @@ -85,10 +85,10 @@ GitLab can display the results of one or more reports in: ## `artifacts:reports:cobertura` (removed) > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.9. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7. > - [Removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.9 and +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78132) in GitLab 14.7 and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/348980) in GitLab 15.0. Use `artifacts:reports:coverage_report` instead. @@ -179,7 +179,7 @@ The collected Dependency Scanning report uploads to GitLab as an artifact. GitLab can display the results of one or more reports in: -- The merge request [dependency scanning widget](../../user/application_security/dependency_scanning/index.md#overview). +- The merge request [dependency scanning widget](../../user/application_security/dependency_scanning/index.md). - The pipeline [**Security** tab](../../user/application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline). - The [security dashboard](../../user/application_security/security_dashboard/index.md). - The [Project Vulnerability report](../../user/application_security/vulnerability_report/index.md). @@ -209,6 +209,7 @@ The exceptions to the [original dotenv rules](https://github.com/motdotla/dotenv self-managed instances is 150, and can be changed by changing the `dotenv_variables` [application limit](../../administration/instance_limits.md#limit-dotenv-variables). - Variable substitution in the `.env` file is not supported. +- [Multiline values in the `.env` file](https://github.com/motdotla/dotenv#multiline-values) are not supported. - The `.env` file can't have empty lines or comments (starting with `#`). - Key values in the `env` file cannot have spaces or newline characters (`\n`), including when using single or double quotes. - Quote escaping during parsing (`key = 'value'` -> `{key: "value"}`) is not supported. @@ -219,7 +220,7 @@ The `junit` report collects [JUnit report format XML files](https://www.ibm.com/ The collected Unit test reports upload to GitLab as an artifact. Although JUnit was originally developed in Java, there are many third-party ports for other languages such as JavaScript, Python, and Ruby. -See [Unit test reports](../unit_test_reports.md) for more details and examples. +See [Unit test reports](../testing/unit_test_reports.md) for more details and examples. Below is an example of collecting a JUnit report format XML file from Ruby's RSpec test tool: ```yaml @@ -235,8 +236,8 @@ rspec: GitLab can display the results of one or more reports in: -- The merge request [code quality widget](../../ci/unit_test_reports.md#how-it-works). -- The [full report](../../ci/unit_test_reports.md#viewing-unit-test-reports-on-gitlab). +- The merge request [code quality widget](../testing/unit_test_reports.md#how-it-works). +- The [full report](../testing/unit_test_reports.md#view-unit-test-reports-on-gitlab). Some JUnit tools export to multiple XML files. You can specify multiple test report paths in a single job to concatenate them into a single file. Use either: diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index eb587db6d5f..912eca364c9 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -175,6 +175,8 @@ Use `include:local` instead of symbolic links. - The YAML file must have the extension `.yml` or `.yaml`. - You can [use `*` and `**` wildcards in the file path](includes.md#use-includelocal-with-wildcard-file-paths). +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `include:local`**: ```yaml @@ -209,6 +211,8 @@ use `include:file`. You can use `include:file` in combination with `include:proj - A full path, relative to the root directory (`/`). The YAML file must have the extension `.yml` or `.yaml`. +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `include:file`**: ```yaml @@ -226,7 +230,7 @@ include: file: '/templates/.gitlab-ci-template.yml' - project: 'my-group/my-project' - ref: v1.0.0 + ref: v1.0.0 # Git Tag file: '/templates/.gitlab-ci-template.yml' - project: 'my-group/my-project' @@ -267,6 +271,8 @@ Use `include:remote` with a full URL to include a file from a different location - A public URL accessible by an HTTP/HTTPS `GET` request. Authentication with the remote URL is not supported. The YAML file must have the extension `.yml` or `.yaml`. +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `include:remote`**: ```yaml @@ -292,6 +298,8 @@ Use `include:template` to include [`.gitlab-ci.yml` templates](https://gitlab.co - [`.gitlab-ci.yml` templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `include:template`**: ```yaml @@ -331,6 +339,11 @@ The order of the items in `stages` defines the execution order for jobs: - Jobs in the same stage run in parallel. - Jobs in the next stage run after the jobs from the previous stage complete successfully. +If a pipeline contains only jobs in the `.pre` or `.post` stages, it does not run. +There must be at least one other job in a different stage. `.pre` and `.post` stages +can be used in [required pipeline configuration](../../user/admin_area/settings/continuous_integration.md#required-pipeline-configuration) +to define compliance jobs that must run before or after project pipeline jobs. + **Keyword type**: Global keyword. **Example of `stages`**: @@ -504,6 +517,8 @@ Use `after_script` to define an array of commands that run after each job, inclu - Long commands [split over multiple lines](script.md#split-long-commands). - [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `after_script`**: ```yaml @@ -809,8 +824,8 @@ If not defined, the default name is `artifacts`, which becomes `artifacts.zip` w **Possible inputs**: -- The name of the artifacts archive. Can use [CI/CD variables](../variables/index.md). Must be combined with - [`artifacts:paths`](#artifactspaths). +- The name of the artifacts archive. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + Must be combined with [`artifacts:paths`](#artifactspaths). **Example of `artifacts:name`**: @@ -977,7 +992,7 @@ failure. - `on_success` (default): Upload artifacts only when the job succeeds. - `on_failure`: Upload artifacts only when the job fails. - `always`: Always upload artifacts (except when jobs time out). For example, when - [uploading artifacts](../unit_test_reports.md#viewing-junit-screenshots-on-gitlab) + [uploading artifacts](../testing/unit_test_reports.md#view-junit-screenshots-on-gitlab) required to troubleshoot failing tests. **Example of `artifacts:when`**: @@ -1002,6 +1017,8 @@ Use `before_script` to define an array of commands that should run before each j - Long commands [split over multiple lines](script.md#split-long-commands). - [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `before_script`**: ```yaml @@ -1016,6 +1033,7 @@ job: - Scripts you specify in `before_script` are concatenated with any scripts you specify in the main [`script`](#script). The combined scripts execute together in a single shell. +- Using `before_script` at the top level, but not in the `default` section, [is deprecated](#globally-defined-image-services-cache-before_script-after_script). **Related topics**: @@ -1087,7 +1105,7 @@ no `cache:key` share the `default` cache. **Possible inputs**: - A string. -- A [predefined variables](../variables/index.md). +- A predefined [CI/CD variable](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). - A combination of both. **Example of `cache:key`**: @@ -1504,7 +1522,8 @@ Common environment names are `qa`, `staging`, and `production`, but you can use formats: - Plain text, including letters, digits, spaces, and these characters: `-`, `_`, `/`, `$`, `{`, `}`. -- CI/CD variables, including predefined, project, group, instance, or variables defined in the +- [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file), + including predefined, project, group, instance, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. **Example of `environment:name`**: @@ -1526,7 +1545,8 @@ Set a URL for an [environment](../environments/index.md). **Possible inputs**: A single URL, in one of these formats: - Plain text, like `https://prod.example.com`. -- CI/CD variables, including predefined, project, group, instance, or variables defined in the +- [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file), + including predefined, project, group, instance, or variables defined in the `.gitlab-ci.yml` file. You can't use variables defined in a `script` section. **Example of `environment:url`**: @@ -1788,6 +1808,8 @@ Use `image` to specify a Docker image that the job runs in. - `<image-name>:<tag>` - `<image-name>@<digest>` +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `image`**: ```yaml @@ -1861,6 +1883,52 @@ image: - [Override the entrypoint of an image](../docker/using_docker_images.md#override-the-entrypoint-of-an-image). +#### `image:pull_policy` + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21619) in GitLab 15.1 [with a flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. Disabled by default. +> - Requires GitLab Runner 15.1 or later. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_docker_image_pull_policy`. +The feature is not ready for production use. + +The pull policy that the runner uses to fetch the Docker image. + +**Keyword type**: Job keyword. You can use it only as part of a job or in the [`default` section](#default). + +**Possible inputs**: + +- A single pull policy, or multiple pull policies in an array. + Can be `always`, `if-not-present`, or `never`. + +**Examples of `image:pull_policy`**: + +```yaml +job1: + script: echo "A single pull policy." + image: + name: ruby:3.0 + pull_policy: if-not-present + +job2: + script: echo "Multiple pull policies." + image: + name: ruby:3.0 + pull_policy: [always, if-not-present] +``` + +**Additional details**: + +- If the runner does not support the defined pull policy, the job fails with an error similar to: + `ERROR: Job failed (system failure): the configured PullPolicies ([always]) are not allowed by AllowedPullPolicies ([never])`. + +**Related topics**: + +- [Run your CI/CD jobs in Docker containers](../docker/using_docker_images.md). +- [How runner pull policies work](https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work). +- [Using multiple pull policies](https://docs.gitlab.com/runner/executors/docker.html#using-multiple-pull-policies). + ### `inherit` > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. @@ -1945,9 +2013,9 @@ job2: Use `interruptible` if a job should be canceled when a newer pipeline starts before the job completes. -This keyword is used with the [automatic cancellation of redundant pipelines](../pipelines/settings.md#auto-cancel-redundant-pipelines) -feature. When enabled, a running job with `interruptible: true` can be cancelled when -a new pipeline starts on the same branch. +This keyword has no effect if [automatic cancellation of redundant pipelines](../pipelines/settings.md#auto-cancel-redundant-pipelines) +is disabled. When enabled, a running job with `interruptible: true` is cancelled when +starting a pipeline for a new change on the same branch. You can't cancel subsequent jobs after a job with `interruptible: false` starts. @@ -2169,8 +2237,8 @@ build_job: In this example, `build_job` downloads the artifacts from the latest successful `build-1` job on the `main` branch in the `group/project-name` project. -In GitLab 13.3 and later, you can use [CI/CD variables](../variables/index.md) in `needs:project`, -for example: +In GitLab 13.3 and later, you can use [CI/CD variables](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) +in `needs:project`, for example: ```yaml build_job: @@ -2268,43 +2336,58 @@ can use that variable in `needs:pipeline` to download artifacts from the parent To need a job that sometimes does not exist in the pipeline, add `optional: true` to the `needs` configuration. If not defined, `optional: false` is the default. -Jobs that use [`rules`](#rules), [`only`, or `except`](#only--except), might -not always exist in a pipeline. When the pipeline is created, GitLab checks the `needs` -relationships before starting it. Without `optional: true`, needs relationships that -point to a job that does not exist stops the pipeline from starting and causes a pipeline -error similar to: +Jobs that use [`rules`](#rules), [`only`, or `except`](#only--except) might not always +be added to a pipeline. GitLab checks the `needs` relationships before starting a +pipeline: -- `'job1' job needs 'job2' job, but it was not added to the pipeline` +- If the needs entry has `optional: true` and the needed job is present in the pipeline, + the job waits for it to complete before starting. +- If the needed job is not present, the job can start when all other needs requirements are met. +- If the `needs` section contains only optional jobs, and none are added to the pipeline, + the job starts immediately (the same as an empty `needs` entry: `needs: []`). +- If a needed job has `optional: false`, but it was not added to the pipeline, the + pipeline fails to start with an error similar to: `'job1' job needs 'job2' job, but it was not added to the pipeline`. **Keyword type**: Job keyword. You can use it only as part of a job. -**Possible inputs**: - -- `job`: The job to make optional. -- `true` or `false` (default). - **Example of `needs:optional`**: ```yaml -build: +build-job: stage: build + +test-job1: + stage: test + +test-job2: + stage: test rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH -rspec: - stage: test +deploy-job: + stage: deploy needs: - - job: build + - job: test-job2 + optional: true + - job: test-job1 + +review-job: + stage: deploy + needs: + - job: test-job2 optional: true ``` In this example: -- When the branch is the default branch, the `build` job exists in the pipeline, and the `rspec` - job waits for it to complete before starting. -- When the branch is not the default branch, the `build` job does not exist in the pipeline. - The `rspec` job runs immediately (similar to `needs: []`) because its `needs` - relationship to the `build` job is optional. +- `build-job`, `test-job1`, and `test-job2` start in stage order. +- When the branch is the default branch, `test-job2` is added to the pipeline, so: + - `deploy-job` waits for both `test-job1` and `test-job2` to complete. + - `review-job` waits for `test-job2` to complete. +- When the branch is not the default branch, `test-job2` is not added to the pipeline, so: + - `deploy-job` waits for only `test-job1` to complete, and does not wait for the missing `test-job2`. + - `review-job` has no other needed jobs and starts immediately (at the same time as `build-job`), + like `needs: []`. #### `needs:pipeline` @@ -2735,7 +2818,7 @@ This example creates a release: **Related topics**: -- [CI/CD example of the `release` keyword](../../user/project/releases/index.md#cicd-example-of-the-release-keyword). +- [CI/CD example of the `release` keyword](../../user/project/releases/index.md#creating-a-release-by-using-a-cicd-job). - [Create multiple releases in a single pipeline](../../user/project/releases/index.md#create-multiple-releases-in-a-single-pipeline). - [Use a custom SSL CA certificate authority](../../user/project/releases/index.md#use-a-custom-ssl-ca-certificate-authority). @@ -2750,7 +2833,9 @@ New tags use the SHA associated with the pipeline. **Possible inputs**: -- A tag name. Can use [CI/CD variables](../variables/index.md). +- A tag name. + +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). **Example of `release:tag_name`**: @@ -2899,7 +2984,7 @@ can be deployed to, but only one deployment can occur per device at any given ti **Possible inputs**: - Only letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces. - It can't start or end with `/`. + It can't start or end with `/`. CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). **Example of `resource_group`**: @@ -3177,8 +3262,9 @@ job: with the flags `File::FNM_PATHNAME | File::FNM_DOTMATCH | File::FNM_EXTGLOB`. - For performance reasons, GitLab matches a maximum of 10,000 `exists` patterns or file paths. After the 10,000th check, rules with patterned globs always match. - In other words, the `exists` rule always assumes a match in projects with more - than 10,000 files. + In other words, `exists` always reports `true` if more than 10,000 checks + run. Repositories with less than 10,000 files might still be impacted if the `exists` + rules are checked more than 10,000 times. - `exists` resolves to `true` if any of the listed files are found (an `OR` operation). #### `rules:allow_failure` @@ -3262,6 +3348,8 @@ All jobs except [trigger jobs](#trigger) require a `script` keyword. - Long commands [split over multiple lines](script.md#split-long-commands). - [YAML anchors](yaml_optimization.md#yaml-anchors-for-scripts). +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Example of `script`**: ```yaml @@ -3396,6 +3484,8 @@ to the image specified in the [`image`](#image) keyword. - `<image-name>:<tag>` - `<image-name>@<digest>` +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file), but [not for `alias`](https://gitlab.com/gitlab-org/gitlab/-/issues/19561). + **Example of `services`**: ```yaml @@ -3486,7 +3576,8 @@ Use the `.pre` stage to make a job run at the start of a pipeline. `.pre` is always the first stage in a pipeline. User-defined stages execute after `.pre`. You do not have to define `.pre` in [`stages`](#stages). -You must have a job in at least one stage other than `.pre` or `.post`. +If a pipeline contains only jobs in the `.pre` or `.post` stages, it does not run. +There must be at least one other job in a different stage. **Keyword type**: You can only use it with a job's `stage` keyword. @@ -3521,7 +3612,8 @@ Use the `.post` stage to make a job run at the end of a pipeline. `.post` is always the last stage in a pipeline. User-defined stages execute before `.post`. You do not have to define `.post` in [`stages`](#stages). -You must have a job in at least one stage other than `.pre` or `.post`. +If a pipeline contains only jobs in the `.pre` or `.post` stages, it does not run. +There must be at least one other job in a different stage. **Keyword type**: You can only use it with a job's `stage` keyword. @@ -3566,7 +3658,8 @@ be assigned every tag listed in the job. **Possible inputs**: - An array of tag names. -- [CI/CD variables](../runners/configure_runners.md#use-cicd-variables-in-tags) in GitLab 14.1 and later. +- CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file) + in GitLab 14.1 and later. **Example of `tags`**: @@ -3705,12 +3798,9 @@ successfully complete before starting. #### `trigger:forward` -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213729) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, -ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. -The feature is not ready for production use. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213729) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) in GitLab 14.10. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) in GitLab 15.1. [Feature flag `ci_trigger_forward_variables`](https://gitlab.com/gitlab-org/gitlab/-/issues/355572) removed. Use `trigger:forward` to specify what to forward to the downstream pipeline. You can control what is forwarded to both [parent-child pipelines](../pipelines/parent_child_pipelines.md) @@ -3779,6 +3869,8 @@ variable defined, the [job-level variable takes precedence](../variables/index.m the first character must be a letter. - The value must be a string. +CI/CD variables [are supported](../variables/where_variables_can_be_used.md#gitlab-ciyml-file). + **Examples of `variables`**: ```yaml @@ -3844,18 +3936,18 @@ variables: Use `when` to configure the conditions for when jobs run. If not defined in a job, the default value is `when: on_success`. -**Keyword type**: Job keyword. You can use it only as part of a job. +**Keyword type**: Job keyword. You can use it as part of a job. `when: always` and `when: never` can also be used in [`workflow:rules`](#workflow). **Possible inputs**: - `on_success` (default): Run the job only when all jobs in earlier stages succeed or have `allow_failure: true`. - `manual`: Run the job only when [triggered manually](../jobs/job_control.md#create-a-job-that-must-be-run-manually). -- `always`: Run the job regardless of the status of jobs in earlier stages. +- `always`: Run the job regardless of the status of jobs in earlier stages. Can also be used in `workflow:rules`. - `on_failure`: Run the job only when at least one job in an earlier stage fails. - `delayed`: [Delay the execution of a job](../jobs/job_control.md#run-a-job-after-a-delay) for a specified duration. -- `never`: Don't run the job. +- `never`: Don't run the job. Can only be used in a [`rules`](#rules) section or `workflow: rules`. **Example of `when`**: @@ -3921,18 +4013,20 @@ In this example, the script: The following keywords are deprecated. -### Globally-defined `types` +<!--- start_remove The following content will be removed on remove_date: '2022-08-22' --> -WARNING: -`types` is deprecated, and is [scheduled to be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). +### Globally-defined `types` (removed) + +The `types` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). Use [`stages`](#stages) instead. -### Job-defined `type` +### Job-defined `type` (removed) -WARNING: -`type` is deprecated, and is [scheduled to be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). +The `type` keyword was deprecated in GitLab 9.0, and [removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/346823). Use [`stage`](#stage) instead. +<!--- end_remove --> + ### Globally-defined `image`, `services`, `cache`, `before_script`, `after_script` Defining `image`, `services`, `cache`, `before_script`, and diff --git a/doc/ci/yaml/yaml_optimization.md b/doc/ci/yaml/yaml_optimization.md index 503ba9bbd80..e5d9e011230 100644 --- a/doc/ci/yaml/yaml_optimization.md +++ b/doc/ci/yaml/yaml_optimization.md @@ -450,5 +450,26 @@ test-vars-2: - printenv ``` -You can't reuse a section that already includes a `!reference` tag. Only one level -of nesting is supported. +### Nest `!reference` tags in `script`, `before_script`, and `after_script` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74792) in GitLab 14.8. + +You can nest `!reference` tags up to 10 levels deep in `script`, `before_script`, and `after_script` sections. Use nested tags to define reusable sections when building more complex scripts. For example: + +```yaml +.snippets: + one: + - echo "ONE!" + two: + - !reference [.snippets, one] + - echo "TWO!" + three: + - !reference [.snippets, two] + - echo "THREE!" + +nested-references: + script: + - !reference [.snippets, three] +``` + +In this example, the `nested-references` job runs all three `echo` commands. |
