diff options
Diffstat (limited to 'doc/ci')
49 files changed, 399 insertions, 312 deletions
diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md index 9cf9d2a1f06..5571c51ef27 100644 --- a/doc/ci/cloud_deployment/ecs/quick_start_guide.md +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -40,10 +40,10 @@ For the first step here, you create a demo application from a project template. Use a GitLab project template to get started. As the name suggests, these projects provide a bare-bones application built on some well-known frameworks. -1. In GitLab, click the plus icon (**{plus-square}**) at the top of the navigation bar, and select +1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select **New project**. -1. Click the **Create from template** button, where you can choose from a Ruby on Rails, Spring, or +1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or NodeJS Express project. For this guide, use the Ruby on Rails template.  @@ -52,7 +52,7 @@ bare-bones application built on some well-known frameworks. take advantage of the features available in the [GitLab Ultimate plan](https://about.gitlab.com/pricing/). -1. Click **Create project**. +1. Select **Create project**. Now that you created a demo project, you must containerize the application and push it to the container registry. @@ -65,7 +65,7 @@ GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) and [Container Registry](../../../user/packages/container_registry/index.md). 1. Go to **ecs-demo** project on GitLab. -1. Click **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` +1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` creation form. 1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines [a pipeline for continuous deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs). @@ -75,7 +75,7 @@ and [Container Registry](../../../user/packages/container_registry/index.md). - template: AWS/Deploy-ECS.gitlab-ci.yml ``` -1. Click **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` +1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md).  @@ -97,14 +97,14 @@ later. is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). 1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). -1. Click **Create new Task Definition**. +1. Select **Create new Task Definition**.  -1. Choose **EC2** as the launch type. Click **Next Step**. +1. Choose **EC2** as the launch type. Select **Next Step**. 1. Set `ecs_demo` to **Task Definition Name**. 1. Set `512` to **Task Size > Task memory** and **Task CPU**. -1. Click **Container Definitions > Add container**. This opens a container registration form. +1. Select **Container Definitions > Add container**. This opens a container registration form. 1. Set `web` to **Container name**. 1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). @@ -115,7 +115,7 @@ is a specification about how the application image is started by an [ECS service  -1. Click **Create**. +1. Select **Create**. Now you have the initial task definition. Next, you create an actual infrastructure to run the application image. @@ -127,13 +127,13 @@ is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/lates It's also associated with EC2 or Fargate as the computation resource. 1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). -1. Click **Create Cluster**. -1. Select **EC2 Linux + Networking** as the cluster template. Click **Next Step**. +1. Select **Create Cluster**. +1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. 1. Set `ecs-demo` to **Cluster Name**. 1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. 1. Set all available subnets of the VPC to **Subnets**. -1. Click **Create**. +1. Select **Create**. 1. Make sure that the ECS cluster has been successfully created.  @@ -154,7 +154,7 @@ Note the following: is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). 1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) -1. Click **Deploy**. This opens a service creation form. +1. Select **Deploy**. This opens a service creation form. 1. Select `EC2` in **Launch Type**. 1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). 1. Set `ecs_demo` to **Service name**. @@ -162,7 +162,7 @@ is a daemon to create an application container based on the [ECS task definition  -1. Click **Deploy**. +1. Select **Deploy**. 1. Make sure that the created service is active.  @@ -176,7 +176,7 @@ Now, the demo application is accessible from the internet. 1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) 1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). -1. Click the ID of the EC2 instance. This brings you to the instance detail page. +1. Select the ID of the EC2 instance. This brings you to the instance detail page. 1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application running. @@ -195,15 +195,15 @@ For GitLab to access the ECS cluster, service, and task definition that you crea create a deployer user on AWS: 1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). -1. Click **Add user**. +1. Select **Add user**. 1. Set `ecs_demo` to **User name**. -1. Enable **Programmatic access** checkbox. Click **Next: Permissions**. +1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. 1. Select `Attach existing policies directly` in **Set permissions**. -1. Select `AmazonECS_FullAccess` from the policy list. Click **Next: Tags** and **Next: Review**. +1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**.  -1. Click **Create user**. +1. Select **Create user**. 1. Take note of the **Access key ID** and **Secret access key** of the created user. NOTE: @@ -216,7 +216,7 @@ These variables are injected into the pipeline jobs and can access the ECS API. 1. Go to **ecs-demo** project on GitLab. 1. Go to **Settings > CI/CD > Variables**. -1. Click **Add Variable** and set the following key-value pairs. +1. Select **Add Variable** and set the following key-value pairs. |Key|Value|Note| |---|---|---| @@ -233,9 +233,9 @@ Change a file in the project and see if it's reflected in the demo application o 1. Go to **ecs-demo** project on GitLab. 1. Open the file at **app > views > welcome > `index.html.erb`**. -1. Click **Edit**. +1. Select **Edit**. 1. Change the text to `You're on ECS!`. -1. Click **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. +1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. 1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see this: diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index a80231a04c2..1493a930099 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -18,6 +18,13 @@ GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) t The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. +NOTE: +Configuring OIDC enables JWT token access to the target environments for all pipelines. +When you configure OIDC for a pipeline, you should complete a software supply chain security +review for the pipeline, focusing on the additional access. You can use the [software supply chain security awareness assessment](https://about.gitlab.com/quiz/software-supply-chain-security/) +as a starting point, and for more information about supply chain attacks, see +[How a DevOps Platform helps protect against supply chain attacks](https://about.gitlab.com/blog/2021/04/28/devops-platform-supply-chain-attacks/). + WARNING: The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index abf43390834..5d0aaf7cc45 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -81,7 +81,7 @@ are certain use cases that you may need to work around. For more information, ch ## Needs Visualization -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](https://about.gitlab.com/handbook/product/#beta). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). > - It became a [standard feature](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38517) in 13.3. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52208) in GitLab 13.9. diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index e320118c191..45af78aa036 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -11,7 +11,7 @@ description: Require approvals prior to deploying to a Protected Environment > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/347342) in GitLab 14.8. WARNING: -This feature is in an alpha stage and subject to change without prior notice. +This feature is in a [Beta](../../policy/alpha-beta-support.md#beta-features) stage and subject to change without prior notice. It may be useful to require additional approvals before deploying to certain protected environments (for example, production). This pre-deployment approval requirement is useful to accommodate testing, security, or compliance processes that must happen before each deployment. @@ -79,30 +79,53 @@ Maintainer role. ## Approve or reject a deployment -NOTE: -This functionality is currently only available through the API. UI is planned for the near future. See [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/) in GitLab 14.9 + +Using either the GitLab UI or the API, you can: + +- Approve a deployment to allow it to proceed. +- Reject a deployment to prevent it. + +### Approve or reject a deployment using the UI + +Prerequisites: + +- Permission to deploy to the protected environment. + +To approve or reject a deployment to a protected environment using the UI: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. +1. In the deployment's row, select **Approval options** (**{thumb-up}**). +1. Select **Approve** or **Reject**. + +### Approve or reject a deployment using the API + +Prerequisites: -A blocked deployment is enqueued as soon as it receives the required number of approvals. A single rejection causes the deployment to fail. The creator of a deployment cannot approve it, even if they have permission to deploy. +- Permission to deploy to the protected environment. -Using the [Deployments API](../../api/deployments.md#approve-or-reject-a-blocked-deployment), users who are allowed to deploy to the protected environment can approve or reject a blocked deployment. +To approve or reject a deployment to a protected environment using the API, pass the +required attributes. For more details, see +[Approve or reject a blocked deployment](../../api/deployments.md#approve-or-reject-a-blocked-deployment). Example: ```shell -curl --data "status=approved" \ +curl --data "status=approved&comment=Looks good to me" \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments/1/approval" ``` -### How to see blocked deployments +## How to see blocked deployments -#### Using the UI +### Using the UI 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment being deployed to. 1. Look for the `blocked` label. -#### Using the API +### Using the API Use the [Deployments API](../../api/deployments.md) to see deployments. diff --git a/doc/ci/environments/img/environments_list_v14_3.png b/doc/ci/environments/img/environments_list_v14_3.png Binary files differdeleted file mode 100644 index 8fdb85338e7..00000000000 --- a/doc/ci/environments/img/environments_list_v14_3.png +++ /dev/null diff --git a/doc/ci/environments/img/environments_list_v14_8.png b/doc/ci/environments/img/environments_list_v14_8.png Binary files differnew file mode 100644 index 00000000000..df439fb96dc --- /dev/null +++ b/doc/ci/environments/img/environments_list_v14_8.png diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 63bdd279927..c2612b6ff16 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -35,7 +35,7 @@ To view a list of environments and deployments: 1. On the left sidebar, select **Deployments > Environments**. The environments are displayed. -  +  1. To view a list of deployments for an environment, select the environment name, for example, `staging`. @@ -440,7 +440,7 @@ stop_review: when: manual ``` -Both jobs must have the same [`rules`](../yaml/index.md#only--except) +Both jobs must have the same [`rules`](../yaml/index.md#rules) or [`only/except`](../yaml/index.md#only--except) configuration. Otherwise, the `stop_review` job might not be included in all pipelines that include the `deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index c7d1653aace..adc215c7aa1 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -163,9 +163,8 @@ For more information, see [Deployment safety](deployment_safety.md). Typically, large enterprise organizations have an explicit permission boundary between [developers and operators](https://about.gitlab.com/topics/devops/). Developers build and test their code, and operators deploy and monitor the -application. With group-level protected environments, the permission of each -group is carefully configured in order to prevent unauthorized access and -maintain proper separation of duty. Group-level protected environments +application. With group-level protected environments, operators can +restrict access to critical environments from developers. Group-level protected environments extend the [project-level protected environments](#protecting-environments) to the group-level. @@ -194,12 +193,6 @@ and are protected at the same time. ### Configure group-level memberships -In an enterprise organization, with thousands of projects under a single group, -ensuring that all of the [project-level protected environments](#protecting-environments) -are properly configured is not a scalable solution. For example, a developer -might gain privileged access to a higher environment when they are given the Maintainer role -for a new project. Group-level protected environments can be a solution in this situation. - To maximize the effectiveness of group-level protected environments, [group-level memberships](../../user/group/index.md) must be correctly configured: @@ -237,7 +230,7 @@ Having this configuration in place: - If a user is about to run a deployment job in a project but disallowed to deploy to the environment, the deployment job fails with an error message. -### Protect a group-level environment +### Protect critical environments under a group To protect a group-level environment: diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index edc58684057..5fca3513ff7 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -121,8 +121,8 @@ Then create policies that allow you to read these secrets (one for each secret): $ vault policy write myproject-staging - <<EOF # Policy name: myproject-staging # -# Read-only permission on 'secret/data/myproject/staging/*' path -path "secret/data/myproject/staging/*" { +# Read-only permission on 'secret/myproject/staging/*' path +path "secret/myproject/staging/*" { capabilities = [ "read" ] } EOF @@ -131,8 +131,8 @@ Success! Uploaded policy: myproject-staging $ vault policy write myproject-production - <<EOF # Policy name: myproject-production # -# Read-only permission on 'secret/data/myproject/production/*' path -path "secret/data/myproject/production/*" { +# Read-only permission on 'secret/myproject/production/*' path +path "secret/myproject/production/*" { capabilities = [ "read" ] } EOF diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index a62f670cb12..b0002f559ba 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -23,6 +23,12 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Releases](../../api/releases/index.md). - [Terraform plan](../../user/infrastructure/index.md). +NOTE: +There's an open issue, +[GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), +which prevents you from using a job token with internal projects. This bug only impacts self-managed +GitLab instances. + The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 39e14d0d20a..4fa251eb3cf 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -103,6 +103,9 @@ Job names must be 255 characters or less. [Introduced](https://gitlab.com/gitlab in GitLab 14.5, [with a feature flag](../../administration/feature_flags.md) named `ci_validate_job_length`. Enabled by default. To disable it, ask an administrator to [disable the feature flag](../../administration/feature_flags.md). +Use unique names for your jobs. If multiple jobs have the same name, +only one is added to the pipeline, and it's difficult to predict which one is chosen. + ## Group jobs in a pipeline If you have many similar jobs, your [pipeline graph](../pipelines/index.md#visualize-pipelines) becomes long and hard diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 6523de0ed1e..3a302cf352b 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -85,6 +85,28 @@ be triggered by the same event (a push to the source branch for an open merge re See how to [prevent duplicate pipelines](#avoid-duplicate-pipelines) for more details. +#### Run jobs for scheduled pipelines + +To configure a job to be executed only when the pipeline has been +scheduled, use the [`rules`](../yaml/index.md#rules) keyword. + +In this example, `make world` runs in scheduled pipelines, and `make build` +runs in branch and tag pipelines: + +```yaml +job:on-schedule: + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + script: + - make world + +job: + rules: + - if: $CI_PIPELINE_SOURCE == "push" + script: + - make build +``` + ### Complex rules You can use all `rules` keywords, like `if`, `changes`, and `exists`, in the same @@ -115,7 +137,7 @@ job1: script: - echo This rule uses parentheses. rules: - if: ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE + - if: ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE ``` WARNING: @@ -727,6 +749,38 @@ deploystacks: - ${PROVIDER}-${STACK} ``` +#### Fetch artifacts from a `parallel:matrix` job + +You can fetch artifacts from a job created with [`parallel:matrix`](../yaml/index.md#parallelmatrix) +by using the [`dependencies`](../yaml/index.md#dependencies) keyword. Use the job name +as the value for `dependencies` as a string in the form: + +```plaintext +<job_name> [<matrix argument 1>, <matrix argument 2>, ... <matrix argument N>] +``` + +For example, to fetch the artifacts from the job with a `RUBY_VERSION` of `2.7` and +a `PROVIDER` of `aws`: + +```yaml +ruby: + image: ruby:${RUBY_VERSION} + parallel: + matrix: + - RUBY_VERSION: ["2.5", "2.6", "2.7", "3.0", "3.1"] + PROVIDER: [aws, gcp] + script: bundle install + +deploy: + image: ruby:2.7 + stage: deploy + dependencies: + - "ruby: [2.7, aws]" + script: echo hello +``` + +Quotes around the `dependencies` entry are required. + ## Use predefined CI/CD variables to run jobs only in specific pipeline types You can use [predefined CI/CD variables](../variables/predefined_variables.md) to choose @@ -812,13 +866,9 @@ due to computational complexity, and some features, like negative lookaheads, be Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Regexp.html) are now supported. -From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to -let you use unsafe regexp syntax. After migrating to safe syntax, you should disable -this feature flag again: - -```ruby -Feature.enable(:allow_unsafe_ruby_regexp) -``` +From GitLab 11.9.7 to GitLab 14.9, GitLab provided a feature flag to let you +use unsafe regexp syntax. We've fully migrated to RE2 now, and that feature +flag is no longer available. ## CI/CD variable expressions diff --git a/doc/ci/lint.md b/doc/ci/lint.md index c0df0b2a439..256e669a66e 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -25,9 +25,8 @@ configuration added with the [`includes` keyword](yaml/index.md#include). To check CI/CD configuration with the CI lint tool: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select one of: +1. On the left sidebar, select: - **CI/CD > Pipelines** - - **CI/CD > Jobs** 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Validate**. @@ -48,9 +47,8 @@ Prerequisites: To simulate a pipeline: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select one of: +1. On the left sidebar, select: - **CI/CD > Pipelines** - - **CI/CD > Jobs** 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Simulate pipeline creation for the default branch**. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 5b472eec7b5..542d55665c7 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -55,3 +55,14 @@ An advanced example of an OpenMetrics text file (from the [Prometheus documentat renders in the merge request widget as:  + +## Troubleshooting + +### Metrics reports did not change + +You can see `Metrics reports did not change` when trying to view metrics reports in merge requests. Reasons for this are: + +- The target branch for the merge request doesn't have a baseline metrics report for comparison. +- You don't have GitLab license at the Premium tier or above. + +There is [an issue open](https://gitlab.com/gitlab-org/gitlab/-/issues/343065) to improve this message. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index efaae873588..d95199a70d0 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -169,7 +169,7 @@ job1: - if: '$CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "try-schedule-workflow"' ``` -After the pipeline configuration is saved, you configure the cron schedule in the [GitLab UI](../pipelines/schedules.md#configuring-pipeline-schedules), and can enable or disable schedules in the UI as well. +After the pipeline configuration is saved, you configure the cron schedule in the [GitLab UI](../pipelines/schedules.md#add-a-pipeline-schedule), and can enable or disable schedules in the UI as well. #### Manual run diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 8b10a74bd78..f9d9a4f738b 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -223,3 +223,25 @@ On GitLab SaaS an email notification is sent to the namespace owners when: - The available CI/CD minutes are below 30% of the quota. - The available CI/CD minutes are below 5% of the quota. - All CI/CD minutes have been used. + +## Reduce consumption of CI/CD minutes + +If your project consumes too many CI/CD minutes, there are some strategies you can +use to reduce your CI/CD minutes usage: + +- If you are using project mirrors, ensure that [pipelines for mirror updates](../../user/project/repository/mirror/pull.md#trigger-pipelines-for-mirror-updates) + is disabled. +- Reduce the frequency of [scheduled pipelines](schedules.md). +- [Skip pipelines](index.md#skip-a-pipeline) when not needed. +- Use [interruptible](../yaml/index.md#interruptible) jobs which can be auto-canceled + if a new pipeline starts. +- If a job doesn't have to run in every pipeline, use [`rules`](../jobs/job_control.md) + to make it only run when it's needed. +- [Use private runners](../runners/runners_scope.md#group-runners) for some jobs. +- If you are working from a fork and you submit a merge request to the parent project, + you can ask a maintainer to run a pipeline [in the parent project](merge_request_pipelines.md#run-pipelines-in-the-parent-project). + +If you manage an open source project, these improvements can also reduce CI/CD minutes +consumption for contributor fork projects, enabling more contributions. + +See our [pipeline efficiency guide](pipeline_efficiency.md) for more details. diff --git a/doc/ci/pipelines/img/pipeline_schedule_play.png b/doc/ci/pipelines/img/pipeline_schedule_play.png Binary files differdeleted file mode 100644 index ec6eb0d156b..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedule_play.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedule_variables.png b/doc/ci/pipelines/img/pipeline_schedule_variables.png Binary files differdeleted file mode 100644 index ce3c3dc6af1..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedule_variables.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_list.png b/doc/ci/pipelines/img/pipeline_schedules_list.png Binary files differdeleted file mode 100644 index 541fe4f9b1d..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_list.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_new_form.png b/doc/ci/pipelines/img/pipeline_schedules_new_form.png Binary files differdeleted file mode 100644 index 993fbf8ca00..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_new_form.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_ownership.png b/doc/ci/pipelines/img/pipeline_schedules_ownership.png Binary files differdeleted file mode 100644 index 8fc5c5fbc82..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_ownership.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 5a5fd2b5540..4d1af735b13 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -140,7 +140,7 @@ to its **Pipelines** tab.  -Click a pipeline to open the **Pipeline Details** page and show +Select a pipeline to open the **Pipeline Details** page and show the jobs that were run for that pipeline. From here you can cancel a running pipeline, retry jobs on a failed pipeline, or [delete a pipeline](#delete-a-pipeline). @@ -246,7 +246,7 @@ For each `var` or `file_var`, a key and value are required. [Manual jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually), allow you to require manual interaction before moving forward in the pipeline. -You can do this straight from the pipeline graph. Just click the play button +You can do this straight from the pipeline graph. Just select the play button to execute that particular job. For example, your pipeline can start automatically, but require a manual action to @@ -259,8 +259,8 @@ In the example below, the `production` stage has a job with a manual action: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11. -Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. -After you click this button, each individual manual action is triggered and refreshed +Multiple manual actions in a single stage can be started at the same time using the "Play all manual" +After you select this action, each individual manual action is triggered and refreshed to an updated status. This functionality is only available: @@ -283,9 +283,9 @@ pipelines. Users with the Owner role for a project can delete a pipeline by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** -page, then using the **Delete** button. +page, then selecting **Delete**. - + WARNING: Deleting a pipeline expires all pipeline caches, and deletes all related objects, @@ -314,7 +314,7 @@ sensitive information like deployment credentials and tokens. **Runners** marked as **protected** can run jobs only on protected branches, preventing untrusted code from executing on the protected runner and preserving deployment keys and other credentials from being unintentionally -accessed. In order to ensure that jobs intended to be executed on protected +accessed. To ensure that jobs intended to be executed on protected runners do not use regular runners, they must be tagged accordingly. ### How pipeline duration is calculated @@ -385,6 +385,12 @@ You can group the jobs by: [Multi-project pipeline graphs](multi_project_pipelines.md#multi-project-pipeline-visualization) help you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** +If a stage contains more than 100 jobs, only the first 100 jobs are listed in the +pipeline graph. The remaining jobs still run as normal. To see the jobs: + +- Select the pipeline, and the jobs are listed on the right side of the pipeline details page. +- On the left sidebar, select **CI/CD > Jobs**. + ### View job dependencies in the pipeline graph > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298973) in GitLab 13.12. @@ -428,7 +434,7 @@ fix it. Pipeline mini graphs only display jobs by stage. -Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs. +Stages in pipeline mini graphs are expandable. Hover your mouse over each stage to see the name and status, and select a stage to expand its jobs list. | Mini graph | Mini graph expanded | |:-------------------------------------------------------------|:---------------------------------------------------------------| diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index dcc3e7e6919..d80b745e6bc 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -41,8 +41,10 @@ Both of these types of pipelines can appear on the **Pipelines** tab of a merge The three types of merge request pipelines are: - Merge request pipelines, which run on the changes in the merge request's - source branch. These pipelines display a `detached` label to indicate that the + source branch. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352939) + in GitLab 14.9, these pipelines display a `merge request` label to indicate that the pipeline ran only on the contents of the source branch, ignoring the target branch. + In GitLab 14.8 and earlier, the label is `detached`. - [Merged results pipelines](merged_results_pipelines.md), which run on the result of combining the source branch's changes with the target branch. - [Merge trains](merge_trains.md), which run when merging multiple merge requests diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index ffcce06cfbd..12ea3a70536 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -201,6 +201,10 @@ the merged result is out of date and the pipeline can't be retried. Instead, you should [add the merge request to the train](#add-a-merge-request-to-a-merge-train) again, which triggers a new pipeline. +If a job only fails intermittently, you can try using the [`retry`](../yaml/index.md#retry) +keyword in the `.gitlab-ci.yml` file to have the job retried before the pipeline completes. +If it succeeds after a retry, the merge request is not removed from the merge train. + ### Unable to add to merge train with message "The pipeline for this merge request failed." Sometimes the **Start/Add to merge train** button is not available and the merge request says, diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 4794107cc87..7df9ea3f72f 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -24,7 +24,7 @@ Merged results pipelines can't run when: - The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). In these cases, the pipeline runs as a [merge request pipeline](merge_request_pipelines.md) -and is labeled as `detached`. +and [is labeled as `merge request`](merge_request_pipelines.md#types-of-merge-request-pipelines). ## Prerequisites diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 457f5ce9c30..0c3100a51f1 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -6,4 +6,6 @@ remove_date: '2022-04-27' This document was moved to [another location](merged_results_pipelines.md). <!-- This redirect file can be deleted after 2022-04-27. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index fe9db3306cd..8813f3e1d59 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -6,142 +6,69 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/schedules. type: reference, howto --- -# Pipeline schedules **(FREE)** +# Scheduled pipelines **(FREE)** -Pipelines are normally run based on certain conditions being met. For example, when a branch is pushed to repository. - -Pipeline schedules can be used to also run [pipelines](index.md) at specific intervals. For example: - -- Every month on the 22nd (cron example: `0 0 22 * *`) for a certain branch. -- Every month on the 2nd Monday (cron example: `0 0 * * 1#2`). -- Every other Sunday at 0900 hours (cron example: `0 9 * * sun%2`). -- Once every day (cron example: `0 0 * * *`). - -Schedule timing is configured with [cron notation](../../topics/cron/index.md). -You can use any cron value, but scheduled pipelines cannot run more frequently -than the instance's [maximum frequency for scheduled pipelines](#advanced-configuration). - -In addition to using the GitLab UI, pipeline schedules can be maintained using the -[Pipeline schedules API](../../api/pipeline_schedules.md). +Use scheduled pipelines to run GitLab CI/CD [pipelines](index.md) at regular intervals. ## Prerequisites -In order for a scheduled pipeline to be created successfully: - -- The schedule owner must have [permissions](../../user/permissions.md) to merge into the target branch. -- The pipeline configuration must be valid. - -Otherwise the pipeline is not created. - -## Configuring pipeline schedules - -To schedule a pipeline for project: - -1. Navigate to the project's **CI/CD > Schedules** page. -1. Click the **New schedule** button. -1. Fill in the **Schedule a new pipeline** form. -1. Click the **Save pipeline schedule** button. - - - -NOTE: -Pipelines execution [timing is dependent](#advanced-configuration) on Sidekiq's own schedule. - -In the **Schedules** index page you can see a list of the pipelines that are -scheduled to run. The next run is automatically calculated by the server GitLab -is installed on. - - - -### Using variables - -You can pass any number of arbitrary variables. They are available in -GitLab CI/CD so that they can be used in your [`.gitlab-ci.yml` file](../../ci/yaml/index.md). - - - -### Using `rules` - -To configure a job to be executed only when the pipeline has been -scheduled, use the [`rules`](../yaml/index.md#rules) keyword. - -In this example, `make world` runs in scheduled pipelines, and `make build` -runs in branch and tag pipelines: - -```yaml -job:on-schedule: - rules: - - if: $CI_PIPELINE_SOURCE == "schedule" - script: - - make world - -job: - rules: - - if: $CI_PIPELINE_SOURCE == "push" - script: - - make build -``` - -### Advanced configuration **(FREE SELF)** - -Scheduled pipelines can be configured with any [cron value](../../topics/cron/index.md), -but they do not always run exactly when scheduled. An internal process, called the -_pipeline schedule worker_, queues all the scheduled pipelines, but does not -run continuously. The worker runs on its own schedule, and scheduled pipelines that -are ready to start are only queued the next time the worker runs. Scheduled pipelines -can't run more frequently than the worker. - -The default frequency of the pipeline schedule worker is `3-59/10 * * * *` (every ten minutes, -starting with `0:03`, `0:13`, `0:23`, and so on). The default frequency for GitLab.com -is listed in the [GitLab.com settings](../../user/gitlab_com/index.md#gitlab-cicd). - -To change the frequency of the pipeline schedule worker: - -1. Edit the `gitlab_rails['pipeline_schedule_worker_cron']` value in your instance's `gitlab.rb` file. -1. [Reconfigure GitLab](../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -For example, to set the maximum frequency of pipelines to twice a day, set `pipeline_schedule_worker_cron` -to a cron value of `0 */12 * * *` (`00:00` and `12:00` every day). +For a scheduled pipeline to run: -## Working with scheduled pipelines +- The schedule owner must have the Developer role. For pipelines on protected branches, + the schedule owner must be [allowed to merge](../../user/project/protected_branches.md#configure-a-protected-branch) + to the branch. +- The [CI/CD configuration](../yaml/index.md) must be valid. -After configuration, GitLab supports many functions for working with scheduled pipelines. +Otherwise, the pipeline is not created. No error message is displayed. -### Running manually +## Add a pipeline schedule -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15700) in GitLab 10.4. +> Scheduled pipelines for tags [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23292) in GitLab 14.9. -To trigger a pipeline schedule manually, click the "Play" button: +To add a pipeline schedule: - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. Select **New schedule** and fill in the form. + - **Interval Pattern**: Select one of the preconfigured intervals, or enter a custom + interval in [cron notation](../../topics/cron/index.md). You can use any cron value, + but scheduled pipelines cannot run more frequently than the instance's + [maximum scheduled pipeline frequency](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency). + - **Target branch or tag**: Select the branch or tag for the pipeline. + - **Variables**: Add any number of [CI/CD variables](../variables/index.md) to the schedule. + These variables are available only when the scheduled pipeline runs, + and not in any other pipeline run. -This schedules a background job to run the pipeline schedule. A flash -message provides a link to the CI/CD Pipeline index page. +## Run manually -NOTE: -To help avoid abuse, users are rate limited to triggering a pipeline once per -minute. +To trigger a pipeline schedule manually, so that it runs immediately instead of +the next scheduled time: -### Taking ownership +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. On the right of the list, for + the pipeline you want to run, select **Play** (**{play}**). -Pipelines are executed as a user, who owns a schedule. This influences what projects and other resources the pipeline has access to. +You can manually run scheduled pipelines once per minute. -If a user does not own a pipeline, you can take ownership by clicking the **Take ownership** button. -The next time a pipeline is scheduled, your credentials are used. +## Take ownership - +Scheduled pipelines execute with the permissions of the user +who owns the schedule. The pipeline has access to the same resources as the pipeline owner, +including [protected environments](../environments/protected_environments.md) and the +[CI/CD job token](../jobs/ci_job_token.md). -If the owner of a pipeline schedule cannot create -pipelines on the target branch, the schedule stops creating new -pipelines. +To take ownership of a pipeline created by a different user: -This can happen if, for example: +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. On the right of the list, for + the pipeline you want to become owner of, select **Take ownership**. -- The owner is blocked or removed from the project. -- The target branch or tag is protected. +## Related topics -In this case, someone with sufficient privileges must take ownership of the -schedule. +- Pipeline schedules can be maintained by using the [Pipeline schedules API](../../api/pipeline_schedules.md). +- You can [control which jobs are added to scheduled pipelines](../jobs/job_control.md#run-jobs-for-scheduled-pipelines). <!-- ## Troubleshooting diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index e22746dbfa0..7960d0afa85 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -190,18 +190,13 @@ You can define how long a job can run before it times out. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Timeout** field, enter the number of minutes, or a human-readable value like `2 hours`. + Must be 10 minutes or more, and less than one month. Default is 60 minutes. Jobs that exceed the timeout are marked as failed. You can override this value [for individual runners](../runners/configure_runners.md#set-maximum-job-timeout-for-a-runner). -## Add test coverage results to a merge request (DEPRECATED) - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) -for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. +## Merge request test coverage results If you use test coverage in your code, you can use a regular expression to find coverage results in the job log. You can then include these results @@ -215,27 +210,38 @@ averaged.  -To define a coverage-parsing regular expression: +### Add test coverage results using `coverage` keyword + +To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression +using the [`coverage`](../yaml/index.md#coverage) keyword. + +Setting the regular expression this way takes precedence over project settings. + +### Add test coverage results using project settings (DEPRECATED) + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) +for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. -- Using the project's `.gitlab-ci.yml`, provide a regular expression using the [`coverage`](../yaml/index.md#coverage) - keyword. Setting the regular expression this way takes precedence over the project's CI/CD settings. +You can add test coverage results to merge requests using the Project's CI/CD settings: -- Using the Project's CI/CD settings: - - Set using the GitLab UI: +- Set using the GitLab UI: - 1. On the top bar, select **Menu > Projects** and find your project. - 1. On the left sidebar, select **Settings > CI/CD**. - 1. Expand **General pipelines**. - 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Settings > CI/CD**. + 1. Expand **General pipelines**. + 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. - - Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) - using the GitLab API with the `build_coverage_regex` attribute: +- Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) + using the GitLab API with the `build_coverage_regex` attribute: - ```shell - curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ - --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ - --data "build_coverage_regex=<your-regular-expression>" - ``` + ```shell + curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ + --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ + --data "build_coverage_regex=<your-regular-expression>" + ``` You can use <https://rubular.com> to test your regular expression. The regular expression returns the **last** match found in the output. @@ -324,7 +330,15 @@ lein cloverage | perl -pe 's/\e\[?.*?[\@-~]//g' Pipeline badges indicate the pipeline status and a test coverage value for your project. These badges are determined by the latest successful pipeline. -### View the code for the pipeline status and coverage reports badges +## Latest release badge + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33368) in GitLab 14.8. + +A latest release badge indicates the latest release tag name for your project. +By default, the badge fetches the release sorted using the [`released_at`](../../api/releases/index.md#create-a-release) time. +Support for [`semver`](https://semver.org/) sorting is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352945). + +### View the code for the pipeline status, coverage reports, and latest release badges You can view the exact link for your badges. Then you can embed the badge in your HTML or Markdown pages. @@ -332,7 +346,7 @@ or Markdown pages. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. -1. In the **Pipeline status** or **Coverage report** sections, view the URLs for the images. +1. In the **Pipeline status**, **Coverage report**, or **Latest release** sections, view the URLs for the images.  @@ -364,9 +378,8 @@ https://gitlab.example.com/<namespace>/<project>/badges/<branch>/pipeline.svg?ig ### Test coverage report badge -You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request-deprecated) -that each job log is matched against. This means that each job in the -pipeline can have the test coverage percentage value defined. +You can define the regular expression for the [coverage report](#merge-request-test-coverage-results) that each job log +is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. To access the test coverage badge, use the following link: @@ -406,6 +419,25 @@ If an invalid boundary is set, GitLab automatically adjusts it to be valid. For if `min_good` is set `80`, and `min_acceptable` is set to `85` (too high), GitLab automatically sets `min_acceptable` to `79` (`min_good` - `1`). +### Latest release badge + +When a release exists in your project, it shows the latest release tag name. If there is no release, +it shows `none`. + +You can access a latest release badge image by using the following link: + +```plaintext +https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg +``` + +#### Sorting preferences + +By default, the latest release badge fetches the release using `release_at` time. The use of the query parameter `?order_by=release_at` is optional, and support for `?order_by=semver` is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352945): + +```plaintext +https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg?order_by=release_at +``` + ### Badge styles Pipeline badges can be rendered in different styles by adding the `style=style_name` parameter to the URL. Two styles are available: diff --git a/doc/ci/runners/build_cloud/linux_build_cloud.md b/doc/ci/runners/build_cloud/linux_build_cloud.md deleted file mode 100644 index 2892a30cd2e..00000000000 --- a/doc/ci/runners/build_cloud/linux_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/linux_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/linux_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/macos/environment.md b/doc/ci/runners/build_cloud/macos/environment.md deleted file mode 100644 index a534e87cc34..00000000000 --- a/doc/ci/runners/build_cloud/macos/environment.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../saas/macos/environment.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../../saas/macos/environment.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/macos_build_cloud.md b/doc/ci/runners/build_cloud/macos_build_cloud.md deleted file mode 100644 index 50b7e0cfb79..00000000000 --- a/doc/ci/runners/build_cloud/macos_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/macos_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/macos_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/windows_build_cloud.md b/doc/ci/runners/build_cloud/windows_build_cloud.md deleted file mode 100644 index fb64938eb9f..00000000000 --- a/doc/ci/runners/build_cloud/windows_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/windows_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/windows_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 60e21653a45..6b14cea51d6 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -23,15 +23,15 @@ if smaller than the [project defined timeout](../pipelines/settings.md#set-a-lim This feature can be used to prevent your shared runner from being overwhelmed by a project that has jobs with a long timeout (for example, one week). -When not configured, runners do not override the project timeout. - On GitLab.com, you cannot override the job timeout for shared runners and must use the [project defined timeout](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run). To set the maximum job timeout: 1. In a project, go to **Settings > CI/CD > Runners**. 1. Select your specific runner to edit the settings. -1. Enter a value under **Maximum job timeout**. +1. Enter a value under **Maximum job timeout**. Must be 10 minutes or more. If not + defined, the [project's job timeout setting](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run) + is used. 1. Select **Save changes**. How this feature works: @@ -59,7 +59,7 @@ How this feature works: ## Be careful with sensitive information -With some [runner executors](https://docs.gitlab.com/runner/executors/README.html), +With some [runner executors](https://docs.gitlab.com/runner/executors/), if you can run a job on the runner, you can get full access to the file system, and thus any code it runs as well as the token of the runner. With shared runners, this means that anyone that runs jobs on the runner, can access anyone else's code that runs on the @@ -70,7 +70,7 @@ to create a clone of a runner and submit false jobs, for example. The above is easily avoided by restricting the usage of shared runners on large public GitLab instances, controlling access to your GitLab instance, -and using more secure [runner executors](https://docs.gitlab.com/runner/executors/README.html). +and using more secure [runner executors](https://docs.gitlab.com/runner/executors/). ### Prevent runners from revealing sensitive information diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 064ad64b79f..257dc02805b 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -13,9 +13,9 @@ If you are using self-managed GitLab or you use GitLab.com but want to use your If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. No configuration is required. Your jobs can run on: -- [Linux runners](build_cloud/linux_build_cloud.md). -- [Windows runners](build_cloud/windows_build_cloud.md) (beta). -- [macOS runners](build_cloud/macos_build_cloud.md) (beta). +- [Linux runners](saas/linux_saas_runner.md). +- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). +- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). The number of minutes you can use on these runners depends on the [maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) diff --git a/doc/ci/runners/runner_cloud/linux_runner_cloud.md b/doc/ci/runners/runner_cloud/linux_runner_cloud.md deleted file mode 100644 index 2892a30cd2e..00000000000 --- a/doc/ci/runners/runner_cloud/linux_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/linux_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/linux_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runner_cloud/macos/environment.md b/doc/ci/runners/runner_cloud/macos/environment.md deleted file mode 100644 index 37ad21c28fc..00000000000 --- a/doc/ci/runners/runner_cloud/macos/environment.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../saas/macos/environment.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../../saas/macos/environment.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
\ No newline at end of file diff --git a/doc/ci/runners/runner_cloud/macos_runner_cloud.md b/doc/ci/runners/runner_cloud/macos_runner_cloud.md deleted file mode 100644 index 50b7e0cfb79..00000000000 --- a/doc/ci/runners/runner_cloud/macos_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/macos_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/macos_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runner_cloud/windows_runner_cloud.md b/doc/ci/runners/runner_cloud/windows_runner_cloud.md deleted file mode 100644 index fb64938eb9f..00000000000 --- a/doc/ci/runners/runner_cloud/windows_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/windows_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/windows_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 7cfd8e50f6c..aa7e268e800 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -236,6 +236,10 @@ To enable or disable a specific runner for a project: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. 1. Click **Enable for this project** or **Disable for this project**. +You can edit a specific runner from any of the projects it's enabled for. +The modifications, which include unlocking, editing tags and the description, +affect all projects that use the runner. + ### Prevent a specific runner from being enabled for other projects You can configure a specific runner so it is "locked" and cannot be enabled for other projects. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index 4d1e628b8e7..fd3fedbc3f5 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -26,7 +26,7 @@ Jobs handled by shared runners on GitLab.com (`shared-runners-manager-X.gitlab.c **time out after 3 hours**, regardless of the timeout configured in a project. Check issue [#4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [#4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. -Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the Beta stage. +Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the [Beta](../../../policy/alpha-beta-support.md#beta-features) stage. Below are the runners' settings. diff --git a/doc/ci/runners/saas/macos/environment.md b/doc/ci/runners/saas/macos/environment.md index 3332eab9b44..d1943a487a7 100644 --- a/doc/ci/runners/saas/macos/environment.md +++ b/doc/ci/runners/saas/macos/environment.md @@ -14,7 +14,7 @@ When you use SaaS runners on macOS: ## VM types The virtual machine where your job runs has `sudo` access with no password. -For the Beta, there is only one available machine type, `gbc-macos-large`. +For the [Beta](../../../../policy/alpha-beta-support.md#beta-features), there is only one available machine type, `gbc-macos-large`. | Instance type | vCPUS | Memory (GB) | | --------- | --- | ------- | diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index bfe408f4485..885a76a7b46 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -4,7 +4,10 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SaaS runners on macOS (Beta) **(FREE SAAS)** +# SaaS runners on macOS (beta) **(FREE SAAS)** + +SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) +and shouldn't be relied upon for mission-critical production jobs. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). @@ -12,9 +15,6 @@ Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a build environment. -SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) -and shouldn't be relied upon for mission-critical production jobs. - ## Quickstart To start using SaaS runners on macOS, you must submit an access request [issue](https://gitlab.com/gitlab-com/macos-buildcloud-runners-beta/-/issues/new?issuable_template=beta_access_request). After your diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index ea0c0d9cc84..5595559b5f0 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -9,6 +9,7 @@ type: concepts, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4. > - `file` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. +> - `VAULT_NAMESPACE` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255619) in GitLab 14.9 and GitLab Runner 14.9. Secrets represent sensitive information your CI job needs to complete work. This sensitive information can be items like API tokens, database credentials, or private keys. @@ -90,6 +91,9 @@ To configure your Vault server: If no role is specified, Vault uses the [default role](https://www.vaultproject.io/api/auth/jwt#default_role) specified when the authentication method was configured. - `VAULT_AUTH_PATH` - Optional. The path where the authentication method is mounted, default is `jwt`. + - `VAULT_NAMESPACE` - Optional. The [Vault Enterprise namespace](https://www.vaultproject.io/docs/enterprise/namespaces) to use for reading secrets and authentication. + If no namespace is specified, Vault uses the `root` ("`/`") namespace. + The setting is ignored by Vault Open Source. NOTE: Support for providing these values in the user interface [is tracked in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/218677). diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index d14027cb1ef..e6406818b4c 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -438,3 +438,7 @@ docker rm -f -v build service-mysql service-postgres This forcefully (`-f`) removes the `build` container, the two service containers, and all volumes (`-v`) that were created with the container creation. + +## Security when using services containers + +Docker privileged mode applies to services. This means that the service image container can access the host system. You should use container images from trusted sources only. diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index e2de47c6c62..14350ac884f 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -231,7 +231,7 @@ The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) N package can generate test reports for .Net Framework and .Net Core applications. The following example expects a solution in the root folder of the repository, with one or more project files in sub-folders. One result file is produced per test project, and each file -is placed in a new artifacts folder. This example includes optional formatting arguments, which +is placed in the artifacts folder. This example includes optional formatting arguments, which improve the readability of test data in the test widget. A full .Net Core [example is available](https://gitlab.com/Siphonophora/dot-net-cicd-test-logging-demo). @@ -353,7 +353,7 @@ displays a list of test suites and cases reported from the XML file.  -You can view all the known test suites and click on each of these to see further +You can view all the known test suites and select each of these to see further details, including the cases that make up the suite. You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a-pipelines-test-report). @@ -366,8 +366,7 @@ If parsing JUnit report XML results in an error, an indicator is shown next to t  -NOTE: -GitLab.com has a 500,000 [test case parsing limit](../user/gitlab_com/#gitlab-cicd). Self-managed administrators can manage this setting on their instance. +For test case parsing limits, see [Max test cases per unit test report](../user/gitlab_com/#gitlab-cicd). GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index ba8451110eb..e4e562bb005 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -653,7 +653,7 @@ The order of precedence for variables is (from highest to lowest): 1. These all have the same (highest) precedence: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). - - [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). + - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). - Variables added when [creating a pipeline with the API](../../api/pipelines.md#create-a-new-pipeline). 1. Project [variables](#custom-cicd-variables). diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 9bab4a7fc77..f3773fbf143 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -1,7 +1,7 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 34db6c61d0b..e5a543e7130 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -302,7 +302,7 @@ In `include` sections in your `.gitlab-ci.yml` file, you can use: In GitLab 14.5 and later, you can also use: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). -- [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). +- [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](../variables/index.md#override-a-variable-when-running-a-pipeline-manually). - Pipeline [predefined variables](../variables/predefined_variables.md). @@ -370,6 +370,11 @@ You can only use the following rules with `include` (and only with [certain vari `rules` keyword `changes` is not supported. +You cannot use [`needs:`](index.md#needs) to create a job dependency that points to +a job added with `include:local:rules`. When the configuration is checked for validity, +GitLab returns `undefined need: <job-name>`. An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) +to improve this behavior. + ## Use `include:local` with wildcard file paths > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25921) in GitLab 13.11. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index d9ea5498b63..a3b91da8f08 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -135,7 +135,9 @@ The `include` files are: - Always evaluated first and then merged with the content of the `.gitlab-ci.yml` file, regardless of the position of the `include` keyword. -You can [nest](includes.md#use-nested-includes) up to 100 includes, but you can't have duplicate includes. +You can [nest](includes.md#use-nested-includes) up to 100 includes. In [GitLab 14.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28987), +the same file can be included multiple times in nested includes, but duplicates are ignored. + In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28212), the time limit to resolve all files is 30 seconds. @@ -1349,7 +1351,7 @@ In this example: **Additional details**: - Coverage regular expressions set in `gitlab-ci.yml` take precedence over coverage regular expression set in the - [GitLab UI](../pipelines/settings.md#add-test-coverage-results-to-a-merge-request-deprecated). + [GitLab UI](../pipelines/settings.md#add-test-coverage-results-using-project-settings-deprecated). - If there is more than one matched line in the job output, the last line is used (the first result of reverse search). - If there are multiple matches in a single line, the last match is searched @@ -1863,7 +1865,7 @@ image: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. -Use `inherit` to [control inheritance of globally-defined defaults and variables](../jobs/index.md#control-the-inheritance-of-default-keywords-and-global-variables). +Use `inherit` to [control inheritance of default keywords and variables](../jobs/index.md#control-the-inheritance-of-default-keywords-and-global-variables). #### `inherit:default` @@ -3690,6 +3692,61 @@ trigger_job: In this example, jobs from subsequent stages wait for the triggered pipeline to successfully complete before starting. +#### `trigger:forward` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213729) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. +The feature is not ready for production use. + +Use `trigger:forward` to specify what to forward to the downstream pipeline. You can control +what is forwarded to both [parent-child pipelines](../pipelines/parent_child_pipelines.md) +and [multi-project pipelines](../pipelines/multi_project_pipelines.md). + +**Possible inputs**: + +- `yaml_variables`: `true` (default), or `false`. When `true`, variables defined + in the trigger job are passed to downstream pipelines. +- `pipeline_variables`: `true` or `false` (default). When `true`, [manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) + are passed to downstream pipelines. + +**Example of `trigger:forward`**: + +[Run this pipeline manually](../pipelines/index.md#run-a-pipeline-manually), with +the CI/CD variable `MYVAR = my value`: + +```yaml +variables: # default variables for each job + VAR: value + +# Default behavior: +# - VAR is passed to the child +# - MYVAR is not passed to the child +child1: + trigger: + include: .child-pipeline.yml + +# Forward pipeline variables: +# - VAR is passed to the child +# - MYVAR is passed to the child +child2: + trigger: + include: .child-pipeline.yml + forward: + pipeline_variables: true + +# Do not forward YAML variables: +# - VAR is not passed to the child +# - MYVAR is not passed to the child +child3: + trigger: + include: .child-pipeline.yml + forward: + yaml_variables: false +``` + ### `variables` [CI/CD variables](../variables/index.md) are configurable values that are passed to jobs. |
