diff options
Diffstat (limited to 'doc/ci/jobs')
| -rw-r--r-- | doc/ci/jobs/ci_job_token.md | 2 | ||||
| -rw-r--r-- | doc/ci/jobs/index.md | 101 | ||||
| -rw-r--r-- | doc/ci/jobs/job_control.md | 104 |
3 files changed, 204 insertions, 3 deletions
diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index 308f38b22b7..b6a3011a3d6 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -20,7 +20,7 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Get job artifacts](../../api/job_artifacts.md#get-job-artifacts). - [Get job token's job](../../api/jobs.md#get-job-tokens-job). - [Pipeline triggers](../../api/pipeline_triggers.md), using the `token=` parameter. -- [Release creation](../../api/releases/index.md#create-a-release). +- [Releases](../../api/releases/index.md). - [Terraform plan](../../user/infrastructure/index.md). The token has the same permissions to access the API as the user that executes the diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index cb3b11ebf99..104badb782c 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -82,6 +82,24 @@ For example:  +## Job name limitations + +You can't use these keywords as job names: + +- `image` +- `services` +- `stages` +- `types` +- `before_script` +- `after_script` +- `variables` +- `cache` +- `include` + +Job names must be 255 characters or less. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342800) +in GitLab 14.5, [with a feature flag](../../administration/feature_flags.md) named `ci_validate_job_length`. +Enabled by default. To disable it, ask an administrator to [disable the feature flag](../../administration/feature_flags.md). + ## Group jobs in a pipeline If you have many similar jobs, your [pipeline graph](../pipelines/index.md#visualize-pipelines) becomes long and hard @@ -139,6 +157,89 @@ In [GitLab 13.8 and earlier](https://gitlab.com/gitlab-org/gitlab/-/merge_reques the regular expression is `\d+[\s:\/\\]+\d+\s*`. [Feature flag](../../user/feature_flags.md) removed in [GitLab 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/322080). +## Hide jobs + +To temporarily disable a job without deleting it from the configuration +file: + +- Comment out the job's configuration: + + ```yaml + # hidden_job: + # script: + # - run test + ``` + +- Start the job name with a dot (`.`) and it is not processed by GitLab CI/CD: + + ```yaml + .hidden_job: + script: + - run test + ``` + +You can use hidden jobs that start with `.` as templates for reusable configuration with: + +- The [`extends` keyword](../yaml/index.md#extends). +- [YAML anchors](../yaml/yaml_optimization.md#anchors). + +## Control the inheritance of default keywords and global variables + +You can control the inheritance of: + +- [default keywords](../yaml/index.md#default) with [`inherit:default`](../yaml/index.md#inheritdefault). +- [global variables](../yaml/index.md#default) with [`inherit:variables`](../yaml/index.md#inheritvariables). + +For example: + +```yaml +default: + image: 'ruby:2.4' + before_script: + - echo Hello World + +variables: + DOMAIN: example.com + WEBHOOK_URL: https://my-webhook.example.com + +rubocop: + inherit: + default: false + variables: false + script: bundle exec rubocop + +rspec: + inherit: + default: [image] + variables: [WEBHOOK_URL] + script: bundle exec rspec + +capybara: + inherit: + variables: false + script: bundle exec capybara + +karma: + inherit: + default: true + variables: [DOMAIN] + script: karma +``` + +In this example: + +- `rubocop`: + - inherits: Nothing. +- `rspec`: + - inherits: the default `image` and the `WEBHOOK_URL` variable. + - does **not** inherit: the default `before_script` and the `DOMAIN` variable. +- `capybara`: + - inherits: the default `before_script` and `image`. + - does **not** inherit: the `DOMAIN` and `WEBHOOK_URL` variables. +- `karma`: + - inherits: the default `image` and `before_script`, and the `DOMAIN` variable. + - does **not** inherit: `WEBHOOK_URL` variable. + ## Specifying variables when running manual jobs > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/30485) in GitLab 12.2. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index ad2bbbc883c..0f92ae5ca49 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -179,7 +179,7 @@ job: ``` You should not include both push and merge request pipelines in the same job without -[`workflow:rules` that prevent duplicate pipelines](../yaml/index.md#switch-between-branch-pipelines-and-merge-request-pipelines): +[`workflow:rules` that prevent duplicate pipelines](../yaml/workflow.md#switch-between-branch-pipelines-and-merge-request-pipelines): ```yaml job: @@ -294,7 +294,7 @@ You can use the `$` character for both variables and paths. For example, if the > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322992) in GitLab 14.3. -Use [`!reference` tags](../yaml/index.md#reference-tags) to reuse rules in different +Use [`!reference` tags](../yaml/yaml_optimization.md#reference-tags) to reuse rules in different jobs. You can combine `!reference` rules with regular job-defined rules: ```yaml @@ -640,6 +640,106 @@ This job can no longer be scheduled to run automatically. You can, however, exec To start a delayed job immediately, select **Play** (**{play}**). Soon GitLab Runner starts the job. +## Parallelize large jobs + +To split a large job into multiple smaller jobs that run in parallel, use the +[`parallel`](../yaml/index.md#parallel) keyword in your `.gitlab-ci.yml` file. + +Different languages and test suites have different methods to enable parallelization. +For example, use [Semaphore Test Boosters](https://github.com/renderedtext/test-boosters) +and RSpec to run Ruby tests in parallel: + +```ruby +# Gemfile +source 'https://rubygems.org' + +gem 'rspec' +gem 'semaphore_test_boosters' +``` + +```yaml +test: + parallel: 3 + script: + - bundle + - bundle exec rspec_booster --job $CI_NODE_INDEX/$CI_NODE_TOTAL +``` + +You can then navigate to the **Jobs** tab of a new pipeline build and see your RSpec +job split into three separate jobs. + +WARNING: +Test Boosters reports usage statistics to the author. + +### Run a one-dimensional matrix of parallel jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/26362) in GitLab 13.5. + +You can create a one-dimensional matrix of parallel jobs: + +```yaml +deploystacks: + stage: deploy + script: + - bin/deploy + parallel: + matrix: + - PROVIDER: [aws, ovh, gcp, vultr] +``` + +You can also [create a multi-dimensional matrix](../yaml/index.md#parallelmatrix). + +### Run a matrix of parallel trigger jobs + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/270957) in GitLab 13.10. + +You can run a [trigger](../yaml/index.md#trigger) job multiple times in parallel in a single pipeline, +but with different variable values for each instance of the job. + +```yaml +deploystacks: + stage: deploy + trigger: + include: path/to/child-pipeline.yml + parallel: + matrix: + - PROVIDER: aws + STACK: [monitoring, app1] + - PROVIDER: ovh + STACK: [monitoring, backup] + - PROVIDER: [gcp, vultr] + STACK: [data] +``` + +This example generates 6 parallel `deploystacks` trigger jobs, each with different values +for `PROVIDER` and `STACK`, and they create 6 different child pipelines with those variables. + +```plaintext +deploystacks: [aws, monitoring] +deploystacks: [aws, app1] +deploystacks: [ovh, monitoring] +deploystacks: [ovh, backup] +deploystacks: [gcp, data] +deploystacks: [vultr, data] +``` + +In [GitLab 14.1 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/239737), you can +use the variables defined in `parallel: matrix` with the [`tags`](../yaml/index.md#tags) keyword for +dynamic runner selection. + +```yaml +deploystacks: + stage: deploy + parallel: + matrix: + - PROVIDER: aws + STACK: [monitoring, app1] + - PROVIDER: gcp + STACK: [data] + tags: + - ${PROVIDER}-${STACK} +``` + ## Use predefined CI/CD variables to run jobs only in specific pipeline types You can use [predefined CI/CD variables](../variables/predefined_variables.md) to choose |