diff options
Diffstat (limited to 'doc/ci/jobs')
| -rw-r--r-- | doc/ci/jobs/ci_job_token.md | 65 | ||||
| -rw-r--r-- | doc/ci/jobs/index.md | 6 | ||||
| -rw-r--r-- | doc/ci/jobs/job_artifacts.md | 358 | ||||
| -rw-r--r-- | doc/ci/jobs/job_artifacts_troubleshooting.md | 149 | ||||
| -rw-r--r-- | doc/ci/jobs/job_control.md | 6 |
5 files changed, 553 insertions, 31 deletions
diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index d9cfbdf124e..9cbf45a16e7 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Security info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -12,7 +12,7 @@ When a pipeline job is about to run, GitLab generates a unique token and injects You can use a GitLab CI/CD job token to authenticate with specific API endpoints: - Packages: - - [Package Registry](../../user/packages/package_registry/index.md#use-gitlab-cicd-to-build-packages). + - [Package Registry](../../user/packages/package_registry/index.md#to-build-packages). - [Packages API](../../api/packages.md) (project-level). - [Container Registry](../../user/packages/container_registry/build_and_push_images.md#use-gitlab-cicd) (the `$CI_REGISTRY_PASSWORD` is `$CI_JOB_TOKEN`). @@ -30,7 +30,7 @@ job to run. A user can cause a job to run by taking action like pushing a commit triggering a manual job, or being the owner of a scheduled pipeline. Therefore, this user must be assigned to [a role that has the required privileges](../../user/permissions.md#gitlab-cicd-permissions). -The token is valid only while the pipeline job runs. After the job finishes, you can't +The token is valid only while the pipeline job runs. After the job finishes, you cannot use the token anymore. A job token can access a project's resources without any configuration, but it might @@ -65,7 +65,9 @@ tries to steal tokens from other jobs. You can control what projects a CI/CD job token can access to increase the job token's security. A job token might give extra permissions that aren't necessary -to access specific private resources. +to access specific private resources. The job token scope only controls access +to private projects. If an accessed project is public or internal, token scoping does +not apply. If a job token is leaked, it could potentially be used to access private data to the job token's user. By limiting the job token access scope, private data cannot @@ -74,38 +76,44 @@ be accessed unless projects are explicitly authorized. There is a proposal to add more strategic control of the access permissions, see [epic 3559](https://gitlab.com/groups/gitlab-org/-/epics/3559). +NOTE: +Because `CI_REGISTRY_TOKEN` uses `CI_JOB_TOKEN` to authenticate, the access configuration +also applies to `CI_REGISTRY_TOKEN`. + ### Allow access to your project with a job token -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.9. [Deployed behind the `:inbound_ci_scoped_job_token` feature flag](../../user/feature_flags.md), enabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.9. [Deployed behind the `:inbound_ci_scoped_job_token` feature flag](../../user/feature_flags.md), enabled by default. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/346298/) in GitLab 15.10. -Control your project's job token scope by creating an **inbound** allowlist of projects which can -access your project through its `CI_JOB_TOKEN`. +Create an allowlist of projects which can access your project through +their `CI_JOB_TOKEN`. -For example, you can add project `B` to the inbound allowlist for project `A`. Jobs -in the pipeline for "allowed project" `B` can now use the CI/CD job token to authenticate -API calls to access project `A`. +For example, project `A` can add project `B` to the allowlist. CI/CD jobs +in project `B` (the "allowed project") can now use their CI/CD job token to +authenticate API calls to access project `A`. If project `A` is public or internal, +the project can be accessed by project `B` without adding it to the allowlist. -By default the allowlist includes your current project. +By default, the allowlist of any project only includes itself. It is a security risk to disable this feature, so project maintainers or owners should keep this setting enabled at all times. Add projects to the allowlist only when cross-project access is needed. -### Disable the inbound job token scope allowlist +### Disable the job token scope allowlist WARNING: It is a security risk to disable the allowlist. A malicious user could try to compromise a pipeline created in an unauthorized project. If the pipeline was created by one of your maintainers, the job token could be used in an attempt to access your project. -You can disable the inbound job token scope allowlist for testing or a similar reason, +You can disable the job token scope allowlist for testing or a similar reason, but you should enable it again as soon as possible. Prerequisite: - You must have at least the Maintainer role for the project. -To disable the inbound job token scope allowlist: +To disable the job token scope allowlist: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. @@ -113,9 +121,11 @@ To disable the inbound job token scope allowlist: 1. Toggle **Allow access to this project with a CI_JOB_TOKEN** to disabled. Enabled by default in new projects. -### Add a project to the inbound job token scope allowlist +You can also disable the allowlist [with the API](../../api/graphql/reference/index.md#mutationprojectcicdsettingsupdate). -You can add projects to the inbound allowlist for a project. Projects added to the allowlist +### Add a project to the job token scope allowlist + +You can add projects to the allowlist for a project. Projects added to the allowlist can make API calls from running pipelines by using the CI/CD job token. Prerequisite: @@ -133,6 +143,8 @@ To add a project: 1. Under **Allow CI job tokens from the following projects to access this project**, add projects to the allowlist. +You can also add a target project to the allowlist [with the API](../../api/graphql/reference/index.md#mutationcijobtokenscopeaddproject). + ### Limit your project's job token access > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/328553) in GitLab 14.1. [Deployed behind the `:ci_scoped_job_token` feature flag](../../user/feature_flags.md), disabled by default. @@ -141,9 +153,9 @@ To add a project: NOTE: This feature is disabled by default for all new projects and is [scheduled for removal](https://gitlab.com/gitlab-org/gitlab/-/issues/383084) -in GitLab 16.0. Project maintainers or owners should enable the **inbound** access control instead. +in GitLab 17.0. Project maintainers or owners should enable the access control instead. -Control your project's job token scope by creating an **outbound** allowlist of projects which +Control your project's job token scope by creating an allowlist of projects which can be accessed by your project's job token. By default, the allowlist includes your current project. @@ -155,16 +167,16 @@ limited only by the user's access permissions. For example, when the setting is enabled, jobs in a pipeline in project `A` have a `CI_JOB_TOKEN` scope limited to project `A`. If the job needs to use the token to make an API request to a private project `B`, then `B` must be added to the allowlist for `A`. -If project `B` is public or internal, it's not required to be added to the allowlist. -The job token scope is only for controlling access to private projects. +If project `B` is public or internal, you do not need to add +`B` to the allowlist to grant access. -### Configure the outbound job token scope +### Configure the job token scope Prerequisite: - You must not have more than 100 projects added to the token's scope. -To configure the outbound job token scope: +To configure the job token scope: 1. On the top bar, select **Main menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. @@ -230,10 +242,17 @@ CI job token failures are usually shown as responses like `404 Not Found` or sim While troubleshooting CI/CD job token authentication issues, be aware that: +- A [GraphQL example mutation](../../api/graphql/getting_started.md#update-project-settings) + is available to toggle the scope settings per project. +- [This comment](https://gitlab.com/gitlab-org/gitlab/-/issues/351740#note_1335673157) + demonstrates how to use graphQL with Bash and cURL to: + - Enable the inbound token access scope. + - Give access to project B from project A, or add B to A's allowlist. + - To remove project access. - When the [CI/CD job token scopes](#configure-cicd-job-token-access) are enabled, and the job token is being used to access a different project: - The user that executes the job must be a member of the project that is being accessed. - The user must have the [permissions](../../user/permissions.md) to perform the action. - - The accessed project must have the project attempting to access it [added to the inbound allowlist](#add-a-project-to-the-inbound-job-token-scope-allowlist). + - The accessed project must have the project attempting to access it [added to the allowlist](#add-a-project-to-the-job-token-scope-allowlist). - The CI job token becomes invalid if the job is no longer running, has been erased, or if the project is in the process of being deleted. diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 5e69ecff7b8..b9c2ee409b8 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Pipeline Execution +group: Pipeline Authoring info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -339,8 +339,8 @@ In the example above: - `date +%s`: The Unix timestamp (for example `1560896352`). - `my_first_section`: The name given to the section. - `\r\e[0K`: Prevents the section markers from displaying in the rendered (colored) - job log, but they are displayed in the raw job log. To see them, in the upper right - of the job log, select **{doc-text}** (**Show complete raw**). + job log, but they are displayed in the raw job log. To see them, in the upper-right corner + of the job log, select **Show complete raw** (**{doc-text}**). - `\r`: carriage return. - `\e[0K`: clear line ANSI escape code. diff --git a/doc/ci/jobs/job_artifacts.md b/doc/ci/jobs/job_artifacts.md new file mode 100644 index 00000000000..0c9dd658ce2 --- /dev/null +++ b/doc/ci/jobs/job_artifacts.md @@ -0,0 +1,358 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Job artifacts **(FREE)** + +Jobs can output an archive of files and directories. This output is known as a job artifact. + +You can download job artifacts by using the GitLab UI or the [API](../../api/job_artifacts.md#get-job-artifacts). + +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview of job artifacts, watch the video [GitLab CI pipelines, artifacts, and environments](https://www.youtube.com/watch?v=PCKDICEe10s). +Or, for an introduction, watch [GitLab CI pipeline tutorial for beginners](https://www.youtube.com/watch?v=Jav4vbUrqII). + +For administrator information about job artifact storage, see [administering job artifacts](../../administration/job_artifacts.md). + +## Create job artifacts + +To create job artifacts, use the [`artifacts`](../yaml/index.md#artifacts) keyword in your `.gitlab-ci.yml` file: + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - mycv.pdf +``` + +In this example, a job named `pdf` calls the `xelatex` command to build a PDF file from the +LaTeX source file, `mycv.tex`. + +The [`paths`](../yaml/index.md#artifactspaths) keyword determines which files to add to the job artifacts. +All paths to files and directories are relative to the repository where the job was created. + +### With wildcards + +You can use wildcards for paths and directories. For example, to create an artifact +with all the files inside the directories that end with `xyz`: + +```yaml +job: + script: echo "build xyz project" + artifacts: + paths: + - path/*xyz/* +``` + +### With an expiry + +The [`expire_in`](../yaml/index.md#artifactsexpire_in) keyword determines how long +GitLab keeps the job artifacts. For example: + +```yaml +pdf: + script: xelatex mycv.tex + artifacts: + paths: + - mycv.pdf + expire_in: 1 week +``` + +If `expire_in` is not defined, the [instance-wide setting](../../user/admin_area/settings/continuous_integration.md#default-artifacts-expiration) +is used. + +To prevent artifacts from expiring, you can select **Keep** from the job details page. +The option is not available when an artifact has no expiry set. + +### With a dynamically defined name + +You can use [CI/CD variables](../variables/index.md) to dynamically define the +artifacts file's name. + +For example, to create an archive with a name of the current job: + +```yaml +job: + artifacts: + name: "$CI_JOB_NAME" + paths: + - binaries/ +``` + +To create an archive with a name of the current branch or tag including only +the binaries directory: + +```yaml +job: + artifacts: + name: "$CI_COMMIT_REF_NAME" + paths: + - binaries/ +``` + +If your branch-name contains forward slashes +(for example `feature/my-feature`) use `$CI_COMMIT_REF_SLUG` +instead of `$CI_COMMIT_REF_NAME` for proper naming of the artifact. + +### With a Windows runner or shell executor + +If you use Windows Batch to run your shell scripts you must replace `$` with `%`: + +```yaml +job: + artifacts: + name: "%CI_JOB_STAGE%-%CI_COMMIT_REF_NAME%" + paths: + - binaries/ +``` + +If you use Windows PowerShell to run your shell scripts you must replace `$` with `$env:`: + +```yaml +job: + artifacts: + name: "$env:CI_JOB_STAGE-$env:CI_COMMIT_REF_NAME" + paths: + - binaries/ +``` + +### Without excluded files + +Use [`artifacts:exclude`](../yaml/index.md#artifactsexclude) to prevent files from +being added to an artifacts archive. + +For example, to store all files in `binaries/`, but not `*.o` files located in +subdirectories of `binaries/`. + +```yaml +artifacts: + paths: + - binaries/ + exclude: + - binaries/**/*.o +``` + +Unlike [`artifacts:paths`](../yaml/index.md#artifactspaths), `exclude` paths are not recursive. +To exclude all of the contents of a directory, match them explicitly rather +than matching the directory itself. + +For example, to store all files in `binaries/` but nothing located in the `temp/` subdirectory: + +```yaml +artifacts: + paths: + - binaries/ + exclude: + - binaries/temp/**/* +``` + +### With untracked files + +Use [`artifacts:untracked`](../yaml/index.md#artifactsuntracked) to add all Git untracked +files as artifacts (along with the paths defined in [`artifacts:paths`](../yaml/index.md#artifactspaths)). Untracked +files are those that haven't been added to the repository but exist in the repository checkout. + +For example, to save all Git untracked files and files in `binaries`: + +```yaml +artifacts: + untracked: true + paths: + - binaries/ +``` + +For example, to save all untracked files but [exclude](../yaml/index.md#artifactsexclude) `*.txt` files: + +```yaml +artifacts: + untracked: true + exclude: + - "*.txt" +``` + +## Prevent a job from fetching artifacts + +Jobs downloads all artifacts from the completed jobs in previous stages by default. +To prevent a job from downloading any artifacts, set [`dependencies`](../yaml/index.md#dependencies) +to an empty array (`[]`): + +```yaml +job: + stage: test + script: make build + dependencies: [] +``` + +## View all job artifacts in a project + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/31271) in GitLab 12.4 [with a flag](../../administration/feature_flags.md) named `artifacts_management_page`. Disabled by default. +> - [Improved look](https://gitlab.com/gitlab-org/gitlab/-/issues/33418) in GitLab 15.6. +> - [Improved performance](https://gitlab.com/gitlab-org/gitlab/-/issues/387765) in GitLab 15.9. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/407475) in GitLab 16.0. Feature flag `artifacts_management_page` removed. + +You can view all artifacts stored in a project from the **CI/CD > Artifacts** page. +This list displays all jobs and their associated artifacts. Expand an entry to access +all artifacts associated with a job, including: + +- Artifacts created with the `artifacts:` keyword. +- [Report artifacts](../yaml/artifacts_reports.md). +- Job logs and metadata, which are stored internally as separate artifacts. + +You can download or delete individual artifacts from this list. + +## Download job artifacts + +You can download job artifacts from: + +- Any **Pipelines** list. On the right of the pipeline, select **Download artifacts** (**{download}**). +- Any **Jobs** list. On the right of the job, select **Download artifacts** (**{download}**). +- A job's detail page. On the right of the page, select **Download**. +- A merge request **Overview** page. On the right of the latest pipeline, select **Artifacts** (**{download}**). +- The [**Artifacts**](#view-all-job-artifacts-in-a-project) page. On the right of the job, select **Download** (**{download}**). +- The [artifacts browser](#browse-the-contents-of-the-artifacts-archive). On the top of the page, + select **Download artifacts archive** (**{download}**). + +[Report artifacts](../yaml/artifacts_reports.md) can only be downloaded from the **Pipelines** list +or **Artifacts** page. + +You can download job artifacts from the latest successful pipeline by using [the job artifacts API](../../api/job_artifacts.md). +You cannot download [artifact reports](../yaml/artifacts_reports.md) with the job artifacts API, +unless the report is added as a regular artifact with `artifacts:paths`. + +### From a URL + +You can download the artifacts archive for a specific job with a publicly accessible +URL for the [job artifacts API](../../api/job_artifacts.md#download-the-artifacts-archive). + +For example, to download the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/download?job=build +``` + +For example, to download the file `review/index.html` from the latest job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/api/v4/projects/<project-id>/jobs/artifacts/main/raw/review/index.html?job=build +``` + +In both examples, replace `<project-id>` with a valid project ID, found at the top of the project details page. + +Artifacts for [parent and child pipelines](../pipelines/downstream_pipelines.md#parent-child-pipelines) +are searched in hierarchical order from parent to child. For example, if both parent and +child pipelines have a job with the same name, the job artifacts from the parent pipeline are returned. + +## Browse the contents of the artifacts archive + +You can browse the contents of the artifacts from the UI without downloading the artifact locally, +from: + +- Any **Jobs** list. On the right of the job, select **Browse** (**{folder-open}**). +- A job's detail page. On the right of the page, select **Browse**. +- The **Artifacts** page. On the right of the job, select **Browse** (**{folder-open}**). + +If [GitLab Pages](../../administration/pages/index.md) is enabled in the project, you can preview +HTML files in the artifacts directly in your browser. If the project is internal or private, you must +enable [GitLab Pages access control](../../administration/pages/index.md#access-control) to preview +HTML files. + +### From a URL + +You can browse the job artifacts of the latest successful pipeline for a specific job +with a publicly accessible URL. + +For example, to browse the latest artifacts of a job named `build` in the `main` branch of a project on GitLab.com: + +```plaintext +https://gitlab.com/<full-project-path>/-/jobs/artifacts/main/browse?job=build +``` + +Replace `<full-project-path>` with a valid project path, you can find it in the URL for your project. + +## Delete job log and artifacts + +WARNING: +Deleting the job log and artifacts is a destructive action that cannot be reverted. Use with caution. +Deleting certain files, including report artifacts, job logs, and metadata files, affects +GitLab features that use these files as data sources. + +You can delete a job's artifacts and log. + +Prerequisites: + +- You must be the owner of the job or a user with at least the Maintainer role for the project. + +To delete a job: + +1. Go to a job's detail page. +1. In the upper-right corner of the job's log, select **Erase job log and artifacts** (**{remove}**). + +You can also delete individual artifacts from the [**Artifacts** page](#bulk-delete-artifacts). + +### Bulk delete artifacts + +- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33348) in GitLab 15.10 [with a flag](../../administration/feature_flags.md) named `ci_job_artifact_bulk_destroy`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_job_artifact_bulk_destroy`. +The feature is not ready for production use. + +You can delete multiple artifacts at the same time: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Artifacts**. +1. Select the checkboxes next to the artifacts you want to delete. You can select up to 50 artifacts. +1. Select **Delete selected**. + +## Link to job artifacts in the merge request UI + +Use the [`artifacts:expose_as`](../yaml/index.md#artifactsexpose_as) keyword to display +a link to job artifacts in the [merge request](../../user/project/merge_requests/index.md) UI. + +For example, for an artifact with a single file: + +```yaml +test: + script: ["echo 'test' > file.txt"] + artifacts: + expose_as: 'artifact 1' + paths: ['file.txt'] +``` + +With this configuration, GitLab adds **artifact 1** as a link to `file.txt` to the +**View exposed artifact** section of the relevant merge request. + +## Keep artifacts from most recent successful jobs + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16267) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/229936) in GitLab 13.4. +> - [Made optional with a CI/CD setting](https://gitlab.com/gitlab-org/gitlab/-/issues/241026) in GitLab 13.8. + +By default artifacts are always kept for successful pipelines for the most recent commit on +each ref. This means that the latest artifacts do not immediately expire according +to the `expire_in` specification. + +If a pipeline for a new commit on the same ref completes successfully, the previous pipeline's +artifacts are deleted according to the `expire_in` configuration. The artifacts +of the new pipeline are kept automatically. If multiple pipelines run for the most +recent commit on the ref, all artifacts are kept. + +Keeping the latest artifacts can use a large amount of storage space in projects +with a lot of jobs or large artifacts. If the latest artifacts are not needed in +a project, you can disable this behavior to save space: + +1. On the top bar, select **Main menu > Projects** and find your project. +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Artifacts**. +1. Clear the **Keep artifacts from most recent successful jobs** checkbox. + +You can disable this behavior for all projects on a self-managed instance in the +[instance's CI/CD settings](../../user/admin_area/settings/continuous_integration.md#keep-the-latest-artifacts-for-all-jobs-in-the-latest-successful-pipelines). + +When **Keep artifacts from most recent successful jobs** is enabled, artifacts are always kept for [blocked](job_control.md#types-of-manual-jobs) +pipelines. These artifacts expire only after the blocking job is triggered and the pipeline completes. +[Issue 387087](https://gitlab.com/gitlab-org/gitlab/-/issues/387087) proposes to change this behavior. diff --git a/doc/ci/jobs/job_artifacts_troubleshooting.md b/doc/ci/jobs/job_artifacts_troubleshooting.md new file mode 100644 index 00000000000..f5951350085 --- /dev/null +++ b/doc/ci/jobs/job_artifacts_troubleshooting.md @@ -0,0 +1,149 @@ +--- +stage: Verify +group: Pipeline Security +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# Troubleshooting job artifacts + +When working with [job artifacts](job_artifacts.md), you might encounter the following issues. + +## Job does not retrieve certain artifacts + +By default, jobs fetch all artifacts from previous stages, but jobs using `dependencies` +or `needs` do not fetch artifacts from all jobs by default. + +If you use these keywords, artifacts are fetched from only a subset of jobs. Review +the keyword reference for information on how to fetch artifacts with these keywords: + +- [`dependencies`](../yaml/index.md#dependencies) +- [`needs`](../yaml/index.md#needs) +- [`needs:artifacts`](../yaml/index.md#needsartifacts) + +## Job artifacts use too much disk space + +If job artifacts are using too much disk space, see the +[job artifacts administration documentation](../../administration/job_artifacts.md#job-artifacts-using-too-much-disk-space). + +## Error message `No files to upload` + +This message appears in job logs when a the runner can't find the file to upload. Either +the path to the file is incorrect, or the file was not created. You can check the job +log for other errors or warnings that specify the filename and why it wasn't +generated. + +For more detailed job logs, you can [enable CI/CD debug logging](../variables/index.md#enable-debug-logging) +and try the job again. This logging might provide more information about why the file +wasn't created. + +## Error message `Missing /usr/bin/gitlab-runner-helper. Uploading artifacts is disabled.` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3068) in GitLab 15.2, GitLab Runner uses `RUNNER_DEBUG` instead of `DEBUG`, fixing this issue. + +In GitLab 15.1 and earlier, setting a CI/CD variable named `DEBUG` can cause artifact uploads to fail. + +To work around this, you can: + +- Update to GitLab and GitLab Runner 15.2 +- Use a different variable name +- Set it as an environment variable in a `script` command: + + ```yaml + failing_test_job: # This job might fail due to issue gitlab-org/gitlab-runner#3068 + variables: + DEBUG: true + script: bin/mycommand + artifacts: + paths: + - bin/results + + successful_test_job: # This job does not define a CI/CD variable named `DEBUG` and is not affected by the issue + script: DEBUG=true bin/mycommand + artifacts: + paths: + - bin/results + ``` + +## Error message `FATAL: invalid argument` when uploading a dotenv artifact on a Windows runner + +The PowerShell `echo` command writes files with UCS-2 LE BOM (Byte Order Mark) encoding, +but only UTF-8 is supported. If you try to create a [`dotenv`](../yaml/artifacts_reports.md) +artifact with `echo`, it causes a `FATAL: invalid argument` error. + +Use PowerShell `Add-Content` instead, which uses UTF-8: + +```yaml +test-job: + stage: test + tags: + - windows + script: + - echo "test job" + - Add-Content -Path build.env -Value "MY_ENV_VAR=true" + artifacts: + reports: + dotenv: build.env +``` + +## Job artifacts do not expire + +If some job artifacts are not expiring as expected, check if the +[**Keep artifacts from most recent successful jobs**](job_artifacts.md#keep-artifacts-from-most-recent-successful-jobs) +setting is enabled. + +When this setting is enabled, job artifacts from the latest successful pipeline +of each ref do not expire and are not deleted. + +## Error message `This job could not start because it could not retrieve the needed artifacts.` + +A job configured with the [`needs:artifacts`](../yaml/index.md#needsartifacts) keyword +fails to start and returns this error message if: + +- The job's dependencies cannot be found. +- The job cannot access the relevant resources due to insufficient permissions. + +The troubleshooting steps to follow differ based on the syntax the job uses: + +- [`needs:project`](#for-a-job-configured-with-needsproject) +- [`needs:pipeline:job`](#for-a-job-configured-with-needspipelinejob) + +### For a job configured with `needs:project` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:project`](../yaml/index.md#needsproject) with a configuration similar to: + +```yaml +rspec: + needs: + - project: my-group/my-project + job: dependency-job + ref: master + artifacts: true +``` + +To troubleshoot this error, verify that: + +- Project `my-group/my-project` is in a group with a Premium subscription plan. +- The user running the job can access resources in `my-group/my-project`. +- The `project`, `job`, and `ref` combination exists and results in the desired dependency. +- Any variables in use evaluate to the correct values. + +### For a job configured with `needs:pipeline:job` + +The `could not retrieve the needed artifacts.` error can happen for a job using +[`needs:pipeline:job`](../yaml/index.md#needspipelinejob) with a configuration similar to: + +```yaml +rspec: + needs: + - pipeline: $UPSTREAM_PIPELINE_ID + job: dependency-job + artifacts: true +``` + +To troubleshoot this error, verify that: + +- The `$UPSTREAM_PIPELINE_ID` CI/CD variable is available in the current pipeline's + parent-child pipeline hierarchy. +- The `pipeline` and `job` combination exists and resolves to an existing pipeline. +- `dependency-job` has run and finished successfully. diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 3cd57ff6a6a..fa045a898fa 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -10,11 +10,7 @@ When a new pipeline starts, GitLab checks the pipeline configuration to determin which jobs should run in that pipeline. You can configure jobs to run depending on factors like the status of variables, or the pipeline type. -To configure a job to be included or excluded from certain pipelines, you can use: - -- [`rules`](../yaml/index.md#rules) -- [`only`](../yaml/index.md#only--except) -- [`except`](../yaml/index.md#only--except) +To configure a job to be included or excluded from certain pipelines, use [`rules`](../yaml/index.md#rules). Use [`needs`](../yaml/index.md#needs) to configure a job to run as soon as the earlier jobs it depends on finish running. |