diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-06-22 15:08:59 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-06-22 15:08:59 +0300 |
commit | 421f6c92d5984d035a7a6687d70277ba88f5f92b (patch) | |
tree | 35286624cfbc591c0f60bf1f0851492cd9d74ae3 /doc/user/project | |
parent | 44c4a8ee1e7b995aa2045fd31d637981eb3baccb (diff) |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/user/project')
-rw-r--r-- | doc/user/project/clusters/kubernetes_pod_logs.md | 35 | ||||
-rw-r--r-- | doc/user/project/integrations/bamboo.md | 64 | ||||
-rw-r--r-- | doc/user/project/merge_requests/fail_fast_testing.md | 100 | ||||
-rw-r--r-- | doc/user/project/merge_requests/img/load_performance_testing.png | bin | 17506 -> 0 bytes | |||
-rw-r--r-- | doc/user/project/merge_requests/img/test_coverage_visualization_v12_9.png | bin | 17559 -> 0 bytes | |||
-rw-r--r-- | doc/user/project/merge_requests/load_performance_testing.md | 204 | ||||
-rw-r--r-- | doc/user/project/merge_requests/test_coverage_visualization.md | 444 |
7 files changed, 80 insertions, 767 deletions
diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 58006c29057..e28371a42ed 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -33,9 +33,6 @@ above the log file data, depending on your configuration: - **Namespace** - Select the environment to display. Users with Maintainer or greater [permissions](../../permissions.md) can also see pods in the `gitlab-managed-apps` namespace. -- **Search** - Only available if the [Elastic Stack integration](../../clusters/integrations.md#elastic-stack-cluster-integration) is enabled. -- **Select time range** - Select the range of time to display. - Only available if the [Elastic Stack integration](../../clusters/integrations.md#elastic-stack-cluster-integration) is enabled. - **Scroll to bottom** **{scroll_down}** - Scroll to the end of the displayed logs. - **Refresh** **{retry}** - Reload the displayed logs. @@ -76,8 +73,6 @@ The **Log Explorer** lets you filter the logs by: - Pods. - [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/5769), environments. -- [From GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21656), - [full text search](#full-text-search). - [From GitLab 12.8](https://gitlab.com/gitlab-org/gitlab/-/issues/197879), dates. - [From GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/208790), managed apps. @@ -89,33 +84,3 @@ Support for pods with multiple containers is coming Support for historical data is coming [in a future release](https://gitlab.com/gitlab-org/gitlab/-/issues/196191). - -### Filter by date - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197879) in GitLab 12.8. - -When you enable [Elastic Stack](../../clusters/integrations.md#elastic-stack-cluster-integration) -on your cluster, you can filter logs displayed in the **Log Explorer** by date. - -Select **Show last** in the **Log Explorer** to see the available options. - -### Full text search - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21656) in GitLab 12.7. - -When you enable [Elastic Stack](../../clusters/integrations.md#elastic-stack-cluster-integration) on your cluster, -you can search the content of your logs through a search bar. The search is passed -to Elasticsearch using the -[simple_query_string](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html) -Elasticsearch function, which supports the following operators: - -| Operator | Description | -|----------------------------|-------------------------------------------------------------| -| `\|` | An `OR` operation. | -| `-` | Negates a single token. | -| `+` | An `AND` operation. | -| `"` | Wraps a number of tokens to signify a phrase for searching. | -| `*` (at the end of a term) | A prefix query. | -| `(` and `)` | Precedence. | -| `~N` (after a word) | Edit distance (fuzziness). | -| `~N` (after a phrase) | Slop amount. | diff --git a/doc/user/project/integrations/bamboo.md b/doc/user/project/integrations/bamboo.md index 22e6d45dd96..75f099268cb 100644 --- a/doc/user/project/integrations/bamboo.md +++ b/doc/user/project/integrations/bamboo.md @@ -9,11 +9,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can automatically trigger builds in Atlassian Bamboo when you push changes to your project in GitLab. -When this integration is configured, merge requests also display the following information: - -- A CI/CD status that shows if the build is pending, failed, or has completed successfully. -- A link to the Bamboo build page for more information. - Bamboo doesn't provide the same features as a traditional build system when accepting webhooks and commit data. You must configure a Bamboo build plan before you configure the integration in GitLab. @@ -66,6 +61,65 @@ for example `PROJ-PLAN`. The build key is included in the browser URL when you view a plan in Bamboo. For example, `https://bamboo.example.com/browse/PROJ-PLAN`. +## Update Bamboo build status in GitLab + +You can use a script that uses the [commit status API](../../../api/commits.md#post-the-build-status-to-a-commit) +and Bamboo build variables to: + +- Update the commit with the build status. +- Add the Bamboo build plan URL as the commit's `target_url`. + +For example: + +1. Create an [access token](../../../api/index.md#personalprojectgroup-access-tokens) in GitLab with `:api` permissions. +1. Save the token as a `$GITLAB_TOKEN` variable in Bamboo. +1. Add the following script as a final task to the Bamboo plan's jobs: + + ```shell + #!/bin/bash + + # Script to update CI status on GitLab. + # Add this script as final inline script task in a Bamboo job. + # + # General documentation: https://docs.gitlab.com/ee/user/project/integrations/bamboo.html + # Fix inspired from https://gitlab.com/gitlab-org/gitlab/-/issues/34744 + + # Stop at first error + set -e + + # Access token. Set this as a CI variable in Bamboo. + #GITLAB_TOKEN= + + # Status + cistatus="failed" + if [ "${bamboo_buildFailed}" = "false" ]; then + cistatus="success" + fi + + repo_url="${bamboo_planRepository_repositoryUrl}" + + # Check if we use SSH or HTTPS + protocol=${repo_url::4} + if [ "$protocol" == "git@" ]; then + repo=${repo_url:${#protocol}}; + gitlab_url=${repo%%:*}; + else + protocol="https://" + repo=${repo_url:${#protocol}}; + gitlab_url=${repo%%/*}; + fi + + start=$((${#gitlab_url} + 1)) # +1 for the / (https) or : (ssh) + end=$((${#repo} - $start -4)) # -4 for the .git + repo=${repo:$start:$end} + repo=$(echo "$repo" | sed "s/\//%2F/g") + + # Send request + url="https://${gitlab_url}/api/v4/projects/${repo}/statuses/${bamboo_planRepository_revision}?state=${cistatus}&target_url=${bamboo_buildResultsUrl}" + echo "Sending request to $url" + curl --fail --request POST --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "$url" + ``` + ## Troubleshooting ### Builds not triggered diff --git a/doc/user/project/merge_requests/fail_fast_testing.md b/doc/user/project/merge_requests/fail_fast_testing.md index 355661516a7..c09a7c14c06 100644 --- a/doc/user/project/merge_requests/fail_fast_testing.md +++ b/doc/user/project/merge_requests/fail_fast_testing.md @@ -1,97 +1,11 @@ --- -stage: Verify -group: Pipeline Insights -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: '../../../ci/testing/fail_fast_testing.md' +remove_date: '2022-08-31' --- -# Fail Fast Testing **(PREMIUM)** +This document was moved to [another location](../../../ci/testing/fail_fast_testing.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198550) in GitLab 13.1. - -For applications that use RSpec for running tests, we've introduced the `Verify/Failfast` -[template to run subsets of your test suite](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Verify/FailFast.gitlab-ci.yml), -based on the changes in your merge request. - -The template uses the [test_file_finder (`tff`) gem](https://gitlab.com/gitlab-org/ci-cd/test_file_finder/) -that accepts a list of files as input, and returns a list of spec (test) files -that it believes to be relevant to the input files. - -`tff` is designed for Ruby on Rails projects, so the `Verify/FailFast` template is -configured to run when changes to Ruby files are detected. By default, it runs in -the [`.pre` stage](../../../ci/yaml/index.md#stage-pre) of a GitLab CI/CD pipeline, -before all other stages. - -## Example use case - -Fail fast testing is useful when adding new functionality to a project and adding -new automated tests. - -Your project could have hundreds of thousands of tests that take a long time to complete. -You may be confident that a new test will pass, but you have to wait for all the tests -to complete to verify it. This could take an hour or more, even when using parallelization. - -Fail fast testing gives you a faster feedback loop from the pipeline. It lets you -know quickly that the new tests are passing and the new functionality did not break -other tests. - -## Requirements - -This template requires: - -- A project built in Rails that uses RSpec for testing. -- CI/CD configured to: - - Use a Docker image with Ruby available. - - Use [Merge request pipelines](../../../ci/pipelines/merge_request_pipelines.md#prerequisites) -- [Merged results pipelines](../../../ci/pipelines/merged_results_pipelines.md#enable-merged-results-pipelines) - enabled in the project settings. -- A Docker image with Ruby available. The template uses `image: ruby:2.6` by default, but you [can override](../../../ci/yaml/includes.md#override-included-configuration-values) this. - -## Configuring Fast RSpec Failure - -We use the following plain RSpec configuration as a starting point. It installs all the -project gems and executes `rspec`, on merge request pipelines only. - -```yaml -rspec-complete: - stage: test - rules: - - if: $CI_PIPELINE_SOURCE == "merge_request_event" - script: - - bundle install - - bundle exec rspec -``` - -To run the most relevant specs first instead of the whole suite, [`include`](../../../ci/yaml/index.md#include) -the template by adding the following to your CI/CD configuration: - -```yaml -include: - - template: Verify/FailFast.gitlab-ci.yml -``` - -To customize the job, specific options may be set to override the template. For example, to override the default Docker image: - -```yaml -include: - - template: Verify/FailFast.gitlab-ci.yml - -rspec-rails-modified-path-specs: - image: custom-docker-image-with-ruby -``` - -### Example test loads - -For illustrative purposes, let's say our Rails app spec suite consists of 100 specs per model for ten models. - -If no Ruby files are changed: - -- `rspec-rails-modified-paths-specs` does not run any tests. -- `rspec-complete` runs the full suite of 1000 tests. - -If one Ruby model is changed, for example `app/models/example.rb`, then `rspec-rails-modified-paths-specs` -runs the 100 tests for `example.rb`: - -- If all of these 100 tests pass, then the full `rspec-complete` suite of 1000 tests is allowed to run. -- If any of these 100 tests fail, they fail quickly, and `rspec-complete` does not run any tests. - -The final case saves resources and time as the full 1000 test suite does not run. +<!-- This redirect file can be deleted after <2022-09-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/img/load_performance_testing.png b/doc/user/project/merge_requests/img/load_performance_testing.png Binary files differdeleted file mode 100644 index d5623867ee7..00000000000 --- a/doc/user/project/merge_requests/img/load_performance_testing.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/test_coverage_visualization_v12_9.png b/doc/user/project/merge_requests/img/test_coverage_visualization_v12_9.png Binary files differdeleted file mode 100644 index 1922a566dd5..00000000000 --- a/doc/user/project/merge_requests/img/test_coverage_visualization_v12_9.png +++ /dev/null diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index 8d58cf36cdb..04b62c5d8fe 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -1,201 +1,11 @@ --- -stage: Verify -group: Pipeline Insights -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: '../../../ci/testing/load_performance_testing.md' +remove_date: '2022-08-31' --- -# Load Performance Testing **(PREMIUM)** +This document was moved to [another location](../../../ci/testing/load_performance_testing.md). -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10683) in GitLab 13.2. - -With Load Performance Testing, you can test the impact of any pending code changes -to your application's backend in [GitLab CI/CD](../../../ci/index.md). - -GitLab uses [k6](https://k6.io/), a free and open source -tool, for measuring the system performance of applications under -load. - -Unlike [Browser Performance Testing](../../../ci/testing/browser_performance_testing.md), which is -used to measure how web sites perform in client browsers, Load Performance Testing -can be used to perform various types of [load tests](https://k6.io/docs/#use-cases) -against application endpoints such as APIs, Web Controllers, and so on. -This can be used to test how the backend or the server performs at scale. - -For example, you can use Load Performance Testing to perform many concurrent -GET calls to a popular API endpoint in your application to see how it performs. - -## How Load Performance Testing works - -First, define a job in your `.gitlab-ci.yml` file that generates the -[Load Performance report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsload_performance). -GitLab checks this report, compares key load performance metrics -between the source and target branches, and then shows the information in a merge request widget: - -![Load Performance Widget](img/load_performance_testing.png) - -Next, you need to configure the test environment and write the k6 test. - -The key performance metrics that the merge request widget shows after the test completes are: - -- Checks: The percentage pass rate of the [checks](https://k6.io/docs/using-k6/checks) configured in the k6 test. -- TTFB P90: The 90th percentile of how long it took to start receiving responses, aka the [Time to First Byte](https://en.wikipedia.org/wiki/Time_to_first_byte) (TTFB). -- TTFB P95: The 95th percentile for TTFB. -- RPS: The average requests per second (RPS) rate the test was able to achieve. - -NOTE: -If the Load Performance report has no data to compare, such as when you add the -Load Performance job in your `.gitlab-ci.yml` for the very first time, -the Load Performance report widget doesn't display. It must have run at least -once on the target branch (`main`, for example), before it displays in a -merge request targeting that branch. - -## Configure the Load Performance Testing job - -Configuring your Load Performance Testing job can be broken down into several distinct parts: - -- Determine the test parameters such as throughput, and so on. -- Set up the target test environment for load performance testing. -- Design and write the k6 test. - -### Determine the test parameters - -The first thing you need to do is determine the [type of load test](https://k6.io/docs/test-types/introduction) -you want to run, and how it will run (for example, the number of users, throughput, and so on). - -Refer to the [k6 docs](https://k6.io/docs/), especially the [k6 testing guides](https://k6.io/docs/testing-guides), -for guidance on the above and more. - -### Test Environment setup - -A large part of the effort around load performance testing is to prepare the target test environment -for high loads. You should ensure it's able to handle the -[throughput](https://k6.io/blog/monthly-visits-concurrent-users) it will be tested with. - -It's also typically required to have representative test data in the target environment -for the load performance test to use. - -We strongly recommend [not running these tests against a production environment](https://k6.io/our-beliefs#load-test-in-a-pre-production-environment). - -### Write the load performance test - -After the environment is prepared, you can write the k6 test itself. k6 is a flexible -tool and can be used to run [many kinds of performance tests](https://k6.io/docs/test-types/introduction). -Refer to the [k6 documentation](https://k6.io/docs/) for detailed information on how to write tests. - -### Configure the test in GitLab CI/CD - -When your k6 test is ready, the next step is to configure the load performance -testing job in GitLab CI/CD. The easiest way to do this is to use the -[`Verify/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Verify/Load-Performance-Testing.gitlab-ci.yml) -template that is included with GitLab. - -NOTE: -For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual -test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) -for spec details. The [default shared GitLab.com runners](../../../ci/runners/saas/linux_saas_runner.md) -likely have insufficient specs to handle most large k6 tests. - -This template runs the -[k6 Docker container](https://hub.docker.com/r/loadimpact/k6/) in the job and provides several ways to customize the -job. - -An example configuration workflow: - -1. Set up GitLab Runner to run Docker containers, like the - [Docker-in-Docker workflow](../../../ci/docker/using_docker_build.md#use-docker-in-docker). -1. Configure the default Load Performance Testing CI/CD job in your `.gitlab-ci.yml` file. - You need to include the template and configure it with CI/CD variables: - - ```yaml - include: - template: Verify/Load-Performance-Testing.gitlab-ci.yml - - load_performance: - variables: - K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT> - ``` - -The above example creates a `load_performance` job in your CI/CD pipeline that runs -the k6 test. - -NOTE: -For Kubernetes setups a different template should be used: [`Jobs/Load-Performance-Testing.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Load-Performance-Testing.gitlab-ci.yml). - -k6 has [various options](https://k6.io/docs/using-k6/options) to configure how it will run tests, such as what throughput (RPS) to run with, -how long the test should run, and so on. Almost all options can be configured in the test itself, but as -you can also pass command line options via the `K6_OPTIONS` variable. - -For example, you can override the duration of the test with a CLI option: - -```yaml - include: - template: Verify/Load-Performance-Testing.gitlab-ci.yml - - load_performance: - variables: - K6_TEST_FILE: <PATH TO K6 TEST FILE IN PROJECT> - K6_OPTIONS: '--duration 30s' -``` - -GitLab only displays the key performance metrics in the MR widget if k6's results are saved -via [summary export](https://k6.io/docs/results-visualization/json#summary-export) -as a [Load Performance report artifact](../../../ci/yaml/artifacts_reports.md#artifactsreportsload_performance). -The latest Load Performance artifact available is always used, using the -summary values from the test. - -If [GitLab Pages](../pages/index.md) is enabled, you can view the report directly in your browser. - -### Load Performance testing in Review Apps - -The CI/CD YAML configuration example above works for testing against static environments, -but it can be extended to work with [review apps](../../../ci/review_apps) or -[dynamic environments](../../../ci/environments) with a few extra steps. - -The best approach is to capture the dynamic URL in a [`.env` file](https://docs.docker.com/compose/env-file/) -as a job artifact to be shared, then use a custom CI/CD variable we've provided named `K6_DOCKER_OPTIONS` -to configure the k6 Docker container to use the file. With this, k6 can then use any -environment variables from the `.env` file in scripts using standard JavaScript, -such as: ``http.get(`${__ENV.ENVIRONMENT_URL}`)``. - -For example: - -1. In the `review` job: - 1. Capture the dynamic URL and save it into a `.env` file, for example, `echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env`. - 1. Set the `.env` file to be a [job artifact](../../../ci/pipelines/job_artifacts.md#job-artifacts). -1. In the `load_performance` job: - 1. Set it to depend on the review job, so it inherits the environment file. - 1. Set the `K6_DOCKER_OPTIONS` variable with the [Docker CLI option for environment files](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file), for example `--env-file review.env`. -1. Configure the k6 test script to use the environment variable in it's steps. - -Your `.gitlab-ci.yml` file might be similar to: - -```yaml -stages: - - deploy - - performance - -include: - template: Verify/Load-Performance-Testing.gitlab-ci.yml - -review: - stage: deploy - environment: - name: review/$CI_COMMIT_REF_SLUG - url: http://$CI_ENVIRONMENT_SLUG.example.com - script: - - run_deploy_script - - echo "ENVIRONMENT_URL=$CI_ENVIRONMENT_URL" >> review.env - artifacts: - paths: - - review.env - rules: - - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. - -load_performance: - dependencies: - - review - variables: - K6_DOCKER_OPTIONS: '--env-file review.env' - rules: - - if: $CI_COMMIT_BRANCH # Modify to match your pipeline rules, or use `only/except` if needed. -``` +<!-- This redirect file can be deleted after <2022-09-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index fcbd732f8ee..53d45e6940d 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -1,441 +1,11 @@ --- -stage: Verify -group: Pipeline Insights -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: '../../../ci/testing/test_coverage_visualization.md' +remove_date: '2022-08-31' --- -# Test coverage visualization **(FREE)** +This document was moved to [another location](../../../ci/testing/test_coverage_visualization.md). -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/3708) in GitLab 12.9. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/249811) in GitLab 13.5. - -With the help of [GitLab CI/CD](../../../ci/index.md), you can collect the test -coverage information of your favorite testing or coverage-analysis tool, and visualize -this information inside the file diff view of your merge requests (MRs). This will allow you -to see which lines are covered by tests, and which lines still require coverage, before the -MR is merged. - -![Test Coverage Visualization Diff View](img/test_coverage_visualization_v12_9.png) - -## How test coverage visualization works - -Collecting the coverage information is done via GitLab CI/CD's -[artifacts reports feature](../../../ci/yaml/index.md#artifactsreports). -You can specify one or more coverage reports to collect, including wildcard paths. -GitLab then takes the coverage information in all the files and combines it -together. Coverage files are parsed in a background job so there can be a delay -between pipeline completion and the visualization loading on the page. - -For the coverage analysis to work, you have to provide a properly formatted -[Cobertura XML](https://cobertura.github.io/cobertura/) report to -[`artifacts:reports:coverage_report`](../../../ci/yaml/artifacts_reports.md#artifactsreportscoverage_report). -This format was originally developed for Java, but most coverage analysis frameworks -for other languages have plugins to add support for it, like: - -- [simplecov-cobertura](https://rubygems.org/gems/simplecov-cobertura) (Ruby) -- [gocover-cobertura](https://github.com/boumenot/gocover-cobertura) (Golang) - -Other coverage analysis frameworks support the format out of the box, for example: - -- [Istanbul](https://istanbul.js.org/docs/advanced/alternative-reporters/#cobertura) (JavaScript) -- [Coverage.py](https://coverage.readthedocs.io/en/coverage-5.0.4/cmd.html#xml-reporting) (Python) -- [PHPUnit](https://github.com/sebastianbergmann/phpunit-documentation-english/blob/master/src/textui.rst#command-line-options) (PHP) - -Once configured, if you create a merge request that triggers a pipeline which collects -coverage reports, the coverage is shown in the diff view. This includes reports -from any job in any stage in the pipeline. The coverage displays for each line: - -- `covered` (green): lines which have been checked at least once by tests -- `no test coverage` (orange): lines which are loaded but never executed -- no coverage information: lines which are non-instrumented or not loaded - -Hovering over the coverage bar provides further information, such as the number -of times the line was checked by tests. - -Uploading a test coverage report does not enable: - -- [Test coverage results in merge requests](../../../ci/pipelines/settings.md#merge-request-test-coverage-results). -- [Code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). - -You must configure these separately. - -### Limits - -A limit of 100 `<source>` nodes for Cobertura format XML files applies. If your Cobertura report exceeds -100 nodes, there can be mismatches or no matches in the merge request diff view. - -A single Cobertura XML file can be no more than 10MiB. For large projects, split the Cobertura XML into -smaller files. See [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/328772) for more details. -When submitting many files, it can take a few minutes for coverage to show on a merge request. - -The visualization only displays after the pipeline is complete. If the pipeline has -a [blocking manual job](../../../ci/jobs/job_control.md#types-of-manual-jobs), the -pipeline waits for the manual job before continuing and is not considered complete. -The visualization cannot be displayed if the blocking manual job did not run. - -### Artifact expiration - -By default, the [pipeline artifact](../../../ci/pipelines/pipeline_artifacts.md#storage) used -to draw the visualization on the merge request expires **one week** after creation. - -### Coverage report from child pipeline - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/363301) in GitLab 15.1 [with a flag](../../../administration/feature_flags.md). Disabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `ci_child_pipeline_coverage_reports`. -On GitLab.com, this feature is not available. -The feature is not ready for production use. - -If the test coverage is created in jobs that are in a child pipeline, the parent pipeline must use -`strategy: depend`. - -```yaml -child_test_pipeline: - trigger: - include: - - local: path/to/child_pipeline.yml - - template: Security/SAST.gitlab-ci.yml - strategy: depend -``` - -### Automatic class path correction - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217664) in GitLab 13.8. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284822) in GitLab 13.9. - -The coverage report properly matches changed files only if the `filename` of a `class` element -contains the full path relative to the project root. However, in some coverage analysis frameworks, -the generated Cobertura XML has the `filename` path relative to the class package directory instead. - -To make an intelligent guess on the project root relative `class` path, the Cobertura XML parser -attempts to build the full path by: - -- Extracting a portion of the `source` paths from the `sources` element and combining them with the - class `filename` path. -- Checking if the candidate path exists in the project. -- Using the first candidate that matches as the class full path. - -#### Path correction example - -As an example, a project with: - -- A full path of `test-org/test-project`. -- The following files relative to the project root: - - ```shell - Auth/User.cs - Lib/Utils/User.cs - src/main/java - ``` - -In the: - -- Cobertura XML, the `filename` attribute in the `class` element assumes the value is a relative - path to the project's root: - - ```xml - <class name="packet.name" filename="src/main/java" line-rate="0.0" branch-rate="0.0" complexity="5"> - ``` - -- `sources` from Cobertura XML, the following paths in the format - `<CI_BUILDS_DIR>/<PROJECT_FULL_PATH>/...`: - - ```xml - <sources> - <source>/builds/test-org/test-project/Auth</source> - <source>/builds/test-org/test-project/Lib/Utils</source> - </sources> - ``` - -The parser: - -- Extracts `Auth` and `Lib/Utils` from the `sources` and uses these to determine the `class` path - relative to the project root. -- Combines these extracted `sources` and the class filename. For example, if there is a `class` - element with the `filename` value of `User.cs`, the parser takes the first candidate path that - matches, which is `Auth/User.cs`. -- For each `class` element, attempts to look for a match for each extracted `source` path up to - 100 iterations. If it reaches this limit without finding a matching path in the file tree, the - class is not included in the final coverage report. - -NOTE: -Automatic class path correction only works on `source` paths in the format `<CI_BUILDS_DIR>/<PROJECT_FULL_PATH>/...`. -The `source` is ignored if the path does not follow this pattern. The parser assumes that the -`filename` of a `class` element contains the full path relative to the project root. - -## Example test coverage configurations - -This section provides test coverage configuration examples for different programming languages. You can also see a working example in -the [`coverage-report`](https://gitlab.com/gitlab-org/ci-sample-projects/coverage-report/) demonstration project. - -### JavaScript example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example uses [Mocha](https://mochajs.org/) -JavaScript testing and [nyc](https://github.com/istanbuljs/nyc) coverage-tooling to -generate the coverage artifact: - -```yaml -test: - script: - - npm install - - npx nyc --reporter cobertura mocha - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: coverage/cobertura-coverage.xml -``` - -### Java and Kotlin examples - -#### Maven example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for Java or Kotlin uses [Maven](https://maven.apache.org/) -to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to -generate the coverage artifact. -You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. - -GitLab expects the artifact in the Cobertura format, so you have to execute a few -scripts before uploading it. The `test-jdk11` job tests the code and generates an -XML artifact. The `coverage-jdk-11` job converts the artifact into a Cobertura report: - -```yaml -test-jdk11: - stage: test - image: maven:3.6.3-jdk-11 - script: - - mvn $MAVEN_CLI_OPTS clean org.jacoco:jacoco-maven-plugin:prepare-agent test jacoco:report - artifacts: - paths: - - target/site/jacoco/jacoco.xml - -coverage-jdk11: - # Must be in a stage later than test-jdk11's stage. - # The `visualize` stage does not exist by default. - # Please define it first, or choose an existing stage like `deploy`. - stage: visualize - image: registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 - script: - # convert report from jacoco to cobertura, using relative project path - - python /opt/cover2cover.py target/site/jacoco/jacoco.xml $CI_PROJECT_DIR/src/main/java/ > target/site/cobertura.xml - needs: ["test-jdk11"] - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: target/site/cobertura.xml -``` - -#### Gradle example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for Java or Kotlin uses [Gradle](https://gradle.org/) -to build the project and [JaCoCo](https://www.eclemma.org/jacoco/) coverage-tooling to -generate the coverage artifact. -You can check the [Docker image configuration and scripts](https://gitlab.com/haynes/jacoco2cobertura) if you want to build your own image. - -GitLab expects the artifact in the Cobertura format, so you have to execute a few -scripts before uploading it. The `test-jdk11` job tests the code and generates an -XML artifact. The `coverage-jdk-11` job converts the artifact into a Cobertura report: - -```yaml -test-jdk11: - stage: test - image: gradle:6.6.1-jdk11 - script: - - 'gradle test jacocoTestReport' # jacoco must be configured to create an xml report - artifacts: - paths: - - build/jacoco/jacoco.xml - -coverage-jdk11: - # Must be in a stage later than test-jdk11's stage. - # The `visualize` stage does not exist by default. - # Please define it first, or chose an existing stage like `deploy`. - stage: visualize - image: registry.gitlab.com/haynes/jacoco2cobertura:1.0.7 - script: - # convert report from jacoco to cobertura, using relative project path - - python /opt/cover2cover.py build/jacoco/jacoco.xml $CI_PROJECT_DIR/src/main/java/ > build/cobertura.xml - needs: ["test-jdk11"] - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: build/cobertura.xml -``` - -### Python example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for Python uses [pytest-cov](https://pytest-cov.readthedocs.io/) to collect test coverage data and [coverage.py](https://coverage.readthedocs.io/) to convert the report to use full relative paths. -The information isn't displayed without the conversion. - -This example assumes that the code for your package is in `src/` and your tests are in `tests.py`: - -```yaml -run tests: - stage: test - image: python:3 - script: - - pip install pytest pytest-cov - - coverage run -m pytest - - coverage report - - coverage xml - coverage: '/(?i)total.*? (100(?:\.0+)?\%|[1-9]?\d(?:\.\d+)?\%)$/' - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: coverage.xml -``` - -### PHP example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for PHP uses [PHPUnit](https://phpunit.readthedocs.io/) -to collect test coverage data and generate the report. - -With a minimal [`phpunit.xml`](https://phpunit.readthedocs.io/en/9.5/configuration.html) file (you may reference -[this example repository](https://gitlab.com/yookoala/code-coverage-visualization-with-php/)), you can run the test and -generate the `coverage.xml`: - -```yaml -run tests: - stage: test - image: php:latest - variables: - XDEBUG_MODE: coverage - before_script: - - apt-get update && apt-get -yq install git unzip zip libzip-dev zlib1g-dev - - docker-php-ext-install zip - - pecl install xdebug && docker-php-ext-enable xdebug - - php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" - - php composer-setup.php --install-dir=/usr/local/bin --filename=composer - - composer install - - composer require --dev phpunit/phpunit phpunit/php-code-coverage - script: - - php ./vendor/bin/phpunit --coverage-text --coverage-cobertura=coverage.cobertura.xml - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: coverage.cobertura.xml -``` - -[Codeception](https://codeception.com/), through PHPUnit, also supports generating Cobertura report with -[`run`](https://codeception.com/docs/reference/Commands#run). The path for the generated file -depends on the `--coverage-cobertura` option and [`paths`](https://codeception.com/docs/reference/Configuration#paths) -configuration for the [unit test suite](https://codeception.com/docs/05-UnitTests). Configure `.gitlab-ci.yml` -to find Cobertura in the appropriate path. - -### C/C++ example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for C/C++ with -`gcc` or `g++` as the compiler uses [`gcovr`](https://gcovr.com/en/stable/) to generate the coverage -output file in Cobertura XML format. - -This example assumes: - -- That the `Makefile` is created by `cmake` in the `build` directory, - within another job in a previous stage. - (If you use `automake` to generate the `Makefile`, - then you need to call `make check` instead of `make test`.) -- `cmake` (or `automake`) has set the compiler option `--coverage`. - -```yaml -run tests: - stage: test - script: - - cd build - - make test - - gcovr --xml-pretty --exclude-unreachable-branches --print-summary -o coverage.xml --root ${CI_PROJECT_DIR} - coverage: /^\s*lines:\s*\d+.\d+\%/ - artifacts: - name: ${CI_JOB_NAME}-${CI_COMMIT_REF_NAME}-${CI_COMMIT_SHA} - expire_in: 2 days - reports: - coverage_report: - coverage_format: cobertura - path: build/coverage.xml -``` - -### Go example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for Go uses: - -- [`go test`](https://go.dev/doc/tutorial/add-a-test) to run tests. -- [`gocover-cobertura`](https://github.com/boumenot/gocover-cobertura) to convert Go's coverage profile into the Cobertura XML format. - -This example assumes that [Go modules](https://go.dev/ref/mod) -are being used. Please note that the `-covermode count` option does not work with the `-race` flag. -If you want to generate code coverage while also using the `-race` flag, you must switch to -`-covermode atomic` which is slower than `-covermode count`. See [this blog post](https://go.dev/blog/cover) -for more details. - -```yaml -run tests: - stage: test - image: golang:1.17 - script: - - go install - - go test ./... -coverprofile=coverage.txt -covermode count - - go get github.com/boumenot/gocover-cobertura - - go run github.com/boumenot/gocover-cobertura < coverage.txt > coverage.xml - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: coverage.xml -``` - -### Ruby example - -The following [`.gitlab-ci.yml`](../../../ci/yaml/index.md) example for Ruby uses - -- [`rspec`](https://rspec.info/) to run tests. -- [`simplecov`](https://github.com/simplecov-ruby/simplecov) and [`simplecov-cobertura`](https://github.com/dashingrocket/simplecov-cobertura) - to record the coverage profile and create a report in the Cobertura XML format. - -This example assumes: - -- That [`bundler`](https://bundler.io/) is being used for dependency management. - The `rspec`, `simplecov` and `simplecov-cobertura` gems have been added to your `Gemfile`. -- The `CoberturaFormatter` has been added to your `SimpleCov.formatters` - configuration within the `spec_helper.rb` file. - -```yaml -run tests: - stage: test - image: ruby:3.1 - script: - - bundle install - - bundle exec rspec - artifacts: - reports: - coverage_report: - coverage_format: cobertura - path: coverage/coverage.xml -``` - -## Troubleshooting - -### Test coverage visualization not displayed - -If the test coverage visualization is not displayed in the diff view, you can check -the coverage report itself and verify that: - -- The file you are viewing in the diff view is mentioned in the coverage report. -- The `source` and `filename` nodes in the report follows the [expected structure](#automatic-class-path-correction) - to match the files in your repository. - -Report artifacts are not downloadable by default. If you want the report to be downloadable -from the job details page, add your coverage report to the artifact `paths`: - -```yaml -artifacts: - paths: - - coverage/cobertura-coverage.xml - reports: - coverage_report: - coverage_format: cobertura - path: coverage/cobertura-coverage.xml -``` +<!-- This redirect file can be deleted after <2022-09-22>. --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> |