diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-08-24 15:12:07 +0300 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-08-24 15:12:07 +0300 |
commit | 263baf70a1f64bb773bfb57d74516a008c2bc7e4 (patch) | |
tree | 3fd07a2e7bccc2f6d19a1423c322240bd9808234 | |
parent | 0086677f7cad8c0d7e73d8584ce317f1fce5534e (diff) |
Add latest changes from gitlab-org/gitlab@master
33 files changed, 452 insertions, 219 deletions
diff --git a/app/controllers/concerns/issuable_actions.rb b/app/controllers/concerns/issuable_actions.rb index f1d80e37674..7c3401a7e90 100644 --- a/app/controllers/concerns/issuable_actions.rb +++ b/app/controllers/concerns/issuable_actions.rb @@ -193,7 +193,10 @@ module IssuableActions end def render_cached_discussions(discussions, serializer, cache_context) - render_cached(discussions, with: serializer, cache_context: -> (_) { cache_context }, context: self) + render_cached(discussions, + with: serializer, + cache_context: -> (_) { cache_context }, + context: self) end def paginated_discussions diff --git a/config/feature_flags/development/add_timing_to_certain_cache_actions.yml b/config/feature_flags/development/add_timing_to_certain_cache_actions.yml new file mode 100644 index 00000000000..c03e49dae8d --- /dev/null +++ b/config/feature_flags/development/add_timing_to_certain_cache_actions.yml @@ -0,0 +1,8 @@ +--- +name: add_timing_to_certain_cache_actions +introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/94966 +rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/371657 +milestone: '15.4' +type: development +group: group::code review +default_enabled: false diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 46793a7b8d8..9618ec0140b 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -146,6 +146,8 @@ The following metrics are available: | `gitlab_snowplow_failed_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission failures | | | `gitlab_snowplow_successful_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission successes | | | `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` | +| `gitlab_presentable_object_cacheless_render_real_duration_seconds` | Histogram | 15.3 | Duration of real time spent caching and representing specific web request objects | `controller`, `action` | +| `cached_object_operations_total` | Counter | 15.3 | Total number of objects cached for specific web requests | `controller`, `action` | ## Metrics controlled by a feature flag diff --git a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md index 868dae4fc6c..dce9883be4b 100644 --- a/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md +++ b/doc/architecture/blueprints/ci_data_decay/pipeline_partitioning.md @@ -201,6 +201,27 @@ find this number, though we might not need to do this. The single and uniform `partition_id` value for pipeline data gives us more choices later on than primary-keys-based partitioning. +## Altering partitioned tables + +It will still be possible to run `ALTER TABLE` statements against partitioned tables, +similarly to how the tables behaved before partitioning. When PostgreSQL runs +an `ALTER TABLE` statement against a parent partitioned table, it acquires the same +lock on all child partitions and updates each to keep them in sync. This differs from +running `ALTER TABLE` on a non-partitioned table in a few key ways: + +- PostgreSQL acquires `ACCESS EXCLUSIVE` locks against a larger number of tables, but + not a larger amount of data, than it would were the table not partitioned. + Each partition will be locked similarly to the parent table, and all will be updated + in a single transaction. +- Lock duration will be increased based on the number of partitions involved. + All `ALTER TABLE` statements executed on the GitLab database (other than `VALIDATE CONSTRAINT`) + take small constant amounts of time per table modified. PostgreSQL will need + to modify each partition in sequence, increasing the runtime of the lock. This + time will still remain very small until there are many partitions involved. +- If thousands of partitions are involved in an `ALTER TABLE`, we will need to verify that + the value of `max_locks_per_transaction` is high enough to support all of the locks that + need to be taken during the operation. + ## Splitting large partitions into smaller ones We want to start with the initial `pipeline_id` number `100` (or higher, like diff --git a/doc/ci/pipelines/downstream_pipelines.md b/doc/ci/pipelines/downstream_pipelines.md index 224b171625f..d92a632fce3 100644 --- a/doc/ci/pipelines/downstream_pipelines.md +++ b/doc/ci/pipelines/downstream_pipelines.md @@ -51,15 +51,162 @@ Multi-project pipelines: In the [pipeline graph view](index.md#view-full-pipeline-graph), downstream pipelines display as a list of cards on the right of the graph. -### Cancel or retry downstream pipelines from the graph view +### Retry a downstream pipeline -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. -> - [Generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. -To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**) -on the pipeline's card. +To retry a completed downstream pipeline, select **Retry** (**{retry}**): -To retry a failed downstream pipeline, select **Retry** (**{retry}**) -on the pipeline's card. +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). -![downstream pipeline actions](img/downstream_pipeline_actions.png) +### Cancel a downstream pipeline + +> - Retry from graph view [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/354974) in GitLab 15.0 [with a flag](../../administration/feature_flags.md) named `downstream_retry_action`. Disabled by default. +> - Retry from graph view [generally available and feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/357406) in GitLab 15.1. + +To cancel a downstream pipeline that is still running, select **Cancel** (**{cancel}**): + +- From the downstream pipeline's details page. +- On the pipeline's card in the [pipeline graph view](index.md#view-full-pipeline-graph). + +### Mirror the status of a downstream pipeline in the trigger job + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. + +You can mirror the pipeline status from the triggered pipeline to the source trigger job +by using [`strategy: depend`](../yaml/index.md#triggerstrategy). For example: + +```yaml +trigger_job: + trigger: + project: my/project + strategy: depend +``` + +## Pass CI/CD variables to a downstream pipeline + +You can pass CI/CD variables to a downstream pipeline with a few different methods, +based on where the variable is created or defined. + +### Pass YAML-defined CI/CD variables + +You can use the `variables` keyword to pass CI/CD variables to a downstream pipeline, +just like you would for any other job. + +For example, in a [multi-project pipeline](multi_project_pipelines.md): + +```yaml +rspec: + stage: test + script: bundle exec rspec + +staging: + variables: + ENVIRONMENT: staging + stage: deploy + trigger: my/deployment +``` + +The `ENVIRONMENT` variable is passed to every job defined in a downstream +pipeline. It is available as a variable when GitLab Runner picks a job. + +In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline +that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` +job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. + +```yaml +variables: + MY_VARIABLE: my-value + +trigger-downstream: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +### Prevent global variables from being passed + +You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). +For example, in a [multi-project pipeline](multi_project_pipelines.md): + +```yaml +variables: + MY_GLOBAL_VAR: value + +trigger-downstream: + inherit: + variables: false + variables: + MY_LOCAL_VAR: value + trigger: my/project +``` + +In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline. + +### Pass a predefined variable + +You might want to pass some information about the upstream pipeline using predefined variables. +To do that, you can use interpolation to pass any variable. For example, +in a [multi-project pipeline](multi_project_pipelines.md): + +```yaml +downstream-job: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's +`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the +context of all downstream builds. + +You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) +to a downstream pipeline, as they are not available in trigger jobs. + +Upstream pipelines take precedence over downstream ones. If there are two +variables with the same name defined in both upstream and downstream projects, +the ones defined in the upstream project take precedence. + +### Pass dotenv variables created in a job **(PREMIUM)** + +You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) +and [`needs:project`](../yaml/index.md#needsproject). + +For example, in a [multi-project pipeline](multi_project_pipelines.md): + +1. Save the variables in a `.env` file. +1. Save the `.env` file as a `dotenv` report. +1. Trigger the downstream pipeline. + + ```yaml + build_vars: + stage: build + script: + - echo "BUILD_VERSION=hello" >> build.env + artifacts: + reports: + dotenv: build.env + + deploy: + stage: deploy + trigger: my/downstream_project + ``` + +1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` + job in the upstream project with `needs`. The `test` job inherits the variables in the + `dotenv` report and it can access `BUILD_VERSION` in the script: + + ```yaml + test: + stage: test + script: + - echo $BUILD_VERSION + needs: + - project: my/upstream_project + job: build_vars + ref: master + artifacts: true + ``` diff --git a/doc/ci/pipelines/img/downstream_pipeline_actions.png b/doc/ci/pipelines/img/downstream_pipeline_actions.png Binary files differdeleted file mode 100644 index 4c4384bab57..00000000000 --- a/doc/ci/pipelines/img/downstream_pipeline_actions.png +++ /dev/null diff --git a/doc/ci/pipelines/multi_project_pipelines.md b/doc/ci/pipelines/multi_project_pipelines.md index e7d5e3e102a..3fb720cfb81 100644 --- a/doc/ci/pipelines/multi_project_pipelines.md +++ b/doc/ci/pipelines/multi_project_pipelines.md @@ -10,7 +10,7 @@ type: reference > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. You can set up [GitLab CI/CD](../index.md) across multiple projects, so that a pipeline -in one project can trigger a pipeline in another project. You can visualize the entire pipeline +in one project can trigger a [downstream](downstream_pipelines.md) pipeline in another project. You can visualize the entire pipeline in one place, including all cross-project interdependencies. For example, you might deploy your web application from three different projects in GitLab. @@ -69,7 +69,7 @@ GitLab then creates a downstream pipeline in the `staging` job succeeds. The full path to the project is `my/deployment`. You can view the status for the pipeline, or you can display -[the downstream pipeline's status instead](#mirror-status-of-a-triggered-pipeline-in-the-trigger-job). +[the downstream pipeline's status instead](downstream_pipelines.md#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job). The user that creates the upstream pipeline must be able to create pipelines in the downstream project (`my/deployment`) too. If the downstream project is not found, @@ -124,117 +124,6 @@ of the user that ran the trigger job in the upstream project. If the user does n have permission to run CI/CD pipelines against the protected branch, the pipeline fails. See [pipeline security for protected branches](index.md#pipeline-security-on-protected-branches). -#### Pass CI/CD variables to a downstream pipeline by using the `variables` keyword - -Sometimes you might want to pass CI/CD variables to a downstream pipeline. -You can do that by using the `variables` keyword, just like you would for any other job. - -```yaml -rspec: - stage: test - script: bundle exec rspec - -staging: - variables: - ENVIRONMENT: staging - stage: deploy - trigger: my/deployment -``` - -The `ENVIRONMENT` variable is passed to every job defined in a downstream -pipeline. It is available as a variable when GitLab Runner picks a job. - -In the following configuration, the `MY_VARIABLE` variable is passed to the downstream pipeline -that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` -job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. - -```yaml -variables: - MY_VARIABLE: my-value - -trigger-downstream: - variables: - ENVIRONMENT: something - trigger: my/project -``` - -You can stop global variables from reaching the downstream pipeline by using the [`inherit:variables` keyword](../yaml/index.md#inheritvariables). -In this example, the `MY_GLOBAL_VAR` variable is not available in the triggered pipeline: - -```yaml -variables: - MY_GLOBAL_VAR: value - -trigger-downstream: - inherit: - variables: false - variables: - MY_LOCAL_VAR: value - trigger: my/project -``` - -You might want to pass some information about the upstream pipeline using, for -example, predefined variables. In order to do that, you can use interpolation -to pass any variable. For example: - -```yaml -downstream-job: - variables: - UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME - trigger: my/project -``` - -In this scenario, the `UPSTREAM_BRANCH` variable with the value of the upstream pipeline's -`$CI_COMMIT_REF_NAME` is passed to `downstream-job`. It is available in the -context of all downstream builds. - -You cannot use this method to forward [job-level persisted variables](../variables/where_variables_can_be_used.md#persisted-variables) -to a downstream pipeline, as they are not available in trigger jobs. - -Upstream pipelines take precedence over downstream ones. If there are two -variables with the same name defined in both upstream and downstream projects, -the ones defined in the upstream project take precedence. - -#### Pass CI/CD variables to a downstream pipeline by using variable inheritance **(PREMIUM)** - -You can pass variables to a downstream pipeline with [`dotenv` variable inheritance](../variables/index.md#pass-an-environment-variable-to-another-job) and [`needs:project`](../yaml/index.md#needsproject). - -In the upstream pipeline: - -1. Save the variables in a `.env` file. -1. Save the `.env` file as a `dotenv` report. -1. Trigger the downstream pipeline. - - ```yaml - build_vars: - stage: build - script: - - echo "BUILD_VERSION=hello" >> build.env - artifacts: - reports: - dotenv: build.env - - deploy: - stage: deploy - trigger: my/downstream_project - ``` - -1. Set the `test` job in the downstream pipeline to inherit the variables from the `build_vars` - job in the upstream project with `needs`. The `test` job inherits the variables in the - `dotenv` report and it can access `BUILD_VERSION` in the script: - - ```yaml - test: - stage: test - script: - - echo $BUILD_VERSION - needs: - - project: my/upstream_project - job: build_vars - ref: master - artifacts: true - ``` - #### Pass artifacts to a downstream pipeline You can pass artifacts to a downstream pipeline by using [`needs:project`](../yaml/index.md#needsproject). @@ -287,7 +176,7 @@ pass this variable to the downstream pipeline using variable inheritance: 1. In a job in the upstream pipeline, save the artifacts using the [`artifacts`](../yaml/index.md#artifacts) keyword. 1. In the job that triggers the downstream pipeline, pass the `$CI_MERGE_REQUEST_REF_PATH` variable by using - [variable inheritance](#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword): + [variable inheritance](downstream_pipelines.md#pass-yaml-defined-cicd-variables): ```yaml build_artifacts: @@ -336,21 +225,6 @@ is `pipeline` for all its jobs. If you use [`only/except`](../yaml/index.md#only--except) to control job behavior, use the [`pipelines`](../yaml/index.md#onlyrefs--exceptrefs) keyword. -#### Mirror status of a triggered pipeline in the trigger job - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/11238) in GitLab Premium 12.3. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. - -You can mirror the pipeline status from the triggered pipeline to the source trigger job -by using [`strategy: depend`](../yaml/index.md#triggerstrategy). For example: - -```yaml -trigger_job: - trigger: - project: my/project - strategy: depend -``` - ### Create multi-project pipelines by using the API > [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/31573) to GitLab Free in 12.4. @@ -370,7 +244,7 @@ When using: - [`only/except`](../yaml/index.md#only--except) to control job behavior, use the `pipelines` keyword. -## Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** +### Trigger a pipeline when an upstream project is rebuilt **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9045) in GitLab 12.8. @@ -409,11 +283,3 @@ In [pipeline mini graphs](index.md#pipeline-mini-graphs), the downstream pipelin displays to the right of the mini graph. ![Multi-project pipeline mini graph](img/pipeline_mini_graph_v15_0.png) - -## Retry or cancel multi-project pipelines - -If you have permission to trigger pipelines in the downstream project, you can -retry or cancel multi-project pipelines: - -- [In the main graph view](downstream_pipelines.md#view-a-downstream-pipeline). -- From the downstream pipeline's details page. diff --git a/doc/ci/pipelines/parent_child_pipelines.md b/doc/ci/pipelines/parent_child_pipelines.md index 2d36aaac1f7..ede90178a35 100644 --- a/doc/ci/pipelines/parent_child_pipelines.md +++ b/doc/ci/pipelines/parent_child_pipelines.md @@ -26,7 +26,7 @@ YAML is dynamically generated. ![Parent pipeline graph expanded](img/parent_pipeline_graph_expanded_v14_3.png) Similarly to [multi-project pipelines](multi_project_pipelines.md), a pipeline can trigger a -set of concurrently running child pipelines, but within the same project: +set of concurrently running [downstream](downstream_pipelines.md) child pipelines, but in the same project: - Child pipelines still execute each of their jobs according to a stage sequence, but would be free to continue forward through their stages without waiting for unrelated @@ -89,8 +89,9 @@ microservice_a: The maximum number of entries that are accepted for `trigger:include` is three. -Similar to [multi-project pipelines](multi_project_pipelines.md#mirror-status-of-a-triggered-pipeline-in-the-trigger-job), -we can set the parent pipeline to depend on the status of the child pipeline upon completion: +Similar to [multi-project pipelines](multi_project_pipelines.md), we can set the +parent pipeline to [depend on the status](downstream_pipelines.md#mirror-the-status-of-a-downstream-pipeline-in-the-trigger-job) +of the child pipeline upon completion: ```yaml microservice_a: @@ -214,15 +215,7 @@ For an overview, see [Nested Dynamic Pipelines](https://youtu.be/C5j3ju9je2M). ## Pass CI/CD variables to a child pipeline -You can pass CI/CD variables to a downstream pipeline using the same methods as -multi-project pipelines: +You can pass variables to a downstream pipeline: -- [By using the `variable` keyword](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword). -- [By using variable inheritance](multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). - -## Retry or cancel child pipelines - -You can retry or cancel child pipelines: - -- [In the main graph view](downstream_pipelines.md#view-a-downstream-pipeline). -- In the child pipeline's details page. +- [By using the `variables` keyword](downstream_pipelines.md#pass-yaml-defined-cicd-variables). +- [By using dotenv variable inheritance](downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index 72df8d56815..e9d4fd8549c 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -647,7 +647,7 @@ deploy_five: artifacts: false ``` -[Multi-project pipelines](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance) +[Multi-project pipelines](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job) can also inherit variables from their upstream pipelines. ## CI/CD variable precedence @@ -695,8 +695,8 @@ You can override the value of a variable when you: 1. Run a job manually in the UI. 1. Use [push options](../../user/project/push_options.md#push-options-for-gitlab-cicd). 1. Trigger a pipeline by using [the API](../triggers/index.md#pass-cicd-variables-in-the-api-call). -1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-the-variables-keyword) - or [by using variable inheritance](../pipelines/multi_project_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline-by-using-variable-inheritance). +1. Pass variables to a downstream pipeline [by using the `variable` keyword](../pipelines/downstream_pipelines.md#pass-cicd-variables-to-a-downstream-pipeline) + or [by using variable inheritance](../pipelines/downstream_pipelines.md#pass-dotenv-variables-created-in-a-job). The pipeline variables declared in these events take [priority over other variables](#cicd-variable-precedence). diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index bbd27a3f070..32600a73c0a 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -3884,7 +3884,7 @@ test: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8997) in GitLab Premium 11.8. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/199224) to GitLab Free in 12.8. -Use `trigger` to start a downstream pipeline that is either: +Use `trigger` to start a [downstream pipeline](../pipelines/downstream_pipelines.md) that is either: - [A multi-project pipeline](../pipelines/multi_project_pipelines.md). - [A child pipeline](../pipelines/parent_child_pipelines.md). diff --git a/doc/development/application_secrets.md b/doc/development/application_secrets.md index 06a38eb238a..93e43856b34 100644 --- a/doc/development/application_secrets.md +++ b/doc/development/application_secrets.md @@ -17,6 +17,7 @@ This page is a development guide for application secrets. |`db_key_base` | The base key to encrypt the data for `attr_encrypted` columns | |`openid_connect_signing_key` | The signing key for OpenID Connect | | `encrypted_settings_key_base` | The base key to encrypt settings files with | +| `ci_jwt_signing_key` | The base key for encrypting the `CI_JOB_JWT` and `CI_JOB_JWT_V2` predefined CI/CD variables | ## Where the secrets are stored diff --git a/doc/development/fe_guide/storybook.md b/doc/development/fe_guide/storybook.md index 45342eb6d72..64bba567123 100644 --- a/doc/development/fe_guide/storybook.md +++ b/doc/development/fe_guide/storybook.md @@ -46,10 +46,12 @@ To add a story: 1. Write the story as per the [official Storybook instructions](https://storybook.js.org/docs/vue/writing-stories/introduction/) - Notes: - - Specify the `title` field of the story as the component's file path from the `javascripts/` directory. - - For example, if the component is located at `app/assets/javascripts/vue_shared/components/sidebar/todo_toggle/todo_button.vue`, specify the story `title` as `vue_shared/components/sidebar/todo_toggle/todo_button`. This will ensure the Storybook navigation maps closely to our internal directory structure. + NOTE: + Specify the `title` field of the story as the component's file path from the `javascripts/` directory. + For example, if the component is located at `app/assets/javascripts/vue_shared/components/sidebar/todo_toggle/todo_button.vue`, + specify the story `title` as `vue_shared/components/sidebar/todo_toggle/todo_button`. + If the component is located in the `ee/` directory, make sure to prefix the story's title with `ee/` as well. + This will ensure the Storybook navigation maps closely to our internal directory structure. ## Mock backend APIs diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index d35a70c6910..fad78c6b2dc 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -85,15 +85,12 @@ SILENCE_DEPRECATIONS=1 bin/rspec spec/models/project_spec.rb ### Test order -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137) in GitLab 15.3. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137) in GitLab 15.4. All new spec files are run in [random order](https://gitlab.com/gitlab-org/gitlab/-/issues/337399) to surface flaky tests that are dependent on test order. -When randomized: - -- We append the `(order random)` to example group description. -- The used seed is shown in the spec output below the test suite summary. For example, `Randomized with seed 27443`. +When randomized the used seed is shown in the spec output below the test suite summary. For example, `Randomized with seed 27443`. For a list of spec files which are still run in defined order, see [`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml). diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 154884c16e7..0297dc161e3 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -376,4 +376,4 @@ corpus file extracts into a folder named `corpus`. If you see this error message when running the fuzzing job with `COVFUZZ_USE_REGISTRY` set to `true`, ensure that duplicates are allowed. For more details, see -[duplicate Generic packages](../../packages/generic_packages/#do-not-allow-duplicate-generic-packages). +[duplicate Generic packages](../../packages/generic_packages/index.md#do-not-allow-duplicate-generic-packages). diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 3aa080d9c4f..bad8e1934fd 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -50,12 +50,12 @@ possible, we encourage you to use all of our security scanning tools: declared software dependencies (and those installed as a sub-dependency). Dependency Scanning can not detect software dependencies that are pre-bundled into the container's base image. To identify pre-bundled dependencies, enable - [Container Scanning](../container_scanning/) language scanning using the - [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/#report-language-specific-findings). -- [Container Scanning](../container_scanning/) analyzes your containers and tells + [Container Scanning](../container_scanning/index.md) language scanning using the + [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). +- [Container Scanning](../container_scanning/index.md) analyzes your containers and tells you about known risks in the operating system's (OS) packages. You can configure it to also report on software and language dependencies, if you enable it and use - the [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/#report-language-specific-findings). + the [`CS_DISABLE_LANGUAGE_VULNERABILITY_SCAN` variable](../container_scanning/index.md#report-language-specific-findings). Turning this variable on can result in some duplicate findings, as we do not yet de-duplicate results between Container Scanning and Dependency Scanning. For more details, efforts to de-duplicate these findings can be tracked in @@ -627,7 +627,7 @@ The following variables are used for configuring specific analyzers (used for a The previous tables are not an exhaustive list of all variables that can be used. They contain all specific GitLab and analyzer variables we support and test. There are many variables, such as environment variables, that you can pass in and they will work. This is a large list, many of which we may be unaware of, and as such is not documented. For example, to pass the non-GitLab environment variable `HTTPS_PROXY` to all Dependency Scanning jobs, -set it as a [custom CI/CD variable in your `.gitlab-ci.yml`](../../../ci/variables/#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file) +set it as a [custom CI/CD variable in your `.gitlab-ci.yml`](../../../ci/variables/index.md#create-a-custom-cicd-variable-in-the-gitlab-ciyml-file) file like this: ```yaml diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index cd405b1192a..57acd5b8150 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -98,7 +98,7 @@ approach. ## Security scanning with Auto DevOps To enable all GitLab Security scanning tools, with default settings, enable -[Auto DevOps](../../topics/autodevops/): +[Auto DevOps](../../topics/autodevops/index.md): - [Auto SAST](../../topics/autodevops/stages.md#auto-sast) - [Auto Secret Detection](../../topics/autodevops/stages.md#auto-secret-detection) @@ -405,7 +405,7 @@ You can interact with the results of the security scanning tools in several loca - [Project Security Dashboard](security_dashboard/index.md) - [Security pipeline tab](security_dashboard/index.md) - [Group Security Dashboard](security_dashboard/index.md) -- [Security Center](security_dashboard/#security-center) +- [Security Center](security_dashboard/index.md#security-center) - [Vulnerability Report](vulnerability_report/index.md) - [Vulnerability Pages](vulnerabilities/index.md) - [Dependency List](dependency_list/index.md) @@ -446,7 +446,7 @@ Security and compliance teams must ensure that security scans: GitLab provides two methods of accomplishing this, each with advantages and disadvantages. -- [Compliance framework pipelines](../project/settings/#compliance-pipeline-configuration) +- [Compliance framework pipelines](../project/settings/index.md#compliance-pipeline-configuration) are recommended when: - Scan execution enforcement is required for any scanner that uses a GitLab template, such as SAST IaC, DAST, Dependency Scanning, diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 7aeb094093c..7344695886a 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -87,9 +87,9 @@ above. You can find more information at each of the pages below: - [Container scanning offline directions](../container_scanning/index.md#running-container-scanning-in-an-offline-environment) - [SAST offline directions](../sast/index.md#running-sast-in-an-offline-environment) -- [Secret Detection offline directions](../secret_detection/#running-secret-detection-in-an-offline-environment) +- [Secret Detection offline directions](../secret_detection/index.md#running-secret-detection-in-an-offline-environment) - [DAST offline directions](../dast/run_dast_offline.md#run-dast-in-an-offline-environment) -- [API Fuzzing offline directions](../api_fuzzing/#running-api-fuzzing-in-an-offline-environment) +- [API Fuzzing offline directions](../api_fuzzing/index.md#running-api-fuzzing-in-an-offline-environment) - [License Compliance offline directions](../../compliance/license_compliance/index.md#running-license-compliance-in-an-offline-environment) - [Dependency Scanning offline directions](../dependency_scanning/index.md#running-dependency-scanning-in-an-offline-environment) @@ -236,4 +236,4 @@ an offline environment. Note that these steps are specific to GitLab Secure with AutoDevOps. Using other stages with AutoDevOps may require other steps covered in the -[Auto DevOps documentation](../../../topics/autodevops/). +[Auto DevOps documentation](../../../topics/autodevops/index.md). diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index eabb4142c4c..c253b8a1092 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -14,10 +14,10 @@ with a long, random job name. In the unlikely event of a job name collision, the any pre-existing job in the pipeline. If a policy is created at the group-level, it will apply to every child project or sub-group. A group-level policy cannot be edited from a child project or sub-group. -This feature has some overlap with [compliance framework pipelines](../../project/settings/#compliance-pipeline-configuration), +This feature has some overlap with [compliance framework pipelines](../../project/settings/index.md#compliance-pipeline-configuration), as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on the similarities and differences between these features, see -[Enforce scan execution](../#enforce-scan-execution). +[Enforce scan execution](../index.md#enforce-scan-execution). NOTE: Policy jobs are created in the `test` stage of the pipeline. If you modify the default pipeline diff --git a/doc/user/application_security/sast/analyzers.md b/doc/user/application_security/sast/analyzers.md index 687d0725edc..f7282ec63b1 100644 --- a/doc/user/application_security/sast/analyzers.md +++ b/doc/user/application_security/sast/analyzers.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Moved](https://gitlab.com/groups/gitlab-org/-/epics/2098) from GitLab Ultimate to GitLab Free in 13.3. Static Application Security Testing (SAST) uses analyzers -to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/#scanner), a third-party code analysis tool. +to detect vulnerabilities in source code. Each analyzer is a wrapper around a [scanner](../terminology/index.md#scanner), a third-party code analysis tool. The analyzers are published as Docker images that SAST uses to launch dedicated containers for each analysis. @@ -20,7 +20,7 @@ For each scanner, an analyzer: - Exposes its detection logic. - Handles its execution. -- Converts its output to a [standard format](../terminology/#secure-report-format). +- Converts its output to a [standard format](../terminology/index.md#secure-report-format). ## SAST analyzers @@ -77,7 +77,7 @@ You can choose to disable the other analyzers early and use Semgrep-based scanni - You'll enjoy significantly faster scanning, reduced CI minutes usage, and more customizable scanning rules. - However, vulnerabilities previously reported by language-specific analyzers will be reported again under certain conditions, including if you've dismissed the vulnerabilities before. The system behavior depends on: - whether you've excluded the Semgrep-based analyzer from running in the past. - - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/). + - which analyzer first discovered the vulnerabilities shown in the project's [Vulnerability Report](../vulnerability_report/index.md). ### Vulnerability translation @@ -103,7 +103,7 @@ You can choose to use Semgrep-based scanning instead of language-specific analyz We recommend taking this approach if any of these cases applies: -- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/). +- You haven't used SAST before on a project, so you don't already have SAST vulnerabilities in your [Vulnerability Report](../vulnerability_report/index.md). - You're having trouble configuring one of the analyzers whose coverage overlaps with Semgrep-based coverage. For example, you might have trouble setting up the SpotBugs-based analyzer to compile your code. - You've already seen and dismissed vulnerabilities created by ESLint, Gosec, or Flawfinder scanning, and you've kept the re-created vulnerabilities created by Semgrep. diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 7d04d063b07..f91e6240841 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -337,7 +337,7 @@ False positive detection is available in a subset of the [supported languages](# > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5144) in GitLab 14.2. Source code is volatile; as developers make changes, source code may move within files or between files. -Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/). +Security analyzers may have already reported vulnerabilities that are being tracked in the [Vulnerability Report](../vulnerability_report/index.md). These vulnerabilities are linked to specific problematic code fragments so that they can be found and fixed. If the code fragments are not tracked reliably as they move, vulnerability management is harder because the same vulnerability could be reported again. diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 93c32f998fa..950a18b95fd 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -19,7 +19,7 @@ repository for secrets. All identified secrets are reported in the: - Merge request widget - Pipelines' **Security** tab -- [Security Dashboard](../security_dashboard/) +- [Security Dashboard](../security_dashboard/index.md) ![Secret Detection in merge request widget](img/secret_detection_v13_2.png) diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index ad397c3fe04..f5ad7d243dd 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -46,7 +46,7 @@ are reintroduced and detected by subsequent scans have a _new_ vulnerability rec existing vulnerability is no longer detected in a project's `default` branch, you should change its status to **Resolved**. This ensures that if it is accidentally reintroduced in a future merge, it is reported again as a new record. You can use the Vulnerability Report's -[Activity filter](../vulnerability_report/#activity-filter) to select all vulnerabilities that are +[Activity filter](../vulnerability_report/index.md#activity-filter) to select all vulnerabilities that are no longer detected, and change their status. ## Change status of a vulnerability diff --git a/doc/user/application_security/vulnerability_report/pipeline.md b/doc/user/application_security/vulnerability_report/pipeline.md index 32916f4c9c7..8d49c5adc51 100644 --- a/doc/user/application_security/vulnerability_report/pipeline.md +++ b/doc/user/application_security/vulnerability_report/pipeline.md @@ -26,7 +26,7 @@ For example, if a pipeline contains DAST and SAST jobs, but the DAST job fails b The pipeline vulnerability report only shows results contained in the security report artifacts. This report differs from the [Vulnerability Report](index.md), which contains cumulative results of all successful jobs, and from the merge request -[security widget](../#view-security-scan-information-in-merge-requests), which combines the branch results with +[security widget](../index.md#view-security-scan-information-in-merge-requests), which combines the branch results with cumulative results. Before GitLab displays results, the vulnerability findings in all pipeline reports are [deduplicated](#deduplication-process). @@ -35,7 +35,7 @@ Before GitLab displays results, the vulnerability findings in all pipeline repor **Scan details** shows a summary of vulnerability findings in the pipeline and the source reports. -GitLab displays one row of information for each [scan type](../terminology/#scan-type-report-type) artifact present in +GitLab displays one row of information for each [scan type](../terminology/index.md#scan-type-report-type) artifact present in the pipeline. Note that each scan type's total number of vulnerabilities includes dismissed findings. If the number of findings @@ -86,8 +86,8 @@ vulnerability finding is present in multiple reports. This duplication is common increase coverage. The deduplication process allows you to maximize the vulnerability scanning coverage while reducing the number of findings you need to manage. -A finding is considered a duplicate of another finding when their [scan type](../terminology/#scan-type-report-type), -[location](../terminology/#location-fingerprint) and +A finding is considered a duplicate of another finding when their [scan type](../terminology/index.md#scan-type-report-type), +[location](../terminology/index.md#location-fingerprint) and [identifiers](../../../development/integrations/secure.md#identifiers) are the same. The scan type must match because each can have its own definition for the location of a vulnerability. For example, diff --git a/lib/gitlab/audit/auditor.rb b/lib/gitlab/audit/auditor.rb index c96be19f02d..4a6e4e2e06e 100644 --- a/lib/gitlab/audit/auditor.rb +++ b/lib/gitlab/audit/auditor.rb @@ -117,7 +117,7 @@ module Gitlab # Only capture real users for successful authentication events. user: author_if_user, user_name: @author.name, - ip_address: @ip_address, + ip_address: Gitlab::RequestContext.instance.client_ip || @author.current_sign_in_ip, result: AuthenticationEvent.results[:success], provider: @authentication_provider } diff --git a/lib/gitlab/cache/helpers.rb b/lib/gitlab/cache/helpers.rb index 7b11d6bc9ff..dc410ec4f34 100644 --- a/lib/gitlab/cache/helpers.rb +++ b/lib/gitlab/cache/helpers.rb @@ -57,9 +57,19 @@ module Gitlab # @param expires_in [ActiveSupport::Duration, Integer] an expiry time for the cache entry # @return [String] def cached_object(object, presenter:, presenter_args:, context:, expires_in:) - cache.fetch(contextual_cache_key(presenter, object, context), expires_in: expires_in) do - Gitlab::Json.dump(presenter.represent(object, **presenter_args).as_json) + misses = 0 + + json = cache.fetch(contextual_cache_key(presenter, object, context), expires_in: expires_in) do + time_action(render_type: :object) do + misses += 1 + + Gitlab::Json.dump(presenter.represent(object, **presenter_args).as_json) + end end + + increment_cache_metric(render_type: :object, total_count: 1, miss_count: misses) + + json end # Used for fetching or rendering multiple objects @@ -71,10 +81,19 @@ module Gitlab # @param expires_in [ActiveSupport::Duration, Integer] an expiry time for the cache entry # @return [Array<String>] def cached_collection(collection, presenter:, presenter_args:, context:, expires_in:) + total_count = collection.size + misses = 0 + json = fetch_multi(presenter, collection, context: context, expires_in: expires_in) do |obj| - Gitlab::Json.dump(presenter.represent(obj, **presenter_args).as_json) + time_action(render_type: :collection) do + misses += 1 + + Gitlab::Json.dump(presenter.represent(obj, **presenter_args).as_json) + end end + increment_cache_metric(render_type: :collection, total_count: total_count, miss_count: misses) + json.values end @@ -106,6 +125,57 @@ module Gitlab contextual_cache_key(presenter, object, context) end end + + def increment_cache_metric(render_type:, total_count:, miss_count:) + return unless Feature.enabled?(:add_timing_to_certain_cache_actions) + return unless caller_id + + metric_name = :cached_object_operations_total + hit_count = total_count - miss_count + + current_transaction&.increment( + metric_name, + hit_count, + { caller_id: caller_id, render_type: render_type, cache_hit: true } + ) + + current_transaction&.increment( + metric_name, + miss_count, + { caller_id: caller_id, render_type: render_type, cache_hit: false } + ) + end + + def time_action(render_type:, &block) + if Feature.enabled?(:add_timing_to_certain_cache_actions) + real_start = Gitlab::Metrics::System.monotonic_time + + presented_object = yield + + real_duration_histogram(render_type).observe({}, Gitlab::Metrics::System.monotonic_time - real_start) + + presented_object + else + yield + end + end + + def real_duration_histogram(render_type) + Gitlab::Metrics.histogram( + :gitlab_presentable_object_cacheless_render_real_duration_seconds, + 'Duration of generating presentable objects to be cached in real time', + { caller_id: caller_id, render_type: render_type }, + [0.1, 0.5, 1, 2] + ) + end + + def current_transaction + @current_transaction ||= ::Gitlab::Metrics::WebTransaction.current + end + + def caller_id + @caller_id ||= Gitlab::ApplicationContext.current_context_attribute(:caller_id) + end end end end diff --git a/spec/frontend/design_management/components/toolbar/index_spec.js b/spec/frontend/design_management/components/toolbar/index_spec.js index b6137ba2eee..1776405ece9 100644 --- a/spec/frontend/design_management/components/toolbar/index_spec.js +++ b/spec/frontend/design_management/components/toolbar/index_spec.js @@ -107,7 +107,7 @@ describe('Design management toolbar component', () => { await nextTick(); wrapper.findComponent(DeleteButton).vm.$emit('delete-selected-designs'); - expect(wrapper.emitted().delete).toBeTruthy(); + expect(wrapper.emitted().delete).toHaveLength(1); }); it('renders download button with correct link', () => { diff --git a/spec/frontend/pipeline_wizard/components/commit_spec.js b/spec/frontend/pipeline_wizard/components/commit_spec.js index c987accbb0d..d7e019c642e 100644 --- a/spec/frontend/pipeline_wizard/components/commit_spec.js +++ b/spec/frontend/pipeline_wizard/components/commit_spec.js @@ -174,7 +174,7 @@ describe('Pipeline Wizard - Commit Page', () => { }); it('will not emit a done event', () => { - expect(wrapper.emitted().done?.length).toBeFalsy(); + expect(wrapper.emitted().done?.length).toBeUndefined(); }); afterEach(() => { diff --git a/spec/lib/api/helpers/caching_spec.rb b/spec/lib/api/helpers/caching_spec.rb index 38b7b386d5c..828af7b5f91 100644 --- a/spec/lib/api/helpers/caching_spec.rb +++ b/spec/lib/api/helpers/caching_spec.rb @@ -33,10 +33,7 @@ RSpec.describe API::Helpers::Caching, :use_clean_rails_redis_caching do end describe "#present_cached" do - subject do - instance.present_cached(presentable, **kwargs) - end - + let(:method) { :present_cached } let(:kwargs) do { with: presenter, @@ -44,6 +41,10 @@ RSpec.describe API::Helpers::Caching, :use_clean_rails_redis_caching do } end + subject do + instance.public_send(method, presentable, **kwargs) + end + context 'single object' do let_it_be(:presentable) { create(:todo, project: project) } diff --git a/spec/lib/gitlab/audit/auditor_spec.rb b/spec/lib/gitlab/audit/auditor_spec.rb index fc5917ca583..f743515e616 100644 --- a/spec/lib/gitlab/audit/auditor_spec.rb +++ b/spec/lib/gitlab/audit/auditor_spec.rb @@ -4,7 +4,7 @@ require 'spec_helper' RSpec.describe Gitlab::Audit::Auditor do let(:name) { 'audit_operation' } - let(:author) { create(:user) } + let(:author) { create(:user, :with_sign_ins) } let(:group) { create(:group) } let(:provider) { 'standard' } let(:context) do @@ -37,6 +37,13 @@ RSpec.describe Gitlab::Audit::Auditor do ).and_call_original audit! + + authentication_event = AuthenticationEvent.last + + expect(authentication_event.user).to eq(author) + expect(authentication_event.user_name).to eq(author.name) + expect(authentication_event.ip_address).to eq(author.current_sign_in_ip) + expect(authentication_event.provider).to eq(provider) end it 'logs audit events to database', :aggregate_failures do diff --git a/spec/lib/gitlab/cache/helpers_spec.rb b/spec/lib/gitlab/cache/helpers_spec.rb index 08e0d7729bd..39d37e979b4 100644 --- a/spec/lib/gitlab/cache/helpers_spec.rb +++ b/spec/lib/gitlab/cache/helpers_spec.rb @@ -18,10 +18,7 @@ RSpec.describe Gitlab::Cache::Helpers, :use_clean_rails_redis_caching do end describe "#render_cached" do - subject do - instance.render_cached(presentable, **kwargs) - end - + let(:method) { :render_cached } let(:kwargs) do { with: presenter, @@ -29,6 +26,10 @@ RSpec.describe Gitlab::Cache::Helpers, :use_clean_rails_redis_caching do } end + subject do + instance.public_send(method, presentable, **kwargs) + end + context 'single object' do let_it_be(:presentable) { create(:merge_request, source_project: project, source_branch: 'wip') } diff --git a/spec/services/merge_requests/create_pipeline_service_spec.rb b/spec/services/merge_requests/create_pipeline_service_spec.rb index c443d758a77..dc96b5c0e5e 100644 --- a/spec/services/merge_requests/create_pipeline_service_spec.rb +++ b/spec/services/merge_requests/create_pipeline_service_spec.rb @@ -2,7 +2,7 @@ require 'spec_helper' -RSpec.describe MergeRequests::CreatePipelineService do +RSpec.describe MergeRequests::CreatePipelineService, :clean_gitlab_redis_cache do include ProjectForksHelper let_it_be(:project, reload: true) { create(:project, :repository) } diff --git a/spec/support/rspec_order.rb b/spec/support/rspec_order.rb index 58f1fe29357..e9a33dd7bea 100644 --- a/spec/support/rspec_order.rb +++ b/spec/support/rspec_order.rb @@ -43,9 +43,6 @@ RSpec.configure do |config| config.on_example_group_definition do |example_group| order = Support::RspecOrder.order_for(example_group) - if order - example_group.metadata[:order] = order.to_sym - example_group.metadata[:description] += " (order #{order})" - end + example_group.metadata[:order] = order.to_sym if order end end diff --git a/spec/support/shared_examples/lib/cache_helpers_shared_examples.rb b/spec/support/shared_examples/lib/cache_helpers_shared_examples.rb index 845fa78a827..495ea931728 100644 --- a/spec/support/shared_examples/lib/cache_helpers_shared_examples.rb +++ b/spec/support/shared_examples/lib/cache_helpers_shared_examples.rb @@ -43,6 +43,54 @@ RSpec.shared_examples_for 'object cache helper' do subject end end + + context 'when a caller id is present' do + let(:transaction) { Gitlab::Metrics::WebTransaction.new({}) } + let(:caller_id) { 'caller_id' } + + before do + allow(::Gitlab::Metrics::WebTransaction).to receive(:current).and_return(transaction) + allow(transaction).to receive(:increment) + allow(Gitlab::ApplicationContext).to receive(:current_context_attribute).with(:caller_id).and_return(caller_id) + end + + context 'when feature flag is off' do + before do + stub_feature_flags(add_timing_to_certain_cache_actions: false) + end + + it 'does not call increment' do + expect(transaction).not_to receive(:increment).with(:cached_object_operations_total, any_args) + + subject + end + + it 'does not call histogram' do + expect(Gitlab::Metrics).not_to receive(:histogram) + + subject + end + + it "is valid JSON" do + parsed = Gitlab::Json.parse(subject.to_s) + + expect(parsed).to be_a(Hash) + expect(parsed["id"]).to eq(presentable.id) + end + end + + it 'increments the counter' do + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, 1, { caller_id: caller_id, render_type: :object, cache_hit: false }).once + + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, 0, { caller_id: caller_id, render_type: :object, cache_hit: true }).once + + subject + end + end end RSpec.shared_examples_for 'collection cache helper' do @@ -98,4 +146,73 @@ RSpec.shared_examples_for 'collection cache helper' do subject end end + + context 'when a caller id is present' do + let(:transaction) { Gitlab::Metrics::WebTransaction.new({}) } + let(:caller_id) { 'caller_id' } + + before do + allow(::Gitlab::Metrics::WebTransaction).to receive(:current).and_return(transaction) + allow(transaction).to receive(:increment) + allow(Gitlab::ApplicationContext).to receive(:current_context_attribute).with(:caller_id).and_return(caller_id) + end + + context 'when feature flag is off' do + before do + stub_feature_flags(add_timing_to_certain_cache_actions: false) + end + + it 'does not call increment' do + expect(transaction).not_to receive(:increment).with(:cached_object_operations_total, any_args) + + subject + end + + it 'does not call histogram' do + expect(Gitlab::Metrics).not_to receive(:histogram) + + subject + end + + it "is valid JSON" do + parsed = Gitlab::Json.parse(subject.to_s) + + expect(parsed).to be_an(Array) + + presentable.each_with_index do |item, i| + expect(parsed[i]["id"]).to eq(item.id) + end + end + end + + context 'when the presentables all miss' do + it 'increments the counters' do + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, 0, { caller_id: caller_id, render_type: :collection, cache_hit: true }).once + + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, presentable.size, { caller_id: caller_id, render_type: :collection, cache_hit: false }).once + + subject + end + end + + context 'when the presents hit' do + it 'increments the counters' do + subject + + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, presentable.size, { caller_id: caller_id, render_type: :collection, cache_hit: true }).once + + expect(transaction) + .to receive(:increment) + .with(:cached_object_operations_total, 0, { caller_id: caller_id, render_type: :collection, cache_hit: false }).once + + instance.public_send(method, presentable, **kwargs) + end + end + end end |