Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'doc/development/service_ping')
-rw-r--r--doc/development/service_ping/implement.md30
-rw-r--r--doc/development/service_ping/index.md2
-rw-r--r--doc/development/service_ping/metrics_dictionary.md5
-rw-r--r--doc/development/service_ping/metrics_instrumentation.md82
-rw-r--r--doc/development/service_ping/metrics_lifecycle.md4
-rw-r--r--doc/development/service_ping/performance_indicator_metrics.md2
-rw-r--r--doc/development/service_ping/review_guidelines.md2
-rw-r--r--doc/development/service_ping/troubleshooting.md2
-rw-r--r--doc/development/service_ping/usage_data.md2
9 files changed, 115 insertions, 16 deletions
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md
index 5448bbb4293..88a1454393f 100644
--- a/doc/development/service_ping/implement.md
+++ b/doc/development/service_ping/implement.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Implement Service Ping
@@ -179,7 +179,7 @@ As the HyperLogLog algorithm is probabilistic, the **results always include erro
The highest encountered error rate is 4.9%.
When correctly used, the `estimate_batch_distinct_count` method enables efficient counting over
-columns that contain non-unique values, which can not be assured by other counters.
+columns that contain non-unique values, which cannot be assured by other counters.
##### estimate_batch_distinct_count method
@@ -605,7 +605,7 @@ alt_usage_data(value = nil, fallback: -1, &block)
Arguments:
-- `value`: a simple static value in which case the value is simply returned.
+- `value`: a static value in which case the value is returned.
- or a `block`: which is evaluated
- `fallback: -1`: the common value used for any metrics that are failing.
@@ -712,10 +712,10 @@ We also use `#database-lab` and [explain.depesz.com](https://explain.depesz.com/
- Use specialized indexes. For examples, see these merge requests:
- [Example 1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26871)
- [Example 2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/26445)
-- Use defined `start` and `finish`, and simple queries.
- These values can be memoized and reused, as in this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37155).
-- Avoid joins and write the queries as simply as possible,
- as in this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36316).
+- Use defined `start` and `finish`. These values can be memoized and reused, as in this
+ [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37155).
+- Avoid joins and unnecessary complexity in your queries. See this
+ [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36316) as an example.
- Set a custom `batch_size` for `distinct_count`, as in this [example merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38000).
## Add the metric definition
@@ -839,6 +839,22 @@ However, it has the following limitations:
WARNING:
This feature is intended solely for internal GitLab use.
+The aggregated metrics feature provides insight into the data attributes in a collection of Service Ping metrics.
+This aggregation allows you to count data attributes in events without counting each occurrence of the same data attribute in multiple events.
+For example, you can aggregate the number of users who perform several actions, such as creating a new issue and opening a new merge request.
+You can then count each user that performed any combination of these actions.
+
+### Defining aggregated metric via metric YAML definition
+
+To add data for aggregated metrics to the Service Ping payload,
+create metric YAML definition file following [Aggregated metric instrumentation guide](metrics_instrumentation.md#aggregated-metrics).
+
+### (DEPRECATED) Defining aggregated metric via aggregated metric YAML config file
+
+WARNING:
+This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98206) in GitLab 15.5
+and is planned for removal in 15.5. Use [metrics definition YAMLs](https://gitlab.com/gitlab-org/gitlab/-/issues/370963) instead.
+
To add data for aggregated metrics to the Service Ping payload, add a corresponding definition to:
- [`config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/aggregates/) for metrics available in the Community Edition.
diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md
index f252eb967aa..0a4e5998345 100644
--- a/doc/development/service_ping/index.md
+++ b/doc/development/service_ping/index.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Service Ping Guide **(FREE SELF)**
diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md
index d063c4c7601..3439f581e7f 100644
--- a/doc/development/service_ping/metrics_dictionary.md
+++ b/doc/development/service_ping/metrics_dictionary.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Metrics Dictionary Guide
@@ -136,6 +136,9 @@ We use the following categories to classify a metric:
- `subscription`: Data related to licensing.
- `standard`: Standard set of identifiers that are included when collecting data.
+An aggregate metric is a metric that is the sum of two or more child metrics. Service Ping uses the data category of
+the aggregate metric to determine whether or not the data is included in the reported Service Ping payload.
+
### Metric name suggestion examples
#### Metric with `data_source: database`
diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md
index debb3a68c04..7a3f291460c 100644
--- a/doc/development/service_ping/metrics_instrumentation.md
+++ b/doc/development/service_ping/metrics_instrumentation.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Metrics instrumentation guide
@@ -254,6 +254,86 @@ options:
- i_quickactions_approve
```
+## Aggregated metrics
+
+The aggregated metrics feature provides insight into the number of data attributes, for example `pseudonymized_user_ids`, that occurred in a collection of events. For example, you can aggregate the number of users who perform multiple actions such as creating a new issue and opening
+a new merge request.
+
+You can use a YAML file to define your aggregated metrics. The following arguments are required:
+
+- `options.events`: List of event names to aggregate into metric data. All events in this list must
+ use the same data source. Additional data source requirements are described in
+ [Database sourced aggregated metrics](implement.md#database-sourced-aggregated-metrics) and
+ [Redis sourced aggregated metrics](implement.md#redis-sourced-aggregated-metrics).
+- `options.aggregate.operator`: Operator that defines how the aggregated metric data is counted. Available operators are:
+ - `OR`: Removes duplicates and counts all entries that triggered any of the listed events.
+ - `AND`: Removes duplicates and counts all elements that were observed triggering all of the following events.
+- `options.aggregate.attribute`: Information pointing to the attribute that is being aggregated across events.
+- `time_frame`: One or more valid time frames. Use these to limit the data included in aggregated metrics to events within a specific date-range. Valid time frames are:
+ - `7d`: The last 7 days of data.
+ - `28d`: The last 28 days of data.
+ - `all`: All historical data, only available for `database` sourced aggregated metrics.
+- `data_source`: Data source used to collect all events data included in the aggregated metrics. Valid data sources are:
+ - [`database`](implement.md#database-sourced-aggregated-metrics)
+ - [`redis_hll`](implement.md#redis-sourced-aggregated-metrics)
+
+Refer to merge request [98206](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/98206) for an example of a merge request that adds an `AggregatedMetric` metric.
+
+Count unique `user_ids` that occurred in at least one of the events: `incident_management_alert_status_changed`,
+`incident_management_alert_assigned`, `incident_management_alert_todo`, `incident_management_alert_create_incident`.
+
+```yaml
+time_frame: 28d
+instrumentation_class: AggregatedMetric
+data_source: redis_hll
+options:
+ aggregate:
+ operator: OR
+ attribute: user_id
+ events:
+ - `incident_management_alert_status_changed`
+ - `incident_management_alert_assigned`
+ - `incident_management_alert_todo`
+ - `incident_management_alert_create_incident`
+```
+
+### Availability-restrained Aggregated metrics
+
+If the Aggregated metric should only be available in the report under specific conditions, then you must specify these conditions in a new class that is a child of the `AggregatedMetric` class.
+
+```ruby
+# frozen_string_literal: true
+
+module Gitlab
+ module Usage
+ module Metrics
+ module Instrumentations
+ class MergeUsageCountAggregatedMetric < AggregatedMetric
+ available? { Feature.enabled?(:merge_usage_data_missing_key_paths) }
+ end
+ end
+ end
+ end
+end
+```
+
+You must also use the class's name in the YAML setup.
+
+```yaml
+time_frame: 28d
+instrumentation_class: MergeUsageCountAggregatedMetric
+data_source: redis_hll
+options:
+ aggregate:
+ operator: OR
+ attribute: user_id
+ events:
+ - `incident_management_alert_status_changed`
+ - `incident_management_alert_assigned`
+ - `incident_management_alert_todo`
+ - `incident_management_alert_create_incident`
+```
+
## Numbers metrics
- `operation`: Operations for the given `data` block. Currently we only support `add` operation.
diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md
index 28f77b6f587..f13aebeb16e 100644
--- a/doc/development/service_ping/metrics_lifecycle.md
+++ b/doc/development/service_ping/metrics_lifecycle.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Service Ping metric lifecycle
@@ -120,7 +120,7 @@ To remove a metric:
Do not remove the metric's YAML definition altogether. Some self-managed
instances might not immediately update to the latest version of GitLab, and
therefore continue to report the removed metric. The Product Intelligence team
- requires a record of all removed metrics in order to identify and filter them.
+ requires a record of all removed metrics to identify and filter them.
For example please take a look at this [merge request](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/60149/diffs#b01f429a54843feb22265100c0e4fec1b7da1240_10_10).
diff --git a/doc/development/service_ping/performance_indicator_metrics.md b/doc/development/service_ping/performance_indicator_metrics.md
index d2abc597a22..4c1c61aa05b 100644
--- a/doc/development/service_ping/performance_indicator_metrics.md
+++ b/doc/development/service_ping/performance_indicator_metrics.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Performance Indicator Metrics guide
diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md
index 1b00858be7e..a1806551303 100644
--- a/doc/development/service_ping/review_guidelines.md
+++ b/doc/development/service_ping/review_guidelines.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Service Ping review guidelines
diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md
index 4431d5df3ff..201d6a385eb 100644
--- a/doc/development/service_ping/troubleshooting.md
+++ b/doc/development/service_ping/troubleshooting.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Troubleshooting Service Ping
diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md
index 4181bd90a02..2f2df14f56d 100644
--- a/doc/development/service_ping/usage_data.md
+++ b/doc/development/service_ping/usage_data.md
@@ -1,7 +1,7 @@
---
stage: Analytics
group: Product Intelligence
-info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Usage Data Metrics guide