Add latest changes from gitlab-org/gitlab@master

5 files changed, 63 insertions, 24 deletions

diff --git a/doc/development/documentation/feature_flags.md b/doc/development/documentation/feature_flags.md
index e2fbf25eb8a..8ad7ef64587 100644
--- a/doc/development/documentation/feature_flags.md
+++ b/doc/development/documentation/feature_flags.md
@@ -75,7 +75,7 @@ To enable it:
 # Instance-wide
 Feature.enable(:<feature flag>)
 # or by project
-Feature.enable(:<feature flag>, Project.find(<project id>))
+Feature.enable(:<feature flag>, Project.find(<project ID>))
 ```
 
 To disable it:
@@ -84,7 +84,7 @@ To disable it:
 # Instance-wide
 Feature.disable(:<feature flag>)
 # or by project
-Feature.disable(:<feature flag>, Project.find(<project id>))
+Feature.disable(:<feature flag>, Project.find(<project ID>))
 ```
 ````
 
diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md
index 2f01692e944..614b2579ec3 100644
--- a/doc/development/elasticsearch.md
+++ b/doc/development/elasticsearch.md
@@ -229,7 +229,7 @@ be used both locally in development and on any deployed GitLab instance to
 diagnose poor search performance. This will show the exact queries being made,
 which is useful to diagnose why a search might be slow.
 
-### Correlation ID and X-Opaque-Id
+### Correlation ID and `X-Opaque-Id`
 
 Our [correlation
 ID](./distributed_tracing.md#developer-guidelines-for-working-with-correlation-ids)
diff --git a/doc/development/geo/framework.md b/doc/development/geo/framework.md
index 64c9030e3dd..fc8a1c56d67 100644
--- a/doc/development/geo/framework.md
+++ b/doc/development/geo/framework.md
@@ -128,7 +128,7 @@ When this is set in place, it's easy to access the replicator through
 the model:
 
 ```ruby
-package_file = Packages::PackageFile.find(4) # just a random id as example
+package_file = Packages::PackageFile.find(4) # just a random ID as example
 replicator = package_file.replicator
 ```
 
diff --git a/doc/development/telemetry/snowplow.md b/doc/development/telemetry/snowplow.md
index 547ba36464b..f01c2ec95c3 100644
--- a/doc/development/telemetry/snowplow.md
+++ b/doc/development/telemetry/snowplow.md
@@ -393,7 +393,7 @@ Snowplow Micro is a Docker-based solution for testing frontend and backend event
 1. Send a test Snowplow event from the Rails console:
 
    ```ruby
-   Gitlab::Tracking.self_describing_event('iglu:com.gitlab/pageview_context/jsonschema/1-0-0', { page_type: ‘MY_TYPE' }, context: nil )
+   Gitlab::Tracking.self_describing_event('iglu:com.gitlab/pageview_context/jsonschema/1-0-0', { page_type: 'MY_TYPE' }, context: nil )
    ```
 
 ### Snowplow Mini
diff --git a/doc/development/telemetry/usage_ping.md b/doc/development/telemetry/usage_ping.md
index be599898214..383fb4a126b 100644
--- a/doc/development/telemetry/usage_ping.md
+++ b/doc/development/telemetry/usage_ping.md
@@ -222,38 +222,77 @@ Examples of implementation:
 
 #### Redis HLL Counters
 
-With `Gitlab::Redis::HLL` we have available data structures used to count unique values.
+With `Gitlab::UsageDataCounters::HLLRedisCounter` we have available data structures used to count unique values.
 
 Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd) and [PFCOUNT](https://redis.io/commands/pfcount).
 
-Recommendations:
+##### Adding new events
 
-- Key should expire in 29 days.
-- If possible, data granularity should be a week. For example a key could be composed from the metric's name and week of the year, `2020-33-{metric_name}`.
-- Use a [feature flag](../../operations/feature_flags.md) in order to have a control over the impact when adding new metrics.
-- If possible, data granularity should be week, for example a key could be composed from metric name and week of the year, 2020-33-{metric_name}
-- Use a [feature flag](../../operations/feature_flags.md) in order to have a control over the impact when adding new metrics
+1. Define events in [`known_events.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events.yml).
 
-Examples of implementation:
+   Example event:
 
-- [`Gitlab::UsageDataCounters::TrackUniqueEvents`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/track_unique_actions.rb)
-- [`Gitlab::Analytics::UniqueVisits`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/analytics/unique_visits.rb)
+   ```yaml
+   - name: i_compliance_credential_inventory
+     category: compliance
+     redis_slot: compliance
+     expiry: 42 # 6 weeks
+     aggregation: weekly
+   ```
 
-Example of usage:
+   Keys:
+
+   - `name`: unique event name.
+   - `category`: event category. Used for getting total counts for events in a category, for easier
+     access to a group of events.
+   - `redis_slot`: optional Redis slot; default value: event name. Used if needed to calculate totals
+     for a group of metrics. Ensure keys are in the same slot. For example:
+     `i_compliance_credential_inventory` with `redis_slot: 'compliance'` will build Redis key
+     `i_{compliance}_credential_inventory-2020-34`. If `redis_slot` is not defined the Redis key will
+     be `{i_compliance_credential_inventory}-2020-34`.
+   - `expiry`: expiry time in days. Default: 29 days for daily aggregation and 6 weeks for weekly
+     aggregation.
+   - `aggregation`: aggregation `:daily` or `:weekly`. The argument defines how we build the Redis
+     keys for data storage. For `daily` we keep a key for metric per day of the year, for `weekly` we
+     keep a key for metric per week of the year.
+
+1. Track event using `Gitlab::UsageDataCounters::HLLRedisCounter.track_event(entity_id, event_name)`.
+
+   Arguments:
+
+   - `entity_id`: value we count. For example: user_id, visitor_id.
+   - `event_name`: event name.
+
+1. Get event data using `Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names:, start_date:, end_date)`.
+
+   Arguments:
+
+   - `event_names`: the list of event names.
+   - `start_date`: start date of the period for which we want to get event data.
+   - `end_date`: end date of the period for which we want to get event data.
+
+Recommendations:
+
+- Key should expire in 29 days for daily and 42 days for weekly.
+- If possible, data granularity should be a week. For example a key could be composed from the
+  metric's name and week of the year, `2020-33-{metric_name}`.
+- Use a [feature flag](../../operations/feature_flags.md) to have a control over the impact when
+  adding new metrics.
+
+Example usage:
 
 ```ruby
 # Redis Counters
 redis_usage_data(Gitlab::UsageDataCounters::WikiPageCounter)
 redis_usage_data { ::Gitlab::UsageCounters::PodLogs.usage_totals[:total] }
 
-# Redis HLL counter
-counter = Gitlab::UsageDataCounters::TrackUniqueEvents
-redis_usage_data do
-          counter.count_unique_events(
-            event_action: Gitlab::UsageDataCounters::TrackUniqueEvents::PUSH_ACTION,
-            date_from: time_period[:created_at].first,
-            date_to: time_period[:created_at].last
-          )
+# Define events in known_events.yml https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events.yml
+
+# Tracking events
+Gitlab::UsageDataCounters::HLLRedisCounter.track_event(visitor_id, 'expand_vulnerabilities')
+
+# Get unique events for metric
+redis_usage_data { Gitlab::UsageDataCounters::HLLRedisCounter.unique_events(event_names: 'expand_vulnerabilities', start_date: 28.days.ago, end_date: Date.current) }
 ```
 
 ### Alternative Counters