diff options
Diffstat (limited to 'doc/development/service_ping')
| -rw-r--r-- | doc/development/service_ping/implement.md | 79 | ||||
| -rw-r--r-- | doc/development/service_ping/index.md | 48 | ||||
| -rw-r--r-- | doc/development/service_ping/metrics_dictionary.md | 8 | ||||
| -rw-r--r-- | doc/development/service_ping/metrics_instrumentation.md | 8 | ||||
| -rw-r--r-- | doc/development/service_ping/metrics_lifecycle.md | 64 | ||||
| -rw-r--r-- | doc/development/service_ping/performance_indicator_metrics.md | 2 | ||||
| -rw-r--r-- | doc/development/service_ping/review_guidelines.md | 37 | ||||
| -rw-r--r-- | doc/development/service_ping/troubleshooting.md | 2 | ||||
| -rw-r--r-- | doc/development/service_ping/usage_data.md | 2 |
9 files changed, 88 insertions, 162 deletions
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md index ef2e7e6edf5..73b74feb239 100644 --- a/doc/development/service_ping/implement.md +++ b/doc/development/service_ping/implement.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -19,7 +19,7 @@ To implement a new metric in Service Ping, follow these steps: 1. [Name and place the metric](metrics_dictionary.md#metric-key_path) 1. [Test counters manually using your Rails console](#test-counters-manually-using-your-rails-console) 1. [Generate the SQL query](#generate-the-sql-query) -1. [Optimize queries with `#database-lab`](#optimize-queries-with-database-lab) +1. [Optimize queries with Database Lab](#optimize-queries-with-database-lab) 1. [Add the metric definition to the Metrics Dictionary](#add-the-metric-definition) 1. [Add the metric to the Versions Application](#add-the-metric-to-the-versions-application) 1. [Create a merge request](#create-a-merge-request) @@ -174,7 +174,7 @@ Errors return a value of `-1`. WARNING: This functionality estimates a distinct count of a specific ActiveRecord_Relation in a given column, -which uses the [HyperLogLog](http://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf) algorithm. +which uses the [HyperLogLog](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf) algorithm. As the HyperLogLog algorithm is probabilistic, the **results always include error**. The highest encountered error rate is 4.9%. @@ -330,48 +330,36 @@ Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd/) and [P ```yaml - name: users_creating_epics - category: epics_usage - redis_slot: users aggregation: weekly - feature_flag: track_epics_activity ``` Keys: - `name`: unique event name. - Name format for Redis HLL events `<name>_<redis_slot>`. + Name format for Redis HLL events `{hll_counters}_<name>` [See Metric name](metrics_dictionary.md#metric-name) for a complete guide on metric naming suggestion. - Consider including in the event's name the Redis slot to be able to count totals for a specific category. - Example names: `users_creating_epics`, `users_triggering_security_scans`. - - `category`: event category. Used for getting total counts for events in a category, for easier - access to a group of events. - - `redis_slot`: optional Redis slot. Default value: event name. Only event data that is stored in the same slot - can be aggregated. Ensure keys are in the same slot. For example: - `users_creating_epics` with `redis_slot: 'users'` builds Redis key - `{users}_creating_epics-2020-34`. If `redis_slot` is not defined the Redis key will - be `{users_creating_epics}-2020-34`. - Recommended slots to use are: `users`, `projects`. This is the value we count. - `aggregation`: may be set to a `:daily` or `:weekly` key. Defines how counting data is stored in Redis. Aggregation on a `daily` basis does not pull more fine grained data. - - `feature_flag`: if no feature flag is set then the tracking is enabled. One feature flag can be used for multiple events. For details, see our [GitLab internal Feature flags](../feature_flags/index.md) documentation. The feature flags are owned by the group adding the event tracking. 1. Use one of the following methods to track the event: - - In the controller using the `RedisTracking` module and the following format: + - In the controller using the `ProductAnalyticsTracking` module and the following format: ```ruby - track_event(*controller_actions, name:, conditions: nil, destinations: [:redis_hll], &block) + track_event(*controller_actions, name:, action:, label:, conditions: nil, destinations: [:redis_hll], &block) ``` Arguments: - `controller_actions`: the controller actions to track. - `name`: the event name. + - `action`: required if destination is `:snowplow. Action name for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details. + - `label`: required if destination is `:snowplow. Label for the triggered event. See [event schema](../snowplow/index.md#event-schema) for more details. - `conditions`: optional custom conditions. Uses the same format as Rails callbacks. - `destinations`: optional list of destinations. Currently supports `:redis_hll` and `:snowplow`. Default: `:redis_hll`. - `&block`: optional block that computes and returns the `custom_id` that we want to track. This overrides the `visitor_id`. @@ -381,10 +369,14 @@ Implemented using Redis methods [PFADD](https://redis.io/commands/pfadd/) and [P ```ruby # controller class ProjectsController < Projects::ApplicationController - include RedisTracking + include ProductAnalyticsTracking skip_before_action :authenticate_user!, only: :show - track_event :index, :show, name: 'users_visiting_projects' + track_event :index, :show, + name: 'users_visiting_projects', + action: 'user_perform_visit', + label: 'redis_hll_counters.users_visiting_project_monthly', + destinations: %i[redis_hll snowplow] def index render html: 'index' @@ -497,24 +489,12 @@ We have the following recommendations for [adding new events](#add-new-events): - Event aggregation: weekly. - When adding new metrics, use a [feature flag](../../operations/feature_flags.md) to control the impact. -- For feature flags triggered by another service, set `default_enabled: false`, - - Events can be triggered using the `UsageData` API, which helps when there are > 10 events per change +It's recommended to disable the new feature flag by default (set `default_enabled: false`). +- Events can be triggered using the `UsageData` API, which helps when there are > 10 events per change ##### Enable or disable Redis HLL tracking -Events are tracked behind optional [feature flags](../feature_flags/index.md) due to concerns for Redis performance and scalability. - -For a full list of events and corresponding feature flags, see the [`known_events/`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/known_events/) files. - -To enable or disable tracking for specific event in <https://gitlab.com> or <https://about.staging.gitlab.com>, run commands such as the following to -[enable or disable the corresponding feature](../feature_flags/index.md). - -```shell -/chatops run feature set <feature_name> true -/chatops run feature set <feature_name> false -``` - -We can also disable tracking completely by using the global flag: +We can disable tracking completely by using the global flag: ```shell /chatops run feature set redis_hll_tracking true @@ -529,16 +509,6 @@ For each event we add metrics for the weekly and monthly time frames, and totals - `#{event_name}_weekly`: Data for 7 days for daily [aggregation](#add-new-events) events and data for the last complete week for weekly [aggregation](#add-new-events) events. - `#{event_name}_monthly`: Data for 28 days for daily [aggregation](#add-new-events) events and data for the last 4 complete weeks for weekly [aggregation](#add-new-events) events. -Redis HLL implementation calculates total metrics when both of these conditions are met: - -- The category is manually included in [CATEGORIES_FOR_TOTALS](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/usage_data_counters/hll_redis_counter.rb#L21). -- There is more than one metric for the same category, aggregation, and Redis slot. - -We add total unique counts for the weekly and monthly time frames where applicable: - -- `#{category}_total_unique_counts_weekly`: Total unique counts for events in the same category for the last 7 days or the last complete week, if events are in the same Redis slot and we have more than one metric. -- `#{category}_total_unique_counts_monthly`: Total unique counts for events in same category for the last 28 days or the last 4 complete weeks, if events are in the same Redis slot and we have more than one metric. - Example of `redis_hll_counters` data: ```ruby @@ -563,6 +533,7 @@ Example of `redis_hll_counters` data: "ide_edit_total_unique_counts_weekly"=>0, "ide_edit_total_unique_counts_monthly"=>0} } +} ``` Example: @@ -674,10 +645,9 @@ pry(main)> Gitlab::UsageData.count(User.active) (1.9ms) SELECT COUNT("users"."id") FROM "users" WHERE ("users"."state" IN ('active')) AND ("users"."user_type" IS NULL OR "users"."user_type" IN (6, 4)) AND "users"."id" BETWEEN 1 AND 100000 ``` -## Optimize queries with `#database-lab` +## Optimize queries with Database Lab -`#database-lab` is a Slack channel that uses a production-sized environment to test your queries. -Paste the SQL query into `#database-lab` to see how the query performs at scale. +[Database Lab](../database/database_lab.md) is a service that uses a production clone to test queries. - GitLab.com's production database has a 15 second timeout. - Any single query must stay below the [1 second execution time](../database/query_performance.md#timing-guidelines-for-queries) with cold caches. @@ -693,7 +663,7 @@ to a merge request description: - Query generated for the index and time. - Migration output for up and down execution. -We also use `#database-lab` and [explain.depesz.com](https://explain.depesz.com/). For more details, see the [database review guide](../database_review.md#preparation-when-adding-or-modifying-queries). +For more details, see the [database review guide](../database_review.md#preparation-when-adding-or-modifying-queries). ### Optimization recommendations and examples @@ -720,8 +690,8 @@ Create a merge request for the new Service Ping metric, and do the following: - Add the `feature` label to the merge request. A metric is a user-facing change and is part of expanding the Service Ping feature. - Add a changelog entry that complies with the [changelog entries guide](../changelog.md). -- Ask for a Product Intelligence review. - On GitLab.com, we have DangerBot set up to monitor Product Intelligence related files and recommend a [Product Intelligence review](review_guidelines.md). +- Ask for an Analytics Instrumentation review. + On GitLab.com, we have DangerBot set up to monitor Analytics Instrumentation related files and recommend a [Analytics Instrumentation review](review_guidelines.md). ## Verify your metric @@ -751,7 +721,7 @@ To set up Service Ping locally, you must: Make sure you run `docker-compose up` to start a PostgreSQL and Redis instance. 1. Point GitLab to the Versions Application endpoint instead of the default endpoint: 1. Open [service_ping/submit_service.rb](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/services/service_ping/submit_service.rb#L5) locally and modify `STAGING_BASE_URL`. - 1. Set it to the local Versions Application URL: `http://localhost:3000/usage_data`. + 1. Set it to the local Versions Application URL: `http://localhost:3000`. ### Test local setup @@ -846,7 +816,6 @@ you must fulfill the following requirements: 1. All events listed at `events` attribute must come from [`known_events/*.yml`](#known-events-are-added-automatically-in-service-data-payload) files. -1. All events listed at `events` attribute must have the same `redis_slot` attribute. 1. All events listed at `events` attribute must have the same `aggregation` attribute. 1. `time_frame` does not include `all` value, which is unavailable for Redis sourced aggregated metrics. diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md index 14c9cb33446..3504a730eed 100644 --- a/doc/development/service_ping/index.md +++ b/doc/development/service_ping/index.md @@ -1,10 +1,10 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Service Ping Guide +# Service Ping development guidelines > - Introduced in GitLab Ultimate 11.2, more statistics. > - In GitLab 14.1, [renamed from Usage Ping to Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/5990). In 14.0 and earlier, use the Usage Ping documentation for the Rails commands appropriate to your version. @@ -91,23 +91,23 @@ sequenceDiagram the required URL is <https://version.gitlab.com/>. 1. In case of an error, it will be reported to the Version application along with following pieces of information: - - `uuid` - GitLab instance unique identifier - - `hostname` - GitLab instance hostname - - `version` - GitLab instance current versions - - `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence - - `message` - Error message - - <pre> - <code> - { - "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d", - "hostname"=>"127.0.0.1", - "version"=>"14.7.0-pre", - "elapsed"=>0.006946, - "message"=>'PG::UndefinedColumn: ERROR: column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...' - } - </code> - </pre> + - `uuid` - GitLab instance unique identifier + - `hostname` - GitLab instance hostname + - `version` - GitLab instance current versions + - `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence + - `message` - Error message + + <pre> + <code> + { + "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d", + "hostname"=>"127.0.0.1", + "version"=>"14.7.0-pre", + "elapsed"=>0.006946, + "message"=>'PG::UndefinedColumn: ERROR: column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...' + } + </code> + </pre> 1. Finally, the timing metadata information that is used for diagnostic purposes is submitted to the Versions application. It consists of a list of metric identifiers and the time it took to calculate the metrics: @@ -135,7 +135,7 @@ sequenceDiagram ] } } - ``` +``` ### On a Geo secondary site @@ -177,7 +177,7 @@ The following is example content of the Service Ping payload. "recorded_at": "2020-04-17T07:43:54.162+00:00", "edition": "EEU", "license_md5": "00000000000000000000000000000000", - "license_sha256: "0000000000000000000000000000000000000000000000000000000000000000", + "license_sha256": "0000000000000000000000000000000000000000000000000000000000000000", "license_id": null, "historical_max_users": 999, "licensee": { @@ -405,8 +405,8 @@ To generate Service Ping, use [Teleport](https://goteleport.com/docs/) or a deta #### Trigger Service Ping with Teleport -1. Request temporary [access](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to the required environment. -1. After your approval is issued, [access the Rails console](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#access-approval). +1. Request temporary [access](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to the required environment. +1. After your approval is issued, [access the Rails console](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#access-approval). 1. Run `GitlabServicePingWorker.new.perform('triggered_from_cron' => false)`. #### Trigger Service Ping with a detached screen session @@ -450,7 +450,7 @@ Search in Google Console logs for `time_elapsed`. [Query example](https://cloudl #### Verify with Teleport -1. Follow [the steps](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/Teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to request a new access to the required environment and connect to the Rails console +1. Follow [the steps](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/teleport/Connect_to_Rails_Console_via_Teleport.md#how-to-use-teleport-to-connect-to-rails-console) to request a new access to the required environment and connect to the Rails console 1. Check the last payload in `raw_usage_data` table: `RawUsageData.last.payload` 1. Check the when the payload was sent: `RawUsageData.last.sent_at` diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md index 28581f81f94..f36a97bcf6b 100644 --- a/doc/development/service_ping/metrics_dictionary.md +++ b/doc/development/service_ping/metrics_dictionary.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -37,16 +37,15 @@ Each metric is defined in a separate YAML file consisting of a number of fields: | `product_section` | yes | The [section](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/sections.yml). | | `product_stage` | yes | The [stage](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) for the metric. | | `product_group` | yes | The [group](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml) that owns the metric. | -| `product_category` | no | The [product category](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/categories.yml) for the metric. | | `value_type` | yes | `string`; one of [`string`, `number`, `boolean`, `object`](https://json-schema.org/understanding-json-schema/reference/type.html). | | `status` | yes | `string`; [status](#metric-statuses) of the metric, may be set to `active`, `removed`, `broken`. | | `time_frame` | yes | `string`; may be set to a value like `7d`, `28d`, `all`, `none`. | | `data_source` | yes | `string`; may be set to a value like `database`, `redis`, `redis_hll`, `prometheus`, `system`, `license`. | | `data_category` | yes | `string`; [categories](#data-category) of the metric, may be set to `operational`, `optional`, `subscription`, `standard`. The default value is `optional`.| | `instrumentation_class` | yes | `string`; [the class that implements the metric](metrics_instrumentation.md). | -| `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the tracked feature is available. | +| `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. | | `performance_indicator_type` | no | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, `umau` or `customer_health_score`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). | -| `tier` | yes | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. | +| `tier` | yes | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier](https://about.gitlab.com/handbook/marketing/brand-and-product-marketing/product-and-solution-marketing/tiers/#definitions) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. | | `milestone` | yes | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. | | `milestone_removed` | no | The milestone when the metric is removed. | | `introduced_by_url` | no | The URL to the merge request that introduced the metric to be available for self-managed instances. | @@ -218,7 +217,6 @@ instance unique identifier. ```yaml key_path: uuid description: GitLab instance unique identifier -product_category: collection product_section: analytics product_stage: analytics product_group: product_intelligence diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md index 80500ccc723..7441a2d1bd4 100644 --- a/doc/development/service_ping/metrics_instrumentation.md +++ b/doc/development/service_ping/metrics_instrumentation.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -154,17 +154,17 @@ end You can use Redis metrics to track events not kept in the database, for example, a count of how many times the search bar has been used. -[Example of a merge request that adds a `Redis` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/97009). +[Example of a merge request that adds `Redis` metrics](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/103455). The `RedisMetric` class can only be used as the `instrumentation_class` for Redis metrics with simple counters classes (classes that only inherit `BaseCounter` and set `PREFIX` and `KNOWN_EVENTS` constants). In case the counter class has additional logic included in it, a new `instrumentation_class`, inheriting from `RedisMetric`, needs to be created. This new class needs to include the additional logic from the counter class. -Count unique values for `source_code_pushes` event. - Required options: - `event`: the event name. - `prefix`: the value of the `PREFIX` constant used in the counter classes from the `Gitlab::UsageDataCounters` namespace. +Count unique values for `source_code_pushes` event. + ```yaml time_frame: all data_source: redis diff --git a/doc/development/service_ping/metrics_lifecycle.md b/doc/development/service_ping/metrics_lifecycle.md index 8a8ceae1f4c..318db6895fb 100644 --- a/doc/development/service_ping/metrics_lifecycle.md +++ b/doc/development/service_ping/metrics_lifecycle.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- @@ -14,57 +14,21 @@ Follow the [Implement Service Ping](implement.md) guide. ## Change an existing metric -See [this video tutorial](https://youtu.be/bYf3c01KCls) for help with the update of metric attributes. - -NOTE: -The `key_path` attribute represents the location of the metric in Service Ping payload and must not be changed. - -Because we do not control when customers update their self-managed instances of GitLab, -we **STRONGLY DISCOURAGE** changes to the logic used to calculate any metric. -Any such changes lead to inconsistent reports from multiple GitLab instances. -If there is a problem with an existing metric, it's best to deprecate the existing metric, -and use it, side by side, with the desired new metric. - -If you do need to change a metric, please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the MR. -Many Service Ping metrics are relied upon for health score and XMAU reporting and -unexpected changes to those metrics could break reporting. - -Example: -Consider following change. Before GitLab 12.6, the `example_metric` was implemented as: - -```ruby -{ - ... - example_metric: distinct_count(Project, :creator_id) -} -``` - -For GitLab 12.6, the metric was changed to filter out archived projects: - -```ruby -{ - ... - example_metric: distinct_count(Project.non_archived, :creator_id) -} -``` +WARNING: +We want to **PREVENT** changes to the calculation logic or important attributes on any metric as this invalidates comparisons of the same metric across different versions of GitLab. -In this scenario, all instances running up to GitLab 12.5 continue to report `example_metric`, -including all archived projects, while all instances running GitLab 12.6 and higher filters -out such projects. As Service Ping data is collected from all reporting instances, the -resulting dataset includes mixed data, which distorts any following business analysis. +If you change a metric, you have to consider that not all instances of GitLab are running on the newest version. Old instances will still report the old version of the metric. +Additionally, a metric's reported numbers are primarily interesting compared to previously reported numbers. +As a result, if you need to change one of the following parts of a metric, you need to add a new metric instead. It's your choice whether to keep the old metric alongside the new one or [remove it](#remove-a-metric). -The correct approach is to add a new metric for GitLab 12.6 release with updated logic: +- **calculation logic**: This means any changes that can produce a different value than the previous implementation +- **YAML attributes**: The following attributes are directly used for analysis or calculation: `key_path`, `time_frame`, `value_type`, `data_source`. -```ruby -{ - ... - example_metric_without_archived: distinct_count(Project.non_archived, :creator_id) -} -``` +If you change the `performance_indicator_type` attribute of a metric or think your case needs an exception from the outlined rules then please notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) teams by `@` mentioning those groups in a comment on the merge request or issue. -and update existing business analysis artefacts to use `example_metric_without_archived` instead of `example_metric` +You can change any other attributes without impact to the calculation or analysis. See [this video tutorial](https://youtu.be/bYf3c01KCls) for help updating metric attributes. -Currently, the [Metrics Dictionary](https://metrics.gitlab.com/) is built automatically once a day. When a change to a metric is made in a YAML file, you can see the change in the dictionary within 24 hours. +Currently, the [Metrics Dictionary](https://metrics.gitlab.com/) is built automatically once a day. You can see the change in the dictionary within 24 hours when you change the metric's YAML file. ## Remove a metric @@ -82,10 +46,6 @@ To remove a metric: 1. Create an issue for removing the metric if none exists yet. The issue needs to outline why the metric should be deleted. You can use this issue to document the removal process. -1. Check the following YAML files and verify the metric is not used in an aggregate: - - [`config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/aggregates/) - - [`ee/config/metrics/aggregates/*.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/config/metrics/aggregates/) - 1. Verify the metric is not used to calculate the conversational index. The conversational index is a measure that reports back to self-managed instances to inform administrators of the progress of DevOps adoption for the instance. @@ -115,7 +75,7 @@ To remove a metric: can be safely removed from Service Ping. Use this [example issue](https://gitlab.com/gitlab-data/analytics/-/issues/15266) for guidance. -1. Notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment in the issue regarding the deletion of the metric. +1. Notify the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment in the issue from step 1 regarding the deletion of the metric. Many Service Ping metrics are relied upon for health score and XMAU reporting and unexpected changes to those metrics could break reporting. 1. After you verify the metric can be safely removed, diff --git a/doc/development/service_ping/performance_indicator_metrics.md b/doc/development/service_ping/performance_indicator_metrics.md index 4c1c61aa05b..0ca663ce09a 100644 --- a/doc/development/service_ping/performance_indicator_metrics.md +++ b/doc/development/service_ping/performance_indicator_metrics.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md index 9813c9e0b12..8128f1e371b 100644 --- a/doc/development/service_ping/review_guidelines.md +++ b/doc/development/service_ping/review_guidelines.md @@ -1,13 +1,13 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- # Service Ping review guidelines This page includes introductory material for a -[Product Intelligence](https://about.gitlab.com/handbook/engineering/development/analytics/product-intelligence/) +[Analytics Instrumentation](https://about.gitlab.com/handbook/engineering/development/analytics/analytics-instrumentation/) review, and is specific to Service Ping related reviews. For broader advice and general best practices for code reviews, refer to our [code review guide](../code_review.md). @@ -18,7 +18,7 @@ general best practices for code reviews, refer to our [code review guide](../cod ## Review process -We recommend a Product Intelligence review when a merge request (MR) touches +We recommend a Analytics Instrumentation review when a merge request (MR) touches any of the following Service Ping files: - `usage_data*` files. @@ -26,29 +26,29 @@ any of the following Service Ping files: - [`config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/metrics). - [`ee/config/metrics`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/config/metrics). - [`schema.json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/metrics/schema.json). -- Product Intelligence tooling. For example, +- Analytics Instrumentation tooling. For example, [`Gitlab::UsageMetricDefinitionGenerator`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/generators/gitlab/usage_metric_definition_generator.rb) ### Roles and process #### The merge request **author** should -- Decide whether a Product Intelligence review is needed. You can skip the Product Intelligence -review and remove the labels if the changes are not related to the Product Intelligence domain and +- Decide whether a Analytics Instrumentation review is needed. You can skip the Analytics Instrumentation +review and remove the labels if the changes are not related to the Analytics Instrumentation domain and are regular backend changes. -- If a Product Intelligence review is needed, add the labels - `~product intelligence` and `~product intelligence::review pending`. -- For merge requests authored by Product Intelligence team members: - - Assign both the `~backend` and `~product intelligence` reviews to another Product Intelligence team member. - - Assign the maintainer review to someone outside of the Product Intelligence group. +- If a Analytics Instrumentation review is needed, add the labels + `~analytics instrumentation` and `~analytics instrumentation::review pending`. +- For merge requests authored by Analytics Instrumentation team members: + - Assign both the `~backend` and `~analytics instrumentation` reviews to another Analytics Instrumentation team member. + - Assign the maintainer review to someone outside of the Analytics Instrumentation group. - Assign an - [engineer](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Product Intelligence team for a review. + [engineer](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) from the Analytics Instrumentation team for a review. - Set the correct attributes in the metric's YAML definition: - - `product_section`, `product_stage`, `product_group`, `product_category` + - `product_section`, `product_stage`, `product_group` - Provide a clear description of the metric. - Add a changelog [according to guidelines](../changelog.md). -#### The Product Intelligence **reviewer** should +#### The Analytics Instrumentation **reviewer** should - Perform a first-pass review on the merge request and suggest improvements to the author. - Check the [metrics location](metrics_dictionary.md#metric-key_path) in @@ -59,25 +59,24 @@ are regular backend changes. metrics that are based on Database. - Add `~Data Warehouse::Impact Check` for any database metric that has a query change. Changes in queries can affect [data operations](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/triage/#gitlabcom-db-structure-changes). - For tracking using Redis HLL (HyperLogLog): - - Check the Redis slot. - Check if a [feature flag is needed](implement.md#recommendations). - For a metric's YAML definition: - Check the metric's `description`. - Check the metric's `key_path`. - - Check the `product_section`, `product_stage`, `product_group`, and `product_category` fields. + - Check the `product_section`, `product_stage`, and `product_group` fields. Read the [stages file](https://gitlab.com/gitlab-com/www-gitlab-com/blob/master/data/stages.yml). - Check the file location. Consider the time frame, and if the file should be under `ee`. - Check the tiers. - If a metric was changed or removed: Make sure the MR author notified the Customer Success Ops team (`@csops-team`), Analytics Engineers (`@gitlab-data/analytics-engineers`), and Product Analysts (`@gitlab-data/product-analysts`) by `@` mentioning those groups in a comment on the issue for the MR and all of these groups have acknowledged the removal. - Metrics instrumentations - Recommend using metrics instrumentation for new metrics, [if possible](metrics_instrumentation.md#support-for-instrumentation-classes). -- Approve the MR, and relabel the MR with `~"product intelligence::approved"`. +- Approve the MR, and relabel the MR with `~"analytics instrumentation::approved"`. ## Review workload distribution -[Danger bot](../dangerbot.md) adds the list of changed Product Intelligence files +[Danger bot](../dangerbot.md) adds the list of changed Analytics Instrumentation files and pings the [`@gitlab-org/analytics-section/product-intelligence/engineers`](https://gitlab.com/groups/gitlab-org/analytics-section/product-intelligence/engineers/-/group_members?with_inherited_permissions=exclude) group for merge requests that are not drafts. -Any of the Product Intelligence engineers can be assigned for the Product Intelligence review. +Any of the Analytics Instrumentation engineers can be assigned for the Analytics Instrumentation review. diff --git a/doc/development/service_ping/troubleshooting.md b/doc/development/service_ping/troubleshooting.md index 3b7cd092d97..1b896efb726 100644 --- a/doc/development/service_ping/troubleshooting.md +++ b/doc/development/service_ping/troubleshooting.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md index 1c7a212ed64..b3bdaedd60a 100644 --- a/doc/development/service_ping/usage_data.md +++ b/doc/development/service_ping/usage_data.md @@ -1,6 +1,6 @@ --- stage: Analytics -group: Product Intelligence +group: Analytics Instrumentation info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- |