diff options
Diffstat (limited to 'doc/development/service_ping/implement.md')
| -rw-r--r-- | doc/development/service_ping/implement.md | 77 |
1 files changed, 14 insertions, 63 deletions
diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md index 3a1e4c6d87b..25e841e113b 100644 --- a/doc/development/service_ping/implement.md +++ b/doc/development/service_ping/implement.md @@ -16,7 +16,7 @@ Service Ping consists of two kinds of data: To implement a new metric in Service Ping, follow these steps: 1. [Implement the required counter](#types-of-counters) -1. [Name and place the metric](#name-and-place-the-metric) +1. [Name and place the metric](metrics_dictionary.md#metric-key_path) 1. [Test counters manually using your Rails console](#test-counters-manually-using-your-rails-console) 1. [Generate the SQL query](#generate-the-sql-query) 1. [Optimize queries with `#database-lab`](#optimize-queries-with-database-lab) @@ -26,13 +26,12 @@ To implement a new metric in Service Ping, follow these steps: 1. [Verify your metric](#verify-your-metric) 1. [Set up and test Service Ping locally](#set-up-and-test-service-ping-locally) -NOTE: -When you add or change a Service Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md). -For information about the progress on migrating Service ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547). - ## Instrumentation classes -We recommend you use [instrumentation classes](metrics_instrumentation.md) in `usage_data.rb` where possible. +NOTE: +Implementing metrics directly in `usage_data.rb` is deprecated. +When you add or change a Service Ping Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md). +For information about the progress on migrating Service Ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547). For example, we have the following instrumentation class: `lib/gitlab/usage/metrics/instrumentations/count_boards_metric.rb`. @@ -45,7 +44,7 @@ boards: add_metric('CountBoardsMetric', time_frame: 'all'), ## Types of counters -There are several types of counters in `usage_data.rb`: +There are several types of counters for metrics: - **[Batch counters](#batch-counters)**: Used for counts and sums. - **[Redis counters](#redis-counters):** Used for in-memory counts. @@ -72,64 +71,39 @@ you must add a specialized index on the columns involved in a counter. #### Ordinary batch counters -Simple count of a given `ActiveRecord_Relation`, does a non-distinct batch count, smartly reduces `batch_size`, and handles errors. -Handles the `ActiveRecord::StatementInvalid` error. +Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `count` operation for a given `ActiveRecord_Relation` Method: ```ruby -count(relation, column = nil, batch: true, start: nil, finish: nil) +add_metric('CountIssuesMetric', time_frame: 'all') ``` -Arguments: - -- `relation` the ActiveRecord_Relation to perform the count -- `column` the column to perform the count on, by default is the primary key -- `batch`: default `true` to use batch counting -- `start`: custom start of the batch counting to avoid complex min calculations -- `end`: custom end of the batch counting to avoid complex min calculations - Examples: -```ruby -count(User.active) -count(::Clusters::Cluster.aws_installed.enabled, :cluster_id) -count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id)) -``` +Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md). #### Distinct batch counters -Distinct count of a given `ActiveRecord_Relation` on given column, a distinct batch count, smartly reduces `batch_size`, and handles errors. -Handles the `ActiveRecord::StatementInvalid` error. +Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `distinct_count` operation for a given `ActiveRecord_Relation`. Method: ```ruby -distinct_count(relation, column = nil, batch: true, batch_size: nil, start: nil, finish: nil) +add_metric('CountUsersAssociatingMilestonesToReleasesMetric', time_frame: 'all') ``` -Arguments: - -- `relation`: the ActiveRecord_Relation to perform the count -- `column`: the column to perform the distinct count, by default is the primary key -- `batch`: default `true` to use batch counting -- `batch_size`: if none set it uses default value 10000 from `Gitlab::Database::BatchCounter` -- `start`: custom start of the batch counting to avoid complex min calculations -- `end`: custom end of the batch counting to avoid complex min calculations - WARNING: Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../iterating_tables_in_batches.md) guide. Examples: -```ruby -distinct_count(::Project, :creator_id) -distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id)) -distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id') -``` +Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md). #### Sum batch operation +There is no support for `sum` for database metrics. + Sum the values of a given ActiveRecord_Relation on given column and handles errors. Handles the `ActiveRecord::StatementInvalid` error @@ -686,29 +660,6 @@ We return fallback values in these cases: | Timeouts, general failures | -1 | | Standard errors in counters | -2 | -## Name and place the metric - -Add the metric in one of the top-level keys: - -- `settings`: for settings related metrics. -- `counts_weekly`: for counters that have data for the most recent 7 days. -- `counts_monthly`: for counters that have data for the most recent 28 days. -- `counts`: for counters that have data for all time. - -### How to get a metric name suggestion - -The metric YAML generator can suggest a metric name for you. -To generate a metric name suggestion, first instrument the metric at the provided `key_path`. -Then, generate the metric's YAML definition and -return to the instrumentation and update it. - -1. Add the metric instrumentation to `lib/gitlab/usage_data.rb` inside one - of the [top-level keys](#name-and-place-the-metric), using any name you choose. -1. Run the [metrics YAML generator](metrics_dictionary.md#metrics-definition-and-validation). -1. Use the metric name suggestion to select a suitable metric name. -1. Update the instrumentation you created in the first step and change the metric name to the suggested name. -1. Update the metric's YAML definition with the correct `key_path`. - ## Test counters manually using your Rails console ```ruby |