1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
|
# frozen_string_literal: true
# Blocks of text rendered in CLI
module InternalEventsCli
module Text
extend Helpers
CLI_INSTRUCTIONS = <<~TEXT.freeze
#{format_info('INSTRUCTIONS:')}
To start tracking usage of a feature...
1) Define event (using CLI)
2) Trigger event (from code)
3) Define metric (using CLI)
4) View data in Sisense (after merge & deploy)
This CLI will help you create the correct defintion files, then provide code examples for instrumentation and testing.
Learn more: https://docs.gitlab.com/ee/development/internal_analytics/#fundamental-concepts
TEXT
# TODO: Remove "NEW TOOL" comment after 3 months
FEEDBACK_NOTICE = format_heading <<~TEXT.chomp
Thanks for using the Internal Events CLI!
Please reach out with any feedback!
About Internal Events: https://gitlab.com/gitlab-org/analytics-section/analytics-instrumentation/internal/-/issues/687
About CLI: https://gitlab.com/gitlab-org/gitlab/-/issues/434038
In Slack: #g_analyze_analytics_instrumentation
Let us know that you used the CLI! React with 👍 on the feedback issue or post in Slack!
TEXT
ALTERNATE_RESOURCES_NOTICE = <<~TEXT.freeze
Other resources:
#{format_warning('Tracking GitLab feature usage from database info:')}
https://docs.gitlab.com/ee/development/internal_analytics/metrics/metrics_instrumentation.html#database-metrics
#{format_warning('Migrating existing metrics to use Internal Events:')}
https://docs.gitlab.com/ee/development/internal_analytics/internal_event_instrumentation/migration.html
#{format_warning('Remove an existing metric:')}
https://docs.gitlab.com/ee/development/internal_analytics/metrics/metrics_lifecycle.html
#{format_warning('Finding existing usage data for GitLab features:')}
https://metrics.gitlab.com/ (Customize Table > Sisense query)
https://app.periscopedata.com/app/gitlab/1049395/Service-Ping-Exploration-Dashboard
#{format_warning('Customer wants usage data for their own GitLab instance:')}
https://docs.gitlab.com/ee/user/analytics/
#{format_warning('Customer wants usage data for their own products:')}
https://docs.gitlab.com/ee/user/product_analytics/
TEXT
EVENT_TRACKING_EXAMPLES = <<~TEXT
Product usage can be tracked in several ways.
By tracking events: ex) a user changes the assignee on an issue
ex) a user uploads a CI template
ex) a service desk request is received
ex) all stale runners are cleaned up
ex) a user copies code to the clipboard from markdown
ex) a user uploads an issue template OR a user uploads an MR template
From database data: ex) track whether each gitlab instance allows signups
ex) query how many projects are on each gitlab instance
TEXT
EVENT_EXISTENCE_CHECK_INSTRUCTIONS = <<~TEXT.freeze
To determine what to do next, let's figure out if the event is already tracked & usable.
If you're unsure whether an event exists, you can check the existing defintions.
#{format_info('FROM GDK')}: Check `config/events/` or `ee/config/events`
#{format_info('FROM BROWSER')}: Check https://metrics.gitlab.com/snowplow
Find one? Create a new metric for the event.
Otherwise? Create a new event.
If you find a relevant event that has a different category from 'InternalEventTracking', it can be migrated to
Internal Events. See https://docs.gitlab.com/ee/development/internal_analytics/internal_event_instrumentation/migration.html
TEXT
EVENT_DESCRIPTION_INTRO = <<~TEXT.freeze
#{format_info('EVENT DESCRIPTION')}
Include what the event is supposed to track, where, and when.
The description field helps others find & reuse this event. This will be used by Engineering, Product, Data team, Support -- and also GitLab customers directly. Be specific and explicit.
ex - Debian package published to the registry using a deploy token
ex - Issue confidentiality was changed
TEXT
EVENT_DESCRIPTION_HELP = <<~TEXT.freeze
#{format_warning('Required. 10+ words likely, but length may vary.')}
#{format_info('GOOD EXAMPLES:')}
- Pipeline is created with a CI Template file included in its configuration
- Quick action `/assign @user1` used to assign a single individual to an issuable
- Quick action `/target_branch` used on a Merge Request
- Quick actions `/unlabel` or `/remove_label` used to remove one or more specific labels
- User edits file using the single file editor
- User edits file using the Web IDE
- User removed issue link between issue and incident
- Debian package published to the registry using a deploy token
#{format_info('GUT CHECK:')}
For your description...
1. Would two different engineers likely instrument the event from the same code locations?
2. Would a new GitLab user find where the event is triggered in the product?
3. Would a GitLab customer understand what the description says?
TEXT
EVENT_ACTION_INTRO = <<~TEXT.freeze
#{format_info('EVENT NAME')}
The event name is a unique identifier used from both a) app code and b) metric definitions.
The name should concisely communicate the same information as the event description.
ex - change_time_estimate_on_issue
ex - push_package_to_repository
ex - publish_go_module_to_the_registry_from_pipeline
ex - admin_user_comments_on_issue_while_impersonating_blocked_user
#{format_info('EXPECTED FORMAT:')} #{format_selection('<action>_<target_of_action>_<where/when>')}
ex) click_save_button_in_issue_description_within_15s_of_page_load
- TARGET: save button
- ACTION: click
- WHERE: in issue description
- WHEN: within 15s of page load
TEXT
EVENT_ACTION_HELP = <<~TEXT.freeze
#{format_warning('Required. Must be globally unique. Must use only letters/numbers/underscores.')}
#{format_info('FAQs:')}
- Q: Present tense or past tense?
A: Prefer present tense! But it's up to you.
- Q: Other event names have prefixes like `i_` or the `g_group_name`. Why?
A: Those are leftovers from legacy naming schemes. Changing the names of old events/metrics can break dashboards, so stability is better than uniformity.
TEXT
EVENT_IDENTIFIERS_INTRO = <<~TEXT.freeze
#{format_info('EVENT CONTEXT')}
Identifies the attributes recorded when the event occurs. Generally, we want to include every identifier available to us when the event is triggered.
#{format_info('BACKEND')}: Attributes must be specified when the event is triggered
ex) If the backend event was instrumentuser/project/namespace are the identifiers for this backend instrumentation:
Gitlab::InternalEvents.track_event(
'%s',
user: user,
project: project,
namespace: project.namespace
)
#{format_info('FRONTEND')}: Attributes are automatically included from the URL
ex) When a user takes an action on the MR list page, the URL is https://gitlab.com/gitlab-org/gitlab/-/merge_requests
Because this URL is for a project, we know that all of user/project/namespace are available for the event
#{format_info('NOTE')}: If you're planning to instrument a unique-by-user metric, you should still include project & namespace when possible. This is especially helpful in the data warehouse, where namespace and project can make events relevant for CSM use-cases.
TEXT
DATABASE_METRIC_NOTICE = <<~TEXT
For right now, this script can only define metrics for internal events.
For more info on instrumenting database-backed metrics, see https://docs.gitlab.com/ee/development/internal_analytics/metrics/metrics_instrumentation.html
TEXT
ALL_METRICS_EXIST_NOTICE = <<~TEXT
Looks like the potential metrics for this event either already exist or are unsupported.
Check out https://metrics.gitlab.com/ for improved event/metric search capabilities.
TEXT
METRIC_DESCRIPTION_INTRO = <<~TEXT.freeze
#{format_info('METRIC DESCRIPTION')}
Describes which occurrences of an event are tracked in the metric and how they're grouped.
The description field is critical for helping others find & reuse this event. This will be used by Engineering, Product, Data team, Support -- and also GitLab customers directly. Be specific and explicit.
#{format_info('GOOD EXAMPLES:')}
- Total count of analytics dashboard list views
- Weekly count of unique users who viewed the analytics dashboard list
- Monthly count of unique projects where the analytics dashboard list was viewed
- Total count of issue updates
#{format_info('SELECTED EVENT(S):')}
TEXT
METRIC_DESCRIPTION_HELP = <<~TEXT.freeze
#{format_warning('Required. 10+ words likely, but length may vary.')}
An event description can often be rearranged to work as a metric description.
ex) Event description: A merge request was created
Metric description: Total count of merge requests created
Metric description: Weekly count of unqiue users who created merge requests
Look at the event descriptions above to get ideas!
TEXT
end
end
|
