Welcome to mirror list, hosted at ThFree Co, Russian Federation.

gitlab.com/gitlab-org/gitlab-foss.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat (limited to 'doc/development/experiment_guide/testing_experiments.md')
-rw-r--r--doc/development/experiment_guide/testing_experiments.md150
1 files changed, 150 insertions, 0 deletions
diff --git a/doc/development/experiment_guide/testing_experiments.md b/doc/development/experiment_guide/testing_experiments.md
new file mode 100644
index 00000000000..08ff91a3deb
--- /dev/null
+++ b/doc/development/experiment_guide/testing_experiments.md
@@ -0,0 +1,150 @@
+---
+stage: Growth
+group: Activation
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+---
+
+# Testing experiments
+
+## Testing experiments with RSpec
+
+In the course of working with experiments, you'll probably want to utilize the RSpec
+tooling that's built in. This happens automatically for files in `spec/experiments`, but
+for other files and specs you want to include it in, you can specify the `:experiment` type:
+
+```ruby
+it "tests experiments nicely", :experiment do
+end
+```
+
+### Stub helpers
+
+You can stub experiments using `stub_experiments`. Pass it a hash using experiment
+names as the keys, and the variants you want each to resolve to, as the values:
+
+```ruby
+# Ensures the experiments named `:example` & `:example2` are both "enabled" and
+# that each will resolve to the given variant (`:my_variant` and `:control`
+# respectively).
+stub_experiments(example: :my_variant, example2: :control)
+
+experiment(:example) do |e|
+ e.enabled? # => true
+ e.assigned.name # => 'my_variant'
+end
+
+experiment(:example2) do |e|
+ e.enabled? # => true
+ e.assigned.name # => 'control'
+end
+```
+
+### Exclusion, segmentation, and behavior matchers
+
+You can also test things like the registered behaviors, the exclusions, and
+segmentations using the matchers.
+
+```ruby
+class ExampleExperiment < ApplicationExperiment
+ control { }
+ candidate { '_candidate_' }
+
+ exclude { context.actor.first_name == 'Richard' }
+ segment(variant: :candidate) { context.actor.username == 'jejacks0n' }
+end
+
+excluded = double(username: 'rdiggitty', first_name: 'Richard')
+segmented = double(username: 'jejacks0n', first_name: 'Jeremy')
+
+# register_behavior matcher
+expect(experiment(:example)).to register_behavior(:control)
+expect(experiment(:example)).to register_behavior(:candidate).with('_candidate_')
+
+# exclude matcher
+expect(experiment(:example)).to exclude(actor: excluded)
+expect(experiment(:example)).not_to exclude(actor: segmented)
+
+# segment matcher
+expect(experiment(:example)).to segment(actor: segmented).into(:candidate)
+expect(experiment(:example)).not_to segment(actor: excluded)
+```
+
+### Tracking matcher
+
+Tracking events is a major aspect of experimentation. We try
+to provide a flexible way to ensure your tracking calls are covered.
+
+You can do this on the instance level or at an "any instance" level:
+
+```ruby
+subject = experiment(:example)
+
+expect(subject).to track(:my_event)
+
+subject.track(:my_event)
+```
+
+You can use the `on_next_instance` chain method to specify that it will happen
+on the next instance of the experiment. This helps you if you're calling
+`experiment(:example).track` downstream:
+
+```ruby
+expect(experiment(:example)).to track(:my_event).on_next_instance
+
+experiment(:example).track(:my_event)
+```
+
+A full example of the methods you can chain onto the `track` matcher:
+
+```ruby
+expect(experiment(:example)).to track(:my_event, value: 1, property: '_property_')
+ .on_next_instance
+ .with_context(foo: :bar)
+ .for(:variant_name)
+
+experiment(:example, :variant_name, foo: :bar).track(:my_event, value: 1, property: '_property_')
+```
+
+## Test with Jest
+
+### Stub Helpers
+
+You can stub experiments using the `stubExperiments` helper defined in `spec/frontend/__helpers__/experimentation_helper.js`.
+
+```javascript
+import { stubExperiments } from 'helpers/experimentation_helper';
+import { getExperimentData } from '~/experimentation/utils';
+
+describe('when my_experiment is enabled', () => {
+ beforeEach(() => {
+ stubExperiments({ my_experiment: 'candidate' });
+ });
+
+ it('sets the correct data', () => {
+ expect(getExperimentData('my_experiment')).toEqual({ experiment: 'my_experiment', variant: 'candidate' });
+ });
+});
+```
+
+NOTE:
+This method of stubbing in Jest specs will not automatically un-stub itself at the end of the test. We merge our stubbed experiment in with all the other global data in `window.gl`. If you need to remove the stubbed experiments after your test or ensure a clean global object before your test, you'll need to manage the global object directly yourself:
+
+```javascript
+describe('tests that care about global state', () => {
+ const originalObjects = [];
+
+ beforeEach(() => {
+ // For backwards compatibility for now, we're using both window.gon & window.gl
+ originalObjects.push(window.gon, window.gl);
+ });
+
+ afterEach(() => {
+ [window.gon, window.gl] = originalObjects;
+ });
+
+ it('stubs experiment in fresh global state', () => {
+ stubExperiment({ my_experiment: 'candidate' });
+ // ...
+ });
+})
+```