diff options
-rw-r--r-- | doc/development/README.md | 1 | ||||
-rw-r--r-- | doc/development/benchmarking.md | 69 |
2 files changed, 70 insertions, 0 deletions
diff --git a/doc/development/README.md b/doc/development/README.md index 6bc8e1888db..d5bf166ad32 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -8,3 +8,4 @@ - [UI guide](ui_guide.md) for building GitLab with existing css styles and elements - [Migration Style Guide](migration_style_guide.md) for creating safe migrations - [How to dump production data to staging](dump_db.md) +- [Benchmarking](benchmarking.md) diff --git a/doc/development/benchmarking.md b/doc/development/benchmarking.md new file mode 100644 index 00000000000..88e18ee95f9 --- /dev/null +++ b/doc/development/benchmarking.md @@ -0,0 +1,69 @@ +# Benchmarking + +GitLab CE comes with a set of benchmarks that are executed for every build. This +makes it easier to measure performance of certain components over time. + +Benchmarks are written as RSpec tests using a few extra helpers. To write a +benchmark, first tag the top-level `describe`: + +```ruby +describe MaruTheCat, benchmark: true do + +end +``` + +This ensures the benchmark is executed separately from other test collections. +It also exposes the various RSpec matchers used for writing benchmarks to the +test group. + +Next, lets write the actual benchmark: + +```ruby +describe MaruTheCat, benchmark: true do + let(:maru) { MaruTheChat.new } + + describe '#jump_in_box' do + benchmark_subject { maru.jump_in_box } + + it { is_expected.to iterate_per_second(9000) } + end +end +``` + +Here `benchmark_subject` is a small wrapper around RSpec's `subject` method that +makes it easier to specify the subject of a benchmark. Using RSpec's regular +`subject` would require us to write the following instead: + +```ruby +subject { -> { maru.jump_in_box } } +``` + +The `iterate_per_second` matcher defines the amount of times per second a +subject should be executed. The higher the amount of iterations the better. + +By default the allowed standard deviation is a maximum of 30%. This can be +adjusted by chaining the `with_maximum_stddev` on the `iterate_per_second` +matcher: + +```ruby +it { is_expected.to iterate_per_second(9000).with_maximum_stddev(50) } +``` + +This can be useful if the code in question depends on external resources of +which the performance can vary a lot (e.g. physical HDDs, network calls, etc). +However, in most cases 30% should be enough so only change this when really +needed. + +## Benchmarks Location + +Benchmarks should be stored in `spec/benchmarks` and should follow the regular +Rails specs structure. That is, model benchmarks go in `spec/benchmark/models`, +benchmarks for code in the `lib` directory go in `spec/benchmarks/lib`, etc. + +## Underlying Technology + +The benchmark setup uses [benchmark-ips][benchmark-ips] which takes care of the +heavy lifting such as warming up code, calculating iterations, standard +deviation, etc. + +[benchmark-ips]: https://github.com/evanphx/benchmark-ips |