diff options
author | Rémy Coutable <remy@rymai.me> | 2017-04-10 16:23:26 +0300 |
---|---|---|
committer | Rémy Coutable <remy@rymai.me> | 2017-04-10 17:40:33 +0300 |
commit | 2b606a3a2eb408c4c66a959f7efdfeeb76bb18e7 (patch) | |
tree | 828da5c87cef705cb24401fd42a3de3f7fc37f94 /doc/development/testing.md | |
parent | 0aafb6abb89b4b0fc4c033b21ccbfd1082b56b30 (diff) |
Re-organize testing doc, and add RSpec structure doc
Signed-off-by: Rémy Coutable <remy@rymai.me>
Diffstat (limited to 'doc/development/testing.md')
-rw-r--r-- | doc/development/testing.md | 243 |
1 files changed, 155 insertions, 88 deletions
diff --git a/doc/development/testing.md b/doc/development/testing.md index 0d29697df9e..e530b1c07bd 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -179,52 +179,9 @@ test includes: [picture]: https://twitter.com/withzombies/status/829716565834752000 [tests-cost]: https://medium.com/table-xi/high-cost-tests-and-high-value-tests-a86e27a54df#.2ulyh3a4e -## Factories +## Frontend testing -GitLab uses [factory_girl] as a test fixture replacement. - -- Factory definitions live in `spec/factories/`, named using the pluralization - of their corresponding model (`User` factories are defined in `users.rb`). -- There should be only one top-level factory definition per file. -- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and - should) call `create(...)` instead of `FactoryGirl.create(...)`. -- Make use of [traits] to clean up definitions and usages. -- When defining a factory, don't define attributes that are not required for the - resulting record to pass validation. -- When instantiating from a factory, don't supply attributes that aren't - required by the test. -- Factories don't have to be limited to `ActiveRecord` objects. - [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). - -[factory_girl]: https://github.com/thoughtbot/factory_girl -[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits - -## JavaScript - -GitLab uses [Karma] to run its [Jasmine] JavaScript specs. They can be run on -the command line via `bundle exec karma`. - -- JavaScript tests live in `spec/javascripts/`, matching the folder structure - of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` - has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file. -- Haml fixtures required for JavaScript tests live in - `spec/javascripts/fixtures`. They should contain the bare minimum amount of - markup necessary for the test. - - > **Warning:** Keep in mind that a Rails view may change and - invalidate your test, but everything will still pass because your fixture - doesn't reflect the latest view. Because of this we encourage you to - generate fixtures from actual rails views whenever possible. - -- Keep in mind that in a CI environment, these tests are run in a headless - browser and you will not have access to certain APIs, such as - [`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification), - which will have to be stubbed. - -[Karma]: https://github.com/karma-runner/karma -[Jasmine]: https://github.com/jasmine/jasmine - -For more information, see the [frontend testing guide](fe_guide/testing.md). +Please consult the [dedicated "Frontend testing" guide](./fe_guide/testing.md). ## RSpec @@ -296,6 +253,152 @@ end - Avoid scenario titles that add no information, such as "successfully". - Avoid scenario titles that repeat the feature title. +### Matchers + +Custom matchers should be created to clarify the intent and/or hide the +complexity of RSpec expectations.They should be placed under +`spec/support/matchers/`. Matchers can be placed in subfolder if they apply to +a certain type of specs only (e.g. features, requests etc.) but shouldn't be if +they apply to multiple type of specs. + +### Shared contexts + +All shared contexts should be be placed under `spec/support/shared_contexts/`. +Shared contexts can be placed in subfolder if they apply to a certain type of +specs only (e.g. features, requests etc.) but shouldn't be if they apply to +multiple type of specs. + +Each file should include only one context and have a descriptive name, e.g. +`spec/support/shared_contexts/controllers/githubish_import_controller_shared_context.rb`. + +### Shared examples + +All shared examples should be be placed under `spec/support/shared_examples/`. +Shared examples can be placed in subfolder if they apply to a certain type of +specs only (e.g. features, requests etc.) but shouldn't be if they apply to +multiple type of specs. + +Each file should include only one context and have a descriptive name, e.g. +`spec/support/shared_examples/controllers/githubish_import_controller_shared_example.rb`. + +### Helpers + +Helpers are usually modules that provide some methods to hide the complexity of +specific RSpec examples. You can define helpers in RSpec files if they're not +intended to be shared with other specs. Otherwise, they should be be placed +under `spec/support/helpers/`. Helpers can be placed in subfolder if they apply +to a certain type of specs only (e.g. features, requests etc.) but shouldn't be +if they apply to multiple type of specs. + +Helpers should follow the Rails naming / namespacing convention. For instance +`spec/support/helpers/cycle_analytics_helpers.rb` should define: + +```ruby +module Spec + module Support + module Helpers + module CycleAnalyticsHelpers + def create_commit_referencing_issue(issue, branch_name: random_git_name) + project.repository.add_branch(user, branch_name, 'master') + create_commit("Commit for ##{issue.iid}", issue.project, user, branch_name) + end + end + end + end +end +``` + +Helpers should not change the RSpec config. For instance, the helpers module +described above should not include: + +```ruby +RSpec.configure do |config| + config.include Spec::Support::Helpers::CycleAnalyticsHelpers +end +``` + +### Factories + +GitLab uses [factory_girl] as a test fixture replacement. + +- Factory definitions live in `spec/factories/`, named using the pluralization + of their corresponding model (`User` factories are defined in `users.rb`). +- There should be only one top-level factory definition per file. +- FactoryGirl methods are mixed in to all RSpec groups. This means you can (and + should) call `create(...)` instead of `FactoryGirl.create(...)`. +- Make use of [traits] to clean up definitions and usages. +- When defining a factory, don't define attributes that are not required for the + resulting record to pass validation. +- When instantiating from a factory, don't supply attributes that aren't + required by the test. +- Factories don't have to be limited to `ActiveRecord` objects. + [See example](https://gitlab.com/gitlab-org/gitlab-ce/commit/0b8cefd3b2385a21cfed779bd659978c0402766d). + +[factory_girl]: https://github.com/thoughtbot/factory_girl +[traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits + +### Fixtures + +All fixtures should be be placed under `spec/fixtures/`. + +### Config + +RSpec config files are files that change the RSpec config (i.e. +`RSpec.configure do |config|` blocks). They should be placed under +`spec/support/config/`. + +Each file should be related to a specific domain, e.g. +`spec/support/config/capybara.rb`, `spec/support/config/carrierwave.rb`, etc. + +Helpers can be included in the `spec/support/config/rspec.rb` file. If a +helpers module applies only to a certain kind of specs, it should add modifiers +to the `config.include` call. For instance if +`spec/support/helpers/cycle_analytics_helpers.rb` applies to `:lib` and +`type: :model` specs only, you would write the following: + +```ruby +RSpec.configure do |config| + config.include Spec::Support::Helpers::CycleAnalyticsHelpers, :lib + config.include Spec::Support::Helpers::CycleAnalyticsHelpers, type: :model +end +``` + +## Testing Rake Tasks + +To make testing Rake tasks a little easier, there is a helper that can be included +in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use +`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures +a few other things to make testing Rake tasks easier. + +At a minimum, requiring the Rake helper will redirect `stdout`, include the +runtime task helpers, and include the `RakeHelpers` Spec support module. + +The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make +executing tasks simple. See `spec/support/rake_helpers.rb` for all available +methods. + +Example: + +```ruby +require 'rake_helper' + +describe 'gitlab:shell rake tasks' do + before do + Rake.application.rake_require 'tasks/gitlab/shell' + + stub_warn_user_is_not_gitlab + end + + describe 'install task' do + it 'invokes create_hooks task' do + expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) + + run_rake_task('gitlab:shell:install') + end + end +end +``` + ## Test speed GitLab has a massive test suite that, without [parallelization], can take hours @@ -316,16 +419,7 @@ Here are some things to keep in mind regarding test performance: [parallelization]: #test-suite-parallelization-on-the-ci -### Monitoring - -The GitLab test suite is [monitored] and a [public dashboard] is available for -everyone to see. Feel free to look at the slowest test files and try to improve -them. - -[monitored]: /development/performance.html#rspec-profiling -[public dashboard]: https://redash.gitlab.com/public/dashboards/l1WhHXaxrCWM5Ai9D7YDqHKehq6OU3bx5gssaiWe?org_slug=default - -## Test suite parallelization on the CI +### Test suite parallelization on the CI Our current CI parallelization setup is as follows: @@ -352,41 +446,14 @@ After that, the next pipeline will use the up-to-date `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy is used for Spinach tests as well. -## Testing Rake Tasks - -To make testing Rake tasks a little easier, there is a helper that can be included -in lieu of the standard Spec helper. Instead of `require 'spec_helper'`, use -`require 'rake_helper'`. The helper includes `spec_helper` for you, and configures -a few other things to make testing Rake tasks easier. - -At a minimum, requiring the Rake helper will redirect `stdout`, include the -runtime task helpers, and include the `RakeHelpers` Spec support module. - -The `RakeHelpers` module exposes a `run_rake_task(<task>)` method to make -executing tasks simple. See `spec/support/rake_helpers.rb` for all available -methods. - -Example: - -```ruby -require 'rake_helper' - -describe 'gitlab:shell rake tasks' do - before do - Rake.application.rake_require 'tasks/gitlab/shell' - - stub_warn_user_is_not_gitlab - end +### Monitoring - describe 'install task' do - it 'invokes create_hooks task' do - expect(Rake::Task['gitlab:shell:create_hooks']).to receive(:invoke) +The GitLab test suite is [monitored] and a [public dashboard] is available for +everyone to see. Feel free to look at the slowest test files and try to improve +them. - run_rake_task('gitlab:shell:install') - end - end -end -``` +[monitored]: /development/performance.html#rspec-profiling +[public dashboard]: https://redash.gitlab.com/public/dashboards/l1WhHXaxrCWM5Ai9D7YDqHKehq6OU3bx5gssaiWe?org_slug=default ## Spinach (feature) tests |