From fb8b0041d86080f0f8d53f13be1016b0b199a47f Mon Sep 17 00:00:00 2001 From: Robert Speicher Date: Wed, 30 Mar 2016 18:53:30 -0400 Subject: First pass at a Testing styleguide [ci skip] --- doc/development/README.md | 1 + doc/development/testing.md | 105 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) create mode 100644 doc/development/testing.md (limited to 'doc/development') diff --git a/doc/development/README.md b/doc/development/README.md index 1b281809afc..a8bc4fe5abb 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -9,4 +9,5 @@ - [Shell commands](shell_commands.md) in the GitLab codebase - [Sidekiq debugging](sidekiq_debugging.md) - [SQL guidelines](sql.md) for SQL guidelines +- [Testing standards and style guidelines](testing.md) - [UI guide](ui_guide.md) for building GitLab with existing css styles and elements diff --git a/doc/development/testing.md b/doc/development/testing.md new file mode 100644 index 00000000000..d37ef5d8785 --- /dev/null +++ b/doc/development/testing.md @@ -0,0 +1,105 @@ +# Testing Standards and Style Guidelines + +This guide outlines standards and best practices for automated testing of GitLab +CE and EE. + +It is meant to be an _extension_ of the [thoughtbot testing +styleguide](https://github.com/thoughtbot/guides/tree/master/style/testing). If +this guide defines a rule that contradicts the thoughtbot guide, this guide +takes precedence. Some guidelines may be repeated verbatim to stress their +importance. + +## Factories + +GitLab uses [factory_girl] as a test +fixture replacement. + +- Factory definitions live in `spec/factories/`, named using the pluralization + of their corresponding model (`User` factories are defined in `users.rb`). +- There should be only one top-level factory definition per file. +- Make use of [Traits] to clean up definitions and usages. +- When defining a factory, don't define attributes that are not required for the + resulting record to pass validation. +- When instantiating from a factory, don't supply extraneous attributes that + aren't required by the test. + +[factory_girl]: https://github.com/thoughtbot/factory_girl +[Traits]: http://www.rubydoc.info/gems/factory_girl/file/GETTING_STARTED.md#Traits + +## JavaScript + +GitLab uses [Teaspoon] to run its [Jasmine] JavaScript specs. They can be run on +the command line via `bundle exec teaspoon`, or via a web browser at +`http://localhost:3000/teaspoon` when the Rails server is running. + +- JavaScript tests live in `spec/javascripts/`, matching the folder structure of + `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js.coffee` has a corresponding + `spec/javascripts/behaviors/autosize_spec.js.coffee` file. +- Haml fixtures required for JavaScript tests live in + `spec/javascripts/fixtures`. They should contain the bare minimum amount of + markup necessary for the test. + + > **Warning:** Keep in mind that a Rails view may change and + invalidate your test, but everything will still pass because your fixture + doesn't reflect the latest view. + +- Keep in mind that in a CI environment, these tests are run in a headless + browser and you will not have access to certain APIs, such as + [`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification), + which will have to be stubbed. + +[Teaspoon]: https://github.com/modeset/teaspoon +[Jasmine]: https://github.com/jasmine/jasmine + +## RSpec + +### General Guidelines + +- Use a single, top-level `describe ClassName` block. +- Use `described_class` instead of repeating the class name being described. +- Use `.method` to describe class methods and `#method` to describe instance + methods. +- Use `context` to test branching logic. +- Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)). +- Prefer `not_to` to `to_not`. +- Try to match the ordering of tests to the ordering within the class. + +### Test speed + +GitLab has a massive test suite that, without parallelization, can take more +than an hour to run. It's important that we make an effort to write tests that +are accurate and effective _as well as_ fast. + +Here are some things to keep in mind regarding test performance: + +- `double` and `spy` are faster than `FactoryGirl.build(...)` +- `FactoryGirl.build(...)` and `.build_stubbed` are faster than `.create`. +- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`, + `spy`, or `double` will do. Database persistence is slow! +- Use `create(:empty_project)` instead of `create(:project)` when you don't need + the underlying repository. Filesystem operations are slow! +- Don't mark a feature as requiring JavaScript (through `@javascript` in + Spinach or `js: true` in RSpec) unless it's _actually_ required for the test + to be valid. Headless browser testing is slow! + +### Features / Integration + +- Feature specs live in `spec/features/` and should be named + `ROLE_ACTION_spec.rb`, such as `user_changes_password_spec.rb`. +- Use only one `feature` block per feature spec file. +- Use scenario titles that describe the success and failure paths. +- Avoid scenario titles that add no information, such as "successfully." +- Avoid scenario titles that repeat the feature title. + +## Spinach (feature) tests + +GitLab [moved from Cucumber to Spinach](https://github.com/gitlabhq/gitlabhq/pull/1426) +for its feature/integration tests in September 2012. + +As of March 2016, we are [trying to avoid adding new Spinach +tests](https://gitlab.com/gitlab-org/gitlab-ce/issues/14121) going forward, +opting for [RSpec feature](#features-integration) specs. + +Adding new Spinach scenarios is acceptable _only if_ the new scenario requires +no more than one new `step` definition. If more than that is required, the +test should be re-implemented using RSpec instead. -- cgit v1.2.3