diff options
author | mattab <matthieu.aubry@gmail.com> | 2013-10-02 08:45:33 +0400 |
---|---|---|
committer | mattab <matthieu.aubry@gmail.com> | 2013-10-02 08:45:33 +0400 |
commit | faf36f2038e37356529ce0dfa1dd71f95df6913d (patch) | |
tree | d221f08819d1bf8ea6a796a2f2d14ed7dacfbef7 /tests/README.md | |
parent | 588d71a8c5fa195d9906a82bbffacd94ba621a11 (diff) |
Updating README to link to http://piwik.org/blog/2013/10/our-latest-improvement-to-qa-screenshot-testing/
Diffstat (limited to 'tests/README.md')
-rw-r--r-- | tests/README.md | 72 |
1 files changed, 41 insertions, 31 deletions
diff --git a/tests/README.md b/tests/README.md index 5e4f0e3141..ca7f8d0441 100644 --- a/tests/README.md +++ b/tests/README.md @@ -1,6 +1,18 @@ Piwik comes with unit tests, integration tests, Javascript tests and Webtests. This document briefly describes how to use and modify Piwik tests. - + +## Continuous Integration + +We use Travis CI for our continuous integration server. It automatically runs our battery of thousands of unit/integration/screenshot tests +after each commit to our GIT repo. More information at the links: + + * Piwik on Travis CI: https://travis-ci.org/piwik/piwik + * QA in Piwik: http://piwik.org/qa/ + +Each core Piwik developer is responsible to keep the build green. If a developer breaks the build, he will receive an email from Travis CI. + +The next section explains how you can run the test suite on your own dev machine. + ## How To Run Piwik Tests To run tests, you must use the Git master. Tests files are not in the Piwik zip archive. @@ -21,24 +33,6 @@ To execute the tests: * Next you will need to install PHPUnit -## Integration Tests - -Integration tests files are in `tests/PHPUnit/Integration/*Test.php` - -Integration tests allow to test how major Piwik components interact together. -A test will typically generate hits to the Tracker (record visits and page views) -and then test all API responses and for each API output. It then checks that they match expected XML (or CSV, json, etc.). -If a test fails, you can compare the processed/ and expected/ directories in a graphical -text compare tool, such as WinMerge on Win, or MELD on Linux, to easily view changes between files. - -For example using Meld, click on "Start new comparison", "Directory comparison", -in "Original" select "path/to/piwik/tests/PHPUnit/Integration/expected" -in "Mine" select "path/to/piwik/tests/PHPUnit/Integration/processed" - -If changes are expected due to the code changes you make, simply copy the file from processed/ to -expected/, and test will then pass. Copying files is done easily using Meld (ALT+LEFT). -Otherwise, if you didn't expect to modify the API outputs, it might be that your changes are breaking some features unexpectedly. - ## PHPUnit Tests 1. Install PHPUnit on your system @@ -66,11 +60,10 @@ Otherwise, if you didn't expect to modify the API outputs, it might be that your 4. Run the tests (see the next section to run tests in the browser) $ cd /path/to/piwik/tests/PHPUnit - $ phpunit - - This will run all unit + integration tests. It might take 10-20 minutes to run. + $ phpunit --group Core + $ phpunit --group Plugins + $ phpunit --group Integration - You can also run tests of specified "parts" of Piwik. There are three main groups of tests: Core, Plugins and Integration For example run `phpunit --group Core` to run all Core Piwik tests. You may also combine groups like @@ -81,8 +74,8 @@ Otherwise, if you didn't expect to modify the API outputs, it might be that your ### PHP 5.5: also update PHPUnit to latest -See http://phpunit.de/manual/current/en/installation.html -or try: +See [PHPUnit update](http://phpunit.de/manual/current/en/installation.html) or try this command: + $ sudo pear install -a phpunit/PHPUnit ### Troubleshooting @@ -97,6 +90,26 @@ On your dev server, give your user permissions to write to the directory: $ sudo chmod 777 -R piwik/tmp/ + +## Integration Tests + +Integration tests files are in `tests/PHPUnit/Integration/*Test.php` + +Integration tests allow to test how major Piwik components interact together. +A test will typically generate hits to the Tracker (record visits and page views) +and then test all API responses and for each API output. It then checks that they match expected XML (or CSV, json, etc.). +If a test fails, you can compare the processed/ and expected/ directories in a graphical +text compare tool, such as WinMerge on Win, or MELD on Linux, to easily view changes between files. + +For example using Meld, click on "Start new comparison", "Directory comparison", +in "Original" select "path/to/piwik/tests/PHPUnit/Integration/expected" +in "Mine" select "path/to/piwik/tests/PHPUnit/Integration/processed" + +If changes are expected due to the code changes you make, simply copy the file from processed/ to +expected/, and test will then pass. Copying files is done easily using Meld (ALT+LEFT). +Otherwise, if you didn't expect to modify the API outputs, it might be that your changes are breaking some features unexpectedly. + + ## JavaScript Tests piwik.js is unit tested and you can run tests via piwik/tests/javascript/ @@ -200,13 +213,10 @@ Once you have expected screenshots, you can test the UI by running: $ cd PHPUnit $ phpunit UI -## Continuous Integration - -We run a Jenkins server for continuous integration. It automatically downloads the latest version of the Piwik code -from our GIT repo and runs a battery of thousands of tests. More information at the links: +** Learn more ** - * Official Piwik Jenkins Server: http://qa.piwik.org:8080/ - * QA in Piwik: http://piwik.org/qa/ +Check out this blog post to learn more about Screenshot Tests in Piwik: +[QA Screenshot Testing blog post](http://piwik.org/blog/2013/10/our-latest-improvement-to-qa-screenshot-testing/) ## VisualPHPUnit |