diff options
| -rw-r--r-- | tests/README.md (renamed from tests/README.txt) | 162 |
1 files changed, 93 insertions, 69 deletions
diff --git a/tests/README.txt b/tests/README.md index 06ac2f0c44..ecf9aaf367 100644 --- a/tests/README.txt +++ b/tests/README.md @@ -1,21 +1,25 @@ Piwik comes with unit tests, integration tests, Javascript tests and Webtests. This document briefly describes how to use and modify Piwik tests. - HOW TO RUN PIWIK TESTS -======================= -To run tests, you must use the SVN trunk. Tests files are not in the Piwik zip archive. -You can get the latest SVN at: http://dev.piwik.org/svn/ - $ svn co http://dev.piwik.org/svn/trunk/ +## How To Run Piwik Tests + +To run tests, you must use the Git master. Tests files are not in the Piwik zip archive. +You can get the latest Git revision at: http://github.com/piwik/piwik + +``` +$ git clone https://github.com/piwik/piwik.git +``` To execute the tests: + * In your php.ini make sure you have the setting to show all errors: - error_reporting = E_ALL | E_STRICT + `error_reporting = E_ALL | E_STRICT` * Go to tests/index.php to see the tests homepage and run the Integration tests via a visual UI, or run JS Tests - INTEGRATION TESTS -================== -Integration tests files are in tests/PHPUnit/Integration/*Test.php +## Integration Tests + +Integration tests files are in `tests/PHPUnit/Integration/*Test.php` Integration tests allow to test how major Piwik components interact together. A test will typically generate hits to the Tracker (record visits and page views) @@ -26,24 +30,33 @@ If changes are expected due to the code changes you make, simply copy the file f expected/, and test will then pass. Otherwise, if you didn't expect to modify the API outputs, it might be that your changes are breaking some features unexpectedly. - PHPUNIT TESTS -============== -1) install PHPUnit on your system - $ cd your/php/directory - $ sudo pear upgrade PEAR - $ pear config-set auto_discover 1 - $ sudo pear install --alldeps pear.phpunit.de/PHPUnit +## PHPUnit Tests + +1. Install PHPUnit on your system + + ``` + $ cd your/php/directory + $ sudo pear upgrade PEAR + $ pear config-set auto_discover 1 + $ sudo pear install --alldeps pear.phpunit.de/PHPUnit + ``` + Doc at: http://www.phpunit.de/manual/current/en/installation.html -2) Configure PHPUnit: Copy the file piwik/tests/PHPUnit/phpunit.xml.dist as phpunit.xml. +2. Configure PHPUnit: Copy the file `piwik/tests/PHPUnit/phpunit.xml.dist` to `phpunit.xml`. In this file, you will find the following lines. Please edit HTTP_HOST and REQUEST_URI to match the hostname and path of the Piwik files: - <server name="HTTP_HOST" value="localhost"/> - <server name="REQUEST_URI" value="/path/to/piwik/tests/all_tests.php"/> + + <server name="HTTP_HOST" value="localhost"/> + <server name="REQUEST_URI" value="/path/to/piwik/tests/all_tests.php"/> -3) Run the tests (see the next section to run tests in the browser) - $ cd /path/to/piwik/tests/PHPUnit - $ phpunit +3. Run the tests (see the next section to run tests in the browser) + + ``` + $ cd /path/to/piwik/tests/PHPUnit + $ phpunit + ``` + This will run all unit + integration tests. It might take 30 minutes to run. You can also run tests of specified "parts" of Piwik. @@ -53,35 +66,40 @@ it might be that your changes are breaking some features unexpectedly. to run all Core Piwik tests. You may also combine groups like $ phpunit --group Core,Plugins -4) Write more tests :) +4. Write more tests :) See "Writing Unit tests with PHPUnit" http://www.phpunit.de/manual/current/en/writing-tests-for-phpunit.html - JAVASCRIPT TESTS -================= +## JavaScript Tests + piwik.js is unit tested and you can run tests via piwik/tests/javascript/ - TESTING DATA -============= -You can import data over several days in Piwik -1) Install Piwik -2) Create a site with URL http://piwik.org/ -3) Create a Goal eg. URL Contains "blog" -4) Import data from an anonimized test log file in piwik/tests/resources/ directory. Run the following command: - $ python /home/piwik/misc/log-analytics/import_logs.py --url=http://localhost/path/ /path/to/piwik/tests/resources/access.log-dev-anon-9-days-nov-2012.log.bz2 --idsite=1 --enable-http-errors --enable-http-redirects --enable-static --enable-bots - This will import 9 days worth of data from Nov 20th-Nov 29th 2012. -5) You can then archive the reports with: - $ php5 /home/piwik/misc/cron/archive.php --url=http://localhost/path/ +## Testing Data + +You can import data over several days in Piwik: + +1. Install Piwik +2. Create a site with URL http://piwik.org/ +3. Create a Goal eg. URL Contains "blog" +4. Import data from an anonimized test log file in piwik/tests/resources/ directory. Run the following command: + + $ python /home/piwik/misc/log-analytics/import_logs.py --url=http://localhost/path/ /path/to/piwik/tests/resources/access.log-dev-anon-9-days-nov-2012.log.bz2 --idsite=1 --enable-http-errors --enable-http-redirects --enable-static --enable-bots + + This will import 9 days worth of data from Nov 20th-Nov 29th 2012. + +5. You can then archive the reports with: + + $ php5 /home/piwik/misc/cron/archive.php --url=http://localhost/path/ You should now have some interesting data to test with in November 2012! - WEBTESTS -========= +## Webtests + We would like to add webtests testing installation, auto update, and initial user login. Task is tracked in: http://dev.piwik.org/trac/ticket/2935 - SCHEDULED REPORTS TESTS -======================== +## Schedules Reports Tests + Piwik scheduled reports (HTML, PDF & SMS) are part of the integration test suite. They follow the same principles described in the INTEGRATION TESTS section of this document. @@ -108,25 +126,27 @@ work altered the expected images. The standard procedure described in the INTEGR - set up the vagrant piwik vm (which is used by the integration server) or - retrieve the files from the integration server. - CONTINOUS INTEGRATION -====================== +## Continous Integration + We run a Jenkins server for continuous integration. It automatically downloads the latest version of the Piwik code from our SVN server and runs a battery of thousands of tests. More information at the links: + * Official Piwik Jenkins Server: http://qa.piwik.org:8080/ * QA in Piwik: http://piwik.org/qa/ - VISUALPHPUNIT -================= +## VisualPHPUnit + Piwik comes with a modified copy of VisualPHPUnit (see https://github.com/NSinopoli/VisualPHPUnit) which you can use to run PHPUnit tests in your browser. -- Starting VisualPHPUnit - +### Starting VisualPHPUnit To load VisualPHPUnit point your browser to http://path/to/piwik/trunk/tests/lib/visualphpunit/. VisualPHPUnit will already be configured for use with Piwik. Troubleshooting + * If at this URL you see a listing of files instead of seeing VisualPHPUnit, enable mod_rewrite apache module, and make sure your vhost in apache configuration has "AllowOverride all" so that .htaccess are loaded. @@ -135,7 +155,7 @@ Troubleshooting is not set in your php.ini. Edit in php.ini the value include_path to include the path to your PEAR setup, and restart Apache. -- Running tests - +### Running tests Once VisualPHPUnit is loaded, you can run tests by selecting files or whole directories in the file selector on the left of the screen, then clicking the 'Run tests' button. To select @@ -144,7 +164,7 @@ files/directories ctrl+click them. To run all Piwik tests, ctrl+click the 'Core', 'Integration' and 'Plugins' directory, then click the 'Run tests' button. -- Running tests by URL - +### Running tests by URL If you're in need of a URL that will not only load VisualPHPUnit but run one or more tests, you may add the list of tests to run as the hash of the URL. For example, @@ -154,7 +174,7 @@ http://path/to/piwik/trunk/tests/lib/visualphpunit/#/Core/DataTableTest.php:/Cor will load VisualPHPUnit and immediately run the tests in DataTableTest.php and CookieTest.php. Currently, this feature will not allow you to specify directories with tests to run. -- Using phpunit.xml - +### Using phpunit.xml By default, VisualPHPUnit lets you run tests by selecting individual test files or directories and clicking the 'Run Tests' button. If you want to use a phpunit.xml file, either your own or the @@ -169,7 +189,7 @@ Please note that when a phpunit.xml file is supplied in the configuration, Visua always run tests with it, regardless of what files you select. You can override this behavior in the web UI by selecting 'No' in the 'Use XML Config' input. -- Debugging invalid responses - +### Debugging invalid responses Sometimes, VisualPHPUnit will run PHPUnit tests and get a response it can't read. These problems are usually caused by an unmatched ob_start() call in the code somewhere, or by the program @@ -179,43 +199,46 @@ To find the cause of such issues, it can help to determine what code can & can't output VisualPHPUnit sees. Code that can affect what VisualPHPUnit sees is before the bug in question, and code that can't is after it. - BENCHMARKS -================= +## Benchmarks + See tests/PHPUnit/Benchmarks/README.txt - PROFILING WITH XHPROF -====================== +## Profiling with XHProf Piwik is distributed with a copy of XHProf, the PHP profiler created by Facebook. Piwik also comes with a copy of VisualPHPUnit that has been modified to easily use XHProf. Using these two tools, you can profile tests and benchmarks. -- Installing XHProf - +### Installing XHProf First, XHProf must be built (this guide assumes you're using a linux variant): - * Navigate to the XHProf extension directory. - $ cd /path/to/piwik/trunk/tests/lib/xhprof-0.9.2/extension + * Navigate to the XHProf extension directory. + + $ cd /path/to/piwik/trunk/tests/lib/xhprof-0.9.2/extension - * Build XHProf. - $ ./configure - $ make + * Build XHProf. + + $ ./configure + $ make - xhprof.so will now exist in the ./modules directory. + xhprof.so will now exist in the ./modules directory. - * Configure PHP to use XHProf. Add the following to your php.ini file: + * Configure PHP to use XHProf. Add the following to your php.ini file: - [xhprof] - extension=/path/to/piwik/trunk/tests/lib/xhprof-0.9.2/extension/modules/xhprof.so - xhprof.output_dir=/path/to/output/dir + ``` + [xhprof] + extension=/path/to/piwik/trunk/tests/lib/xhprof-0.9.2/extension/modules/xhprof.so + xhprof.output_dir=/path/to/output/dir + ``` - Replace /path/to/output/dir with an existing directory. All your profiles will be - stored there. + Replace /path/to/output/dir with an existing directory. All your profiles will be + stored there. Restart your webserver and you're done. VisualPHPUnit will automatically detect if XHProf is installed and act accordingly. -- Using XHProf - +### Using XHProf To use XHProf, first load VisualPHPUnit by pointing your browser to: @@ -228,13 +251,14 @@ When the test finishes, a link will be displayed that will let you view the prof was created. NOTE: + * Currently, it is not possible to use XHProf with more than one test, so if multiple tests are selected, XHProf will not be used. * XHProf will not delete old profiles, you must do that yourself, though individual profiles do not take much space. - PARTICIPATE -============ +## Participate + You can help by improving existing tests, or identify some missing tests and implement them. See http://piwik.org/participate/development-process Please contact us at hello@piwik.org |