diff options
Diffstat (limited to 'web/testing')
| -rw-r--r-- | web/testing | 237 |
1 files changed, 232 insertions, 5 deletions
diff --git a/web/testing b/web/testing index f97902ec306..ab60d7fe4d3 100644 --- a/web/testing +++ b/web/testing @@ -1,21 +1,168 @@ * Testing + Daily <a href="http://www.go-mono.com/tests/index.php">test</a> results. + Testing is an important part of the Mono project: every one of its three major components has a test suite tailored for its needs. This is very helpful, because in the course of developing the software it is very common to introduce bugs in existing code. A test suite helps us fix the bugs as soon as they are introduced. -** Class Library Tests + There are various kinds of tests in Mono: + <ul> + <li><a href="#unit"><b>Class Library Unit + Tests:</b></a> These are used to test the class + libraries. + + <li><a href="#compiler"><b>Compiler tests</b></a>: Both + tests that should pass and tests that should fail are included. + + <li><a href="#runtime"><b>Runtime tests</b></a>: Tests for + the virtual machine. + + <li><a href="#aspnet"><b>ASP.NET tests</b></a>: ASP.NET tests. + + <li><a href="#ws"><b>Web Services tests</b></a>: Web Services + client/server tests. + </ul> + +<a name="unit"></a> +* Class Library Tests All classes in Mono libraries should have comprehensive unit test - suites to go with them. Unit testing is a software engineering - methodology that makes it easier to build correct code. Every + suites to go with them. Unit testing is a software engineering + methodology that makes it easier to build correct code. Every method in every class should have a set of tests to verify - that they work correctly. Mono also needs a testing framework + that they work correctly. Mono also needs a testing framework to make it easy to write and run lots of tests. - Try <a href="http://nunit.sourceforge.net">NUnit</a> + In some classes, we might also provide standalone tests because of + some reasons such as too huge testcases, another downloading and so on. + (For example, managed XSLT has standalone test which downloads and + expands some megabytes of OASIS test suite.) + + Here I list them up as long as I know. If you are going to add another + standalone tests, please add one line here. It is also recommended that + you add some notes on how to build and run tests. + + <ul> + + * Mono.Data/test/ + * System.Data/Test, and some individual ADO.NET libraries: + there are some standalone tests. See the bottom of <a href="ado-net.html"> + ADO.NET page</a> for detail. + * System.Web/Test/TestMonoWeb : see README + * System.Web.Services/Test/standalone : see README + * System.Windows.Forms/SWFTest/ + * System.XML/Test/System.Xml/standalone_tests : see README + * System.XML/Test/System.Xml.Schema/standalone_tests : see README + * System.XML/System.Xml.Serialization/standalone_tests/ + * System.XML/Test/System.Xml.Xsl/standalone_tests : see README + * Commons.Xml.Relaxng/Test/standalone_tests : see README + + </ul> + +** Getting started + + If you are new to writing NUnit tests, there is a template you may use + to help get started. The file is: + + <b>mcs/class/doc/TemplateTest.cs</b> + + Save a copy of this file in the appropriate test subdirecty + (see below), and replace all the {text} markers with + appropriate code. Comments in the template are there to guide + you. You should also look at existing tests to see how other + people have written them. + mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs + is a small one that might help. + + The directory that will contain your new file depends on the + assembly/namespace of the class for which you are creating the + tests. Under mcs/class there is a directory for each + assembly. In each assembly there is a Test directory, + e.g. mcs/class/corlib/Test. In the Test directory there are + sub-directories for each namespace in the assembly, + e.g. mcs/class/corlib/Test/Sytem. Put your new test file in + the appropriate sub-directory under Test for the class you are + testing. + + Once all of that is done, you can do a 'make test' from the top mcs + directory. Your test class needs also to be listed in the + .sources file at the top of the Test directory. + +* Tips on writing Unit tests. + + You should look at the <a href="http://nunit.org">NUnit documentation</a>, + as it is a fantastic product, and includes fantastic documentation, + but here are some tips for those of you who are already reading + this web page. + + +** Provide an unique error message for Assert() + + Include an unique message for each Assert() so that when the assert + fails, it is trivial to locate it in the source. Otherwise, it may be + difficult to determine which part of the test is failing. A good way + to ensure unique messages is to use something like #A01, #A02 etc. + + Ok: + <pre> + + AssertEquals("array match", compare[0], i1[0]); + AssertEquals("array match", compare[1], i1[1]); + AssertEquals("array match", compare[2], i1[2]); + AssertEquals("array match", compare[3], i1[3]); + </pre> + + Excellent: + <pre> + AssertEquals("#A01", compare[0], i1[0]); + AssertEquals("#A02", compare[1], i1[1]); + AssertEquals("#A03", compare[2], i1[2]); + AssertEquals("#A04", compare[3], i1[3]); + </pre> + + Once you used such a number in an Assert(), don't change it later on - + people might use it it identify the test in bug reports or in mailing + lists. + +** Use AssertEquals() to compare things, not Assert(). + + Do not compare two values with Assert() - if the test fails, + people have no idea what went wrong while AssertEquals() + reports the failed value. + + Ok: + <pre> + Assert ("A01", myTicks[0] == t1.Ticks); + </pre> + + Excellent: + <pre> + AssertEquals ("A01", myTicks[0], t1.Ticks); + </pre> + +** Test your test with the Microsoft runtime + + If possible, try to run your testsuite with the Microsoft runtime on + .NET on Windows and make sure all tests in it pass. This is especially + important if you're writing a totally new testcase - without this + check you can never be sure that your testcase contains no bugs .... + + Don't worry if you're writing your test on Linux, other people can + test it for you on Windows. + + Sometimes you may discover that a test doesn't show the expected + result when run with the Microsoft runtime - either because there is a + bug in their runtime or something is misleading or wrong in their + documentation. In this case, please put a detailed description of the + problem to mcs/class/doc/API-notes and do also report it to the + <a href="mailing-lists.html">mailing list</a> - we'll forward this to the + Microsoft people from time to time to help them fix their documentation + and runtime. + +** Unit tests. Why do unit testing? It becomes simple to run automated tests for the whole library. Unit tests are a safety net - you can @@ -44,3 +191,83 @@ Normally, after you send a couple of well-written new files and/or patches to the list, you will be given cvs access. +<a name="compiler"></a> +* Compiler tests + + Mono ships with three compilers: C#, VB.NET and JScript. The + tests are ran by running the makefile target `make + run-test-local' in the appropriate directory. + + The C# compilation tests live in mcs/tests, and the C# error + tests live in mcs/errors. + + The VB.NET compilation tests live in mcs/btests. + +<a name="runtime"></a> +* Runtime Tests + + These tests verify the virtual machine, to run these tests, do: + +<pre> + cd mono/mono/tests + make test +</pre> + +<a name="aspnet"></a> +* ASP.NET tests + + XSP, the Mono ASP.NET server has tests for ASP.NET pages. It uses + <a href="http://nunitasp.sourceforge.net">NUnitAsp</a>. Right now + it only has standalone tests, ie., tests that do not need their own + global.asax or web.config files. + + If you want to run them, get the xsp CVS module and install it. Then: +<pre> + cd xsp/nunit-tests + make + cd standalone + xsp +</pre> + + And from another terminal: +<pre> + cd xsp/nunit-tests/standalone + nunit-console standalone-tests.dll +</pre> + +<a name="ws"></a> +* Web Services tests + + The Test directory for the System.Web.Services assembly contains a + standalone test suite for testing web services. It tests: + + <ul> + <li>Proxy generation using the wsdl tool</li> + <li>Access to web services using the generated client proxies</li> + <li>Execution of web services in the server</li> + </ul> + + This suite not only tests web services running on XSP, but it can also test + services running on other platforms and that are available in internet. This + will help track down interoperability issues. + + To build the test suite, just run: + +<pre> + cd mcs/class/System.Web.Services/Test/standalone + xsp --root server +</pre> + + And from another terminal: +<pre> + cd mcs/class/System.Web.Services/Test/standalone + make + nunit-console testclient.dll +</pre> + + This will download the wsdl documents, generate the proxies, build a dll with + the proxies, and build the nunit tests. Then you can use nunit-console or + gnunit to run the tests (the nunit dll is testclient.dll). + + Read the README file in mcs/class/System.Web.Services/Test/standalone for + more info. |