Last Update: 2002-03-02
Rev: 0.3
Purpose
This document captures all the good ideas people have had
about writing NUnit tests for the mono project. This document
will be useful for anyone who writes or maintains unit tests.
Other resources
- mcs/class/README has an explanation of the build process and
how it relates to the tests.
- http://nunit.sourceforge.net is the place to find out about
NUnit
Getting Started
If you are new to writing NUnit tests, there is a template
you may use to help get started. The file is:
mcs/class/doc/TemplateTest.cs
(2.- This is the point two!. This file is just after the end
of the guidelines. Copy/paste it in another buffer. And keep
reading.)
Save a copy of this file in the appropriate test subdirecty
(see below), and replace all the [text] markers with
appropriate code. Comments in the template are there to guide
you. You should also look at existing tests to see how other
people have written them.
mcs/class/corlib/Test/System.Collections/CollectionBaseTest.cs
is a small one that might help.
(3.- You reached the third point. And as expected, it's just
here to tell you that the content of CollectionBaseTest.cs is
after the TemplateTest.cs code at the end of these
guidelines.)
The directory that will contain your new file depends on the
assembly/namespace of the class for which you are creating the
tests. Under mcs/class there is a directory for each assembly.
In each assembly there is a Test directory, e.g.
mcs/class/corlib/Test. In the Test directory there are
sub-directories for each namespace in the assembly, e.g.
mcs/class/corlib/Test/Sytem. Put your new test file in the
appropriate sub-directory under Test for the class you are
testing.
Once your test class is complete, you need to add it to the
AllTests.cs file in the same directory as your new test. Add a
call to "suite.AddTest()" passing the name of your new test
class's suite property as the parameter. You will see examples
in the AllTests.cs file, so just copy and paste inside there.
Once all of that is done, you can do a 'make test' from the top
mcs directory. Your test class will be automagically included
in the build and the tests will be run along with all the
others.
Tips
Provide an unique error message for Assert()
Include an unique message for each Assert() so that when the
assert fails, it is trivial to locate the failing one.
Otherwise, it may be difficult to determine which part of the
test is failing. A good way to ensure unique messages is to use
something like #A01, #A02 etc.
Bad:
AssertEquals("array match", compare[0], i1[0]);
AssertEquals("array match", compare[1], i1[1]);
AssertEquals("array match", compare[2], i1[2]);
AssertEquals("array match", compare[3], i1[3]);
Good:
AssertEquals("#A01", compare[0], i1[0]);
AssertEquals("#A02", compare[1], i1[1]);
AssertEquals("#A03", compare[2], i1[2]);
AssertEquals("#A04", compare[3], i1[3]);
Once you used such a number in an Assert(), don't change it
later on - people might use it it identify the test in bug
reports or in mailing lists.
Use AssertEquals() to compare things, not Assert().
Never compare two values with Assert() - if the test fails,
people have no idea what went wrong while AssertEquals()
reports the failed value.
Bad:
Assert ("A01", myTicks[0] == t1.Ticks);
Good:
AssertEquals ("A01", myTicks[0], t1.Ticks);
Constructors
When writing your testcase, please make sure to provide a
constructor which takes no arguments:
public class DateTimeTest : TestCase
{
public DateTimeTest() : base ("[MonoTests.System.DateTimeTest]") {}
public DateTimeTest (string name): base(name) {}
public static ITest Suite
{
get {
TestSuite suite = new TestSuite ();
return suite;
}
}
}
Namespace
Please keep the namespace within each test directory
consistent - all tests which are referenced in the same
AllTests.cs must be in the same namespace. Of course you can
use subnamespaces as you like - especially for subdirectories
of your testsuite.
For instance, if your AllTests.cs is in namespace "MonoTests"
and you have a subdirectory called "System", you can put all
the tests in that dir into namespace "MonoTests.System".
Test your test with the microsoft runtime
If possible, try to run your testsuite with the Microsoft
runtime on Windows and make sure all tests in it pass. This is
especially important if you're writing a totally new testcase
- without this check you can never be sure that your testcase
contains no bugs ....
Don't worry if you're writing your test on Linux, other people
can test it for you on Windows.
Sometimes you may discover that a test doesn't show the
expected result when run with the Microsoft runtime - either
because there is a bug in their runtime or something is
misleading or wrong in their documentation. In this case,
please put a detailed description of the problem to
mcs/class/doc/API-notes and do also report it to the list -
we'll forward this to the Microsoft people from time to time
to help them fix their documentation and runtime.
-------------------- TemplateTest.cs begins ----------
// this is a template for making NUnit tests. Text enclosed
// in square brackets (and the brackets themselves) should be
// replaced by appropiate code.
// [File Name].cs - NUnit Test Cases for [explain here]
//
// [Author Name] ([Author email Address])
//
// (C) [Copyright holder]
//
// these are the standard namespaces you will need. You may
// need to add more depending on your tests.
using NUnit.Framework;
using System;
// all test namespaces start with "MonoTests." Append the
// Namespace that contains the class you are testing, e.g.
// MonoTests.System.Collections
namespace MonoTests.[Namespace]
{
// the class name should end with "Test" and start with the name
// of the class you are testing, e.g. CollectionBaseTest
public class [Class to be tested]Test : TestCase {
// there should be two constructors for your class. The first
// one (without parameters) should set the name to something
// unique.
// Of course the name of the method is the same as the name of
// the class
public [Constructor]() : base ("[Namespace.Class]") {}
public [Constructor](string name) : base(name) {}
// this method is run before each Test* method is called. You
// can put variable initialization, etc. here that is common to
// each test.
// Just leave the method empty if you don't need to use it.
protected override void SetUp() {}
// this method is run after each Test* method is called. You
// can put clean-up code, etc. here. Whatever needs to be done
// after each test. Just leave the method empty if you don't need
// to use it.
protected override void TearDown() {}
// this property is required. You need change the parameter for
// typeof() below to be your class.
public static ITest Suite {
get {
return new TestSuite(typeof([Classname here]));
}
}
// this is just one of probably many test methods in your test
// class. each test method must start with "Test". All methods
// in your class which start with "Test" will be automagically
// called by the NUnit framework.
public void Test[Something] {
// inside here you will exercise your class and then
// call Assert()
}
}
---------------------- TemplateTest.cs ends --------------
---------------------- CollectionBaseTest.cs begins ------
//
// System.Collections.CollectionBase
// Test suite for System.Collections.CollectionBase
//
// Author:
// Nick D. Drochak II
//
// (C) 2001 Nick D. Drochak II
//
using System;
using System.Collections;
using NUnit.Framework;
namespace MonoTests.System.Collections
{
public class CollectionBaseTest : TestCase
{
public CollectionBaseTest () : base
("System.Collection.CollectionBase testsuite")
{}
public CollectionBaseTest (String name) : base (name)
{}
// We need a concrete class to test the abstract base
// class
public class ConcreteCollection : CollectionBase
{
// These fields are used as markers to test
// the On* hooks.
public bool onClearFired;
public bool onClearCompleteFired;
public bool onInsertFired;
public int onInsertIndex;
public bool onInsertCompleteFired;
public int onInsertCompleteIndex;
public bool onRemoveFired;
public int onRemoveIndex;
public bool onRemoveCompleteFired;
public int onRemoveCompleteIndex;
public bool onSetFired;
public int onSetOldValue;
public int onSetNewValue;
public bool onSetCompleteFired;
public int onSetCompleteOldValue;
public int onSetCompleteNewValue;
// This constructor is used to test OnValid()
public ConcreteCollection()
{
IList listObj;
listObj = this;
listObj.Add(null);
}
// This constructor puts consecutive integers into the list
public ConcreteCollection(int i) {
IList listObj;
listObj = this;
int j;
for (j = 0; j< i; j++) {
listObj.Add(j);
}
}
// A helper method to look at a value in the
// list at a specific index
public int PeekAt(int index)
{
IList listObj;
listObj = this;
return (int) listObj[index];
}
// Mark the flag if this hook is fired
protected override void OnClear() {
this.onClearFired = true;
}
// Mark the flag if this hook is fired
protected override void OnClearComplete()
{
this.onClearCompleteFired = true;
}
// Mark the flag, and save the paramter if
// this hook is fired
protected override void OnInsert(int index,
object value)
{
this.onInsertFired = true;
this.onInsertIndex = index;
}
// Mark the flag, and save the paramter if
// this hook is fired
protected override void OnInsertComplete(int index,
object value)
{
this.onInsertCompleteFired = true;
this.onInsertCompleteIndex = index;
}
// Mark the flag, and save the paramter if this hook
// is fired
protected override void OnRemove(int index,
object value)
{
this.onRemoveFired = true;
this.onRemoveIndex = index;
}
// Mark the flag, and save the paramter if this hook
// is fired
protected override void OnRemoveComplete(int index,
object value)
{
this.onRemoveCompleteFired = true;
this.onRemoveCompleteIndex = index;
}
// Mark the flag, and save the paramters if this hook
// is fired
protected override void OnSet(int index, object oldValue,
object newValue)
{
this.onSetFired = true;
this.onSetOldValue = (int) oldValue;
this.onSetNewValue = (int) newValue;
}
// Mark the flag, and save the paramters if this hook
// is fired
protected override void OnSetComplete(int index,
object oldValue,
object newValue)
{
this.onSetCompleteFired = true;
this.onSetCompleteOldValue = (int) oldValue;
this.onSetCompleteNewValue = (int) newValue;
}
} // public class ConcreteCollection
public static ITest Suite {
get {
return new TestSuite
(typeof(CollectionBaseTest));
}
}
// Check the count property
public void TestCount() {
ConcreteCollection myCollection;
myCollection = new ConcreteCollection(4);
Assert(4 == myCollection.Count);
}
// Make sure GetEnumerator returns an object
public void TestGetEnumerator() {
ConcreteCollection myCollection;
myCollection = new ConcreteCollection(4);
Assert(null != myCollection.GetEnumerator());
}
// OnValid disallows nulls
public void TestOnValid() {
ConcreteCollection myCollection;
try {
myCollection = new ConcreteCollection();
}
catch (ArgumentNullException) {
}
}
// Test various Insert paths
public void TestInsert() {
ConcreteCollection myCollection;
int numberOfItems;
numberOfItems = 3;
// The constructor inserts
myCollection = new ConcreteCollection(numberOfItems);
Assert(myCollection.onInsertFired);
Assert(myCollection.onInsertCompleteFired);
// Using the IList interface, check inserts in the middle
IList listObj = myCollection;
listObj.Insert(1, 9);
Assert(myCollection.onInsertIndex == 1);
Assert(myCollection.onInsertCompleteIndex == 1);
Assert(myCollection.PeekAt(1) == 9);
}
// Test Clear and it's hooks
public void TestClear()
{
ConcreteCollection myCollection;
int numberOfItems;
numberOfItems = 1;
myCollection = new ConcreteCollection(numberOfItems);
myCollection.Clear();
Assert(myCollection.Count == 0);
Assert(myCollection.onClearFired);
Assert(myCollection.onClearCompleteFired);
}
// Test RemoveAt, other removes and the hooks
public void TestRemove()
{
ConcreteCollection myCollection;
int numberOfItems;
numberOfItems = 3;
// Set up a test collection
myCollection = new ConcreteCollection(numberOfItems);
// The list is 0-based. So if we remove the second one
myCollection.RemoveAt(1);
// We should see the original third one in it's place
Assert(myCollection.PeekAt(1) == 2);
Assert(myCollection.onRemoveFired);
Assert(myCollection.onRemoveIndex == 1);
Assert(myCollection.onRemoveCompleteFired);
Assert(myCollection.onRemoveCompleteIndex == 1);
IList listObj = myCollection;
listObj.Remove(0);
// Confirm parameters are being passed to the hooks
Assert(myCollection.onRemoveIndex == 0);
Assert(myCollection.onRemoveCompleteIndex == 0);
}
// Test the random access feature
public void TestSet()
{
ConcreteCollection myCollection;
int numberOfItems;
numberOfItems = 3;
myCollection = new ConcreteCollection(numberOfItems);
IList listObj = myCollection;
listObj[0] = 99;
Assert((int) listObj[0] == 99);
Assert(myCollection.onSetFired);
Assert(myCollection.onSetCompleteFired);
Assert(myCollection.onSetOldValue == 0);
Assert(myCollection.onSetCompleteOldValue == 0);
Assert(myCollection.onSetNewValue == 99);
Assert(myCollection.onSetCompleteNewValue == 99);
}
}
}
----------------------- CollectionBaseTest.cs ends --------
* If you use Emacs, you might want to use the .emacs
file and the package developed by Brad Merrill
mailto:zbrad@cybercom.net. It will allow you to
highlight and indent in C# style in your Emacs
editor. (XEmacs will still work but it'll also
complain).
* CSharpDevelop is a GPLed IDE developed by IC#Code.
Search for it at sourceforge if you are interested
in it.
* For those who Java: "A comparison of Microsoft's
C# programming language to Sun Microsystem's Java
Programming language" by Dare Obasanjo is a really good
(very complete) text to read.
* Suggest this point and more, now I can't think of
anything more.
Enjoy!!.
(c) 2002, Jaime Anguiano Olarra.
The parts included in this document are property of their
respective authors.
Note: The identation of the source code has been changed a bit
so it could fit better in the website. Anyway, as nothing more
changed, the files should work as expected.