diff options
author | Ævar Arnfjörð Bjarmason <avarab@gmail.com> | 2010-07-02 18:59:49 +0400 |
---|---|---|
committer | Junio C Hamano <gitster@pobox.com> | 2010-07-05 22:23:47 +0400 |
commit | 20873f45e7fe23a8530e643d2eea9566d57f2fd4 (patch) | |
tree | d230fb2e8d82da381b47484e50df40d249545c68 /t | |
parent | b5500d16cd1021bc1430ad019512c636dd3311d6 (diff) |
t/README: Document the do's and don'ts of tests
Add a "Do's, don'ts & things to keep in mind" subsection to the
"Writing Tests" documentation. Much of this is based on Junio C
Hamano's "Test your stuff" section in
<7vhbkj2kcr.fsf@alter.siamese.dyndns.org>.
I turned it into a list of do's and don'ts to make it easier to skim
it, and integrated my note that a TAP harness will get confused if you
print "ok" or "not ok" at the beginning of a line.
Thad had to be fixed in 335f87871fe5aa6b3fd55b2b4e80f16fe9681483 when
TAP support was introduced.
Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Diffstat (limited to 't')
-rw-r--r-- | t/README | 78 |
1 files changed, 78 insertions, 0 deletions
@@ -231,6 +231,84 @@ This test harness library does the following things: consistently when command line arguments --verbose (or -v), --debug (or -d), and --immediate (or -i) is given. +Do's, don'ts & things to keep in mind +------------------------------------- + +Here's a few examples of things you probably should and shouldn't do +when writing tests. + +Do: + + - Put as much code as possible inside test_expect_success and other + assertions. + + Even code that isn't a test per se, but merely some setup code + should be inside a test assertion if at all possible. Test scripts + should only have trivial code outside of their assertions. + + - Chain your test assertions + + Write test code like this: + + git merge foo && + git push bar && + test ... + + Instead of: + + git merge hla + git push gh + test ... + + That way all of the commands in your tests will succeed or fail. If + you must ignore the return value of something (e.g. the return + value of export is unportable) it's best to indicate so explicitly + with a semicolon: + + export HLAGH; + git merge hla && + git push gh && + test ... + +Don't: + + - exit() within a <script> part. + + The harness will catch this as a programming error of the test. + Use test_done instead if you need to stop the tests early (see + "Skipping tests" below). + + - Break the TAP output + + The raw output from your test might be interpreted by a TAP + harness. You usually don't have to worry about that. TAP harnesses + will ignore everything they don't know about, but don't step on + their toes in these areas: + + - Don't print lines like "$x..$y" where $x and $y are integers. + + - Don't print lines that begin with "ok" or "not ok". + + A TAP harness expect a line that begins with either "ok" and "not + ok" to signal a test passed or failed (and our harness already + produces such lines), so your script shouldn't emit such lines to + their output. + + You can glean some further possible issues from the TAP grammar + (see http://search.cpan.org/perldoc?TAP::Parser::Grammar#TAP_Grammar) + but the best indication is to just run the tests with prove(1), + it'll complain if anything is amiss. + +Keep in mind: + + - That what you print to stderr and stdout is usually ignored + + Inside <script> part, the standard output and standard error + streams are discarded, and the test harness only reports "ok" or + "not ok" to the end user running the tests. Under --verbose, they + are shown to help debugging the tests. + + Skipping tests -------------- |