diff options
author | Alexander Köplinger <alex.koeplinger@outlook.com> | 2016-01-27 22:15:44 +0300 |
---|---|---|
committer | Alexander Köplinger <alex.koeplinger@outlook.com> | 2016-01-27 22:17:18 +0300 |
commit | 93ed825dcb38f07bcef77caa529edf3b186dabdd (patch) | |
tree | 5d6e17ab7cb339db7494f74192d2c3a1971bdb24 | |
parent | 39d172e19037a01730e9b992d269c2bea2249838 (diff) |
[profiler] Fix test reporting success even though a test failed
I noticed this in https://jenkins.mono-project.com/job/test-mono-pull-request-armhf/1715/ which reported success in the "profiler" test step,
even though the step actually reported errors:
```
MONO_PATH=/media/ssd/jenkins/workspace/test-mono-pull-request-armhf/mcs/class/lib/net_4_x perl ./ptestrunner.pl ../..
Checking test-alloc.exe with report ...
Checking test-busy.exe with report ...
Checking test-busy.exe with report,sample ...
Error: Missing thread Finalizer.
Error: Missing thread Main.
Error: Wrong loaded images 2.
Error: Not enough compiled method.
Error: Not enough compiled code.
Error: Missing thread BusyHelper.
Total errors: 6
Checking test-monitor.exe with report ...
Checking test-excleave.exe with report ...
Checking test-heapshot.exe with report,heapshot ...
Checking test-heapshot.exe with heapshot,output=-traces.mlpd ...
Checking test-traces.exe with output=-traces.mlpd ...
Checking test-traces.exe with nocalls,output=-traces.mlpd ...
make[1]: Leaving directory `/media/ssd/jenkins/workspace/test-mono-pull-request-armhf/mono/profiler'
make: Leaving directory `/media/ssd/jenkins/workspace/test-mono-pull-request-armhf/mono/profiler'
++ bc
+++ date +%s
++ echo 1453874768 - 1453874741
+ echo -e '*** end(27): profiler: \e[42mPassed\e[0m'
*** end(27): profiler: Passed
```
The culprit is that ptestrunner.pl checks whether $total_errors is non-zero to determin the process exit code.
This is wrong, as $total_errors is cleared before each test so unless the last test fails the exit code would always be zero.
Introduced a new $global_errors variable which is only incremented and never cleared to fix this.
-rwxr-xr-x | mono/profiler/ptestrunner.pl | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/mono/profiler/ptestrunner.pl b/mono/profiler/ptestrunner.pl index fc29b43b260..2113c25135f 100755 --- a/mono/profiler/ptestrunner.pl +++ b/mono/profiler/ptestrunner.pl @@ -6,7 +6,8 @@ use strict; my $builddir = shift || die "Usage: ptestrunner.pl mono_build_dir\n"; my @errors = (); -my $total_errors = 0; +my $total_errors = 0; # this is reset before each test +my $global_errors = 0; my $report; my $profbuilddir = $builddir . "/mono/profiler"; @@ -106,7 +107,7 @@ report_errors (); emit_nunit_report(); -exit ($total_errors? 1: 0); +exit ($global_errors ? 1 : 0); # utility functions sub append_path { @@ -156,6 +157,7 @@ sub report_errors foreach my $e (@errors) { print "Error: $e\n"; $total_errors++; + $global_errors++; } print "Total errors: $total_errors\n" if $total_errors; #print $report; @@ -168,7 +170,7 @@ sub emit_nunit_report use Net::Domain qw(hostname hostfqdn); use locale; - my $failed = $total_errors ? 1 : 0; + my $failed = $global_errors ? 1 : 0; my $successbool; my $total = 1; my $mylocale = setlocale (LC_CTYPE); |