Benchmark.pm: bail out earlier on zero delta

countit(), which runs code for $n seconds, performs an initial
set of calibration loops to find out roughly how many iterations
are required to burn 0.1 CPU secs more in the code loop than in the empty
loop (which is then used to guesstimate how many iterations to do for the
main loop).

This calibration is designed to bail out if we consistently get <= 0
difference between the empty loop and the code loop; however, this
bailout can take too long: it looks for a run of 17 *consecutive* zero
deltas during 2048, 4096,8192,... iterations, which means it won't fail
until approx 131 million empty + code iterations are run, even if
the zeroes are consecutive. Of course if just by random noise one delta
is infinitesimally > 0, then the count is reset and the exponential count
continues.

The net effect of this is that the calibration loop can loop forever on
'small' code.

This commit makes it additionally bail out of the calibration loop if
running the iterations for a particular value of N takes more than 8
CPU seconds, while still giving zero delta.

diff --git a/lib/Benchmark.pm b/lib/Benchmark.pm
index e153435..9a43a2b 100644 (file)
--- a/lib/Benchmark.pm
+++ b/lib/Benchmark.pm
@@ -482,7 +482,7 @@ our(@ISA, @EXPORT, @EXPORT_OK, %EXPORT_TAGS, $VERSION);
              clearcache clearallcache disablecache enablecache);
 %EXPORT_TAGS=( all => [ @EXPORT, @EXPORT_OK ] ) ;
 
-$VERSION = 1.17;
+$VERSION = 1.18;
 
 # --- ':hireswallclock' special handling
 
@@ -772,11 +772,23 @@ sub countit {
     # First find the minimum $n that gives a significant timing.
     my $zeros=0;
     for ($n = 1; ; $n *= 2 ) {
+       my $t0 = Benchmark->new(0);
        my $td = timeit($n, $code);
+       my $t1 = Benchmark->new(0);
        $tc = $td->[1] + $td->[2];
        if ( $tc <= 0 and $n > 1024 ) {
-           ++$zeros > 16
-               and die "Timing is consistently zero in estimation loop, cannot benchmark. N=$n\n";
+           my $d = timediff($t1, $t0);
+           # note that $d is the total CPU time taken to call timeit(),
+           # while $tc is is difference in CPU secs between the empty run
+           # and the code run. If the code is trivial, its possible
+           # for $d to get large while $tc is still zero (or slightly
+           # negative). Bail out once timeit() starts taking more than a
+           # few seconds without noticeable difference.
+           if ($d->[1] + $d->[2] > 8
+               || ++$zeros > 16)
+           {
+               die "Timing is consistently zero in estimation loop, cannot benchmark. N=$n\n";
+            }
        } else {
            $zeros = 0;
        }