performance tests should be able to measure runs/sec rather than time
authorrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 14 May 2012 05:23:37 +0000 (05:23 +0000)
committerrniwa@webkit.org <rniwa@webkit.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Mon, 14 May 2012 05:23:37 +0000 (05:23 +0000)
commit1e6be6872d7c1bdc1af43718a423626b9644460f
treee4ff965150482906c33c39d1f8582c88f357fadc
parent025277554e378e647a56bf0bf9bdb976ed0fd3c5
performance tests should be able to measure runs/sec rather than time
https://bugs.webkit.org/show_bug.cgi?id=86021

Reviewed by Ojan Vafai.

PerformanceTests:

Add PerfTestRunner.runPerSecond. It uses _runLoop but replaces _runner by _perSecondRunner
to compute runs/s of runFunction.

When _perSecondRunner is called for the first time, i.e. _completedRuns is 0 (notice this is -1
in regular run/_runner), it slowly increases the number of function calls to runFunction between
time measurements in order to discount the time used by new Date() calls themselves until the
total time spent reaches 100 milliseconds.

By default, runPerSecond runs the test for at least 750 milliseconds in each run, and executes
21 runs, yielding the total run time of roughly 18 seconds. This is significantly faster than
most of existing performance tests. Also see http://ejohn.org/blog/accuracy-of-javascript-time/.

Finally, refactored the existing methods of PerfTestRunner to allow "runs/s" unit and share code.

* Layout/flexbox-column-nowrap.html:
* Layout/flexbox-column-wrap.html:
* Layout/flexbox-row-nowrap.html:
* Layout/flexbox-row-wrap.html:
* resources/runner.js:
(PerfTestRunner.computeStatistics): Takes unit.
(PerfTestRunner.logStatistics): Ditto.
(PerfTestRunner._runLoop):
(PerfTestRunner._runner):
(PerfTestRunner.runPerSecond): Added.
(PerfTestRunner._perSecondRunner): Added. Called by _runLoop.
(PerfTestRunner._perSecondRunnerIterator): Added.

Tools:

Allow " runs/s" or " ms" to appear after numerical values in tests.

* Scripts/webkitpy/performance_tests/perftest.py:
(PerfTest):

LayoutTests:

Add tests for PerfTestRunner.runPerSecond.

* fast/harness/perftests/runs-per-second-iterations-expected.txt: Added.
* fast/harness/perftests/runs-per-second-iterations.html: Added.
* fast/harness/perftests/runs-per-second-log-expected.txt: Added.
* fast/harness/perftests/runs-per-second-log.html: Added.

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@116916 268f45cc-cd09-0410-ab3c-d52691b4dbfc
13 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/harness/perftests/runs-per-second-iterations-expected.txt [new file with mode: 0644]
LayoutTests/fast/harness/perftests/runs-per-second-iterations.html [new file with mode: 0644]
LayoutTests/fast/harness/perftests/runs-per-second-log-expected.txt [new file with mode: 0644]
LayoutTests/fast/harness/perftests/runs-per-second-log.html [new file with mode: 0644]
PerformanceTests/ChangeLog
PerformanceTests/Layout/flexbox-column-nowrap.html
PerformanceTests/Layout/flexbox-column-wrap.html
PerformanceTests/Layout/flexbox-row-nowrap.html
PerformanceTests/Layout/flexbox-row-wrap.html
PerformanceTests/resources/runner.js
Tools/ChangeLog
Tools/Scripts/webkitpy/performance_tests/perftest.py