Modify new-run-webkit-tests to return the percentage difference in image diffs where...
authordpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 29 Sep 2011 00:09:39 +0000 (00:09 +0000)
committerdpranke@chromium.org <dpranke@chromium.org@268f45cc-cd09-0410-ab3c-d52691b4dbfc>
Thu, 29 Sep 2011 00:09:39 +0000 (00:09 +0000)
commit7d4c9705a83ac16ee64107690e3c87bc59640411
tree8f18fa28e6f895cf15410396df6d9e43be099ae3
parent4f70e6dde0474a46ba945795275a0e319d23b7e2
Modify new-run-webkit-tests to return the percentage difference in image diffs where possible.

https://bugs.webkit.org/show_bug.cgi?id=67253

Reviewed by Eric Seidel.

Tools:

Nearly all of the work in this was done by Simon Fraser; I'm
just repackaging it and cleaning it up a bit. This change
modifies port.diff_image() to return a tuple of (pass/fail,
%age), adds the value to the FailureImageHashMismatch, and
writes the value into the full_results.json files to be
displayed in the results page.

* Scripts/webkitpy/layout_tests/controllers/manager.py:
* Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
* Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py:
* Scripts/webkitpy/layout_tests/models/test_failures.py:
* Scripts/webkitpy/layout_tests/port/base.py:
* Scripts/webkitpy/layout_tests/port/chromium.py:
* Scripts/webkitpy/layout_tests/port/chromium_unittest.py:
* Scripts/webkitpy/layout_tests/port/port_testcase.py:
* Scripts/webkitpy/layout_tests/port/test.py:
* Scripts/webkitpy/layout_tests/port/webkit.py:
* Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py:

LayoutTests:

* fast/harness/resources/results-test.js:
(mockExpectation):
(runDefaultSingleRowTest):
():
* fast/harness/results.html:

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@96272 268f45cc-cd09-0410-ab3c-d52691b4dbfc
16 files changed:
LayoutTests/ChangeLog
LayoutTests/fast/harness/resources/results-test.js
LayoutTests/fast/harness/results.html
Tools/ChangeLog
Tools/Scripts/webkitpy/layout_tests/controllers/manager.py
Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py
Tools/Scripts/webkitpy/layout_tests/layout_package/test_result_writer.py
Tools/Scripts/webkitpy/layout_tests/models/test_failures.py
Tools/Scripts/webkitpy/layout_tests/port/base.py
Tools/Scripts/webkitpy/layout_tests/port/chromium.py
Tools/Scripts/webkitpy/layout_tests/port/chromium_unittest.py
Tools/Scripts/webkitpy/layout_tests/port/port_testcase.py
Tools/Scripts/webkitpy/layout_tests/port/test.py
Tools/Scripts/webkitpy/layout_tests/port/webkit.py
Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_integrationtest.py
Tools/Scripts/webkitpy/to_be_moved/rebaseline_chromium_webkit_tests.py