PerformanceTests/SunSpider/ChangeLog

   1 2012-03-11  Maciej Stachowiak  <mjs@apple.com>
   2
   3         Add --instruments option to sunspider to profile with Instruments
   4         https://bugs.webkit.org/show_bug.cgi?id=80783
   5
   6         Reviewed by Filip Pizlo.
   7
   8         * resources/TimeProfile20us.tracetemplate: Added.
   9         * sunspider:
  10         (runTestsOnce):
  11
  12 2011-11-15  Andy Wingo  <wingo@igalia.com>
  13
  14         Extend sunspider driver to be able to run kraken
  15         https://bugs.webkit.org/show_bug.cgi?id=71799
  16
  17         Reviewed by Filip Pizlo.
  18
  19         * resources/sunspider-standalone-driver.js: Try to load a -data
  20         file.  If that succeeds, we have a kraken-like test, so we time
  21         the test using `load'.  Otherwise fall back to using `run'.
  22
  23 2011-08-11  Mark Rowe  <mrowe@apple.com>
  24
  25         Fix sunspider-compare-results to default to the same suite version as the sunspider script.
  26
  27         Rubber-stamped by Maciej Stachowiak.
  28
  29         * sunspider-compare-results:
  30
  31 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  32
  33         Update SunSpider versioning to 1.0
  34         https://bugs.webkit.org/show_bug.cgi?id=63870
  35
  36         Reviewed by Dan Bernstein.
  37
  38         * hosted/sunspider.html:
  39         * hosted/versions.html:
  40         * resources/driver-TEMPLATE.html:
  41         * resources/results-TEMPLATE.html:
  42         * sunspider:
  43
  44 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  45
  46         Sunspider 0.9.1 harness doesn't actually close() its test documents
  47         https://bugs.webkit.org/show_bug.cgi?id=47045
  48
  49         Reviewed by Daniel Bates.
  50
  51         * hosted/versions.html:
  52         * resources/driver-TEMPLATE.html:
  53
  54 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  55
  56         Not Reviewed.
  57
  58         Fix obviou typo in previous commit.
  59
  60         * tests/sunspider-1.0/bitops-bitwise-and.js:
  61
  62 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  63
  64         Reviewed by Anders Carlsson.
  65
  66         SunSpider: all four bitops benchmarks can be replaced with NOP
  67         https://bugs.webkit.org/show_bug.cgi?id=38446
  68
  69         * tests/sunspider-1.0/bitops-3bit-bits-in-byte.js:
  70         (TimeFunc): Save the result.
  71         * tests/sunspider-1.0/bitops-bits-in-byte.js:
  72         (TimeFunc): Save the result.
  73         * tests/sunspider-1.0/bitops-bitwise-and.js: More explicitly
  74         save the results.
  75         * tests/sunspider-1.0/bitops-nsieve-bits.js:
  76         (sieve): Save the result.
  77
  78 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  79
  80         SunSpider: string-validate-input.js uses an undeclared variable named 'name', which is a DOM API
  81         https://bugs.webkit.org/show_bug.cgi?id=60937
  82
  83         Reviewed by Dan Bernstein.
  84
  85         * tests/sunspider-1.0/string-validate-input.js:
  86         (doTest): Rename the variable to username
  87
  88 2011-07-02  Maciej Stachowiak  <mjs@apple.com>
  89
  90         Reviewed by Darin Adler.
  91
  92         SunSpider: The main function of math-cordic is dead code
  93         https://bugs.webkit.org/show_bug.cgi?id=63863
  94
  95         Try to prevent math-cordic from being eliminated as dead code or
  96         compiled down to a constant.
  97
  98         * tests/sunspider-1.0/math-cordic.js:
  99         (cordicsincos):
 100         (cordic):
 101
 102 2011-06-30  Maciej Stachowiak  <mjs@apple.com>
 103
 104         Reviewed by Adam Barth.
 105
 106         Create sunspider-1.0 directory in preparation for fixing a bunch of bugs
 107         https://bugs.webkit.org/show_bug.cgi?id=63782
 108
 109         * make-hosted: Learn about the new directory.
 110         * tests/sunspider-1.0: Copied from PerformanceTests/SunSpider/tests/sunspider-0.9.1.
 111
 112 2010-11-29  Geoffrey Garen  <ggaren@apple.com>
 113
 114         Reviewed by Gavin Barraclough.
 115
 116         Improved accuracy of command-line SunSpider.
 117
 118         * resources/sunspider-standalone-driver.js: Use the returned number of
 119         elapsed milliseconds from "run" and "checkSyntax" instead of measuring
 120         ourselves, for slightly more accurate numbers.
 121
 122 2010-09-16  Gavin Barraclough  <barraclough@apple.com>
 123
 124         Reviewed by Mark Rowe.
 125         https://bugs.webkit.org/show_bug.cgi?id=45924
 126
 127         Modify the SunSpider harness to allow a path to be passed to the --suite
 128         argument, supress the normal prepend-"tests/" behaviour for suite names
 129         containing a '/'.
 130
 131         Also wrap the code in an anonymous function to move variables out of global
 132         scope (currently the harness cannot handle running tests the overwrite the
 133         variable 'j').
 134
 135         * resources/sunspider-standalone-driver.js:
 136         * sunspider:
 137
 138 2010-08-27  Michael Saboff  <msaboff@apple.com>
 139
 140         Reviewed by Stephanie Lewis.
 141
 142         Added version 5 and version 6 of V8 tests with modifications similar
 143         to the version 4 tests so they'll work with the SunSpider test harness.
 144         Note that these files are external source and therefore don't conform
 145         to WebKit coding standards.
 146         https://bugs.webkit.org/show_bug.cgi?id=44638
 147
 148         * tests/v8-v5: Added.
 149         * tests/v8-v5/LIST: Added.
 150         * tests/v8-v5/v8-crypto.js: Added.
 151         * tests/v8-v5/v8-deltablue.js: Added.
 152         * tests/v8-v5/v8-earley-boyer.js: Added.
 153         * tests/v8-v5/v8-raytrace.js: Added.
 154         * tests/v8-v5/v8-regexp.js: Added.
 155         * tests/v8-v5/v8-richards.js: Added.
 156         * tests/v8-v5/v8-splay.js: Added.
 157         * tests/v8-v6: Added.
 158         * tests/v8-v6/LIST: Added.
 159         * tests/v8-v6/v8-crypto.js: Added.
 160         * tests/v8-v6/v8-deltablue.js: Added.
 161         * tests/v8-v6/v8-earley-boyer.js: Added.
 162         * tests/v8-v6/v8-raytrace.js: Added.
 163         * tests/v8-v6/v8-regexp.js: Added.
 164         * tests/v8-v6/v8-richards.js: Added.
 165         * tests/v8-v6/v8-splay.js: Added.
 166
 167 2010-07-15  Stephanie Lewis  <slewis@apple.com>
 168
 169         Reviewed by Geoff Garen.
 170
 171         http://bugs.webkit.org/show_bug.cgi?id=42406
 172         Output location of results file.
 173
 174         * sunspider:
 175
 176 2010-05-17  Maciej Stachowiak  <mjs@apple.com>
 177
 178         Reviewed by Mark Rowe.
 179
 180         Browser-hosted version of SunSpider runs tests twice as many times as advertised
 181         https://bugs.webkit.org/show_bug.cgi?id=39201
 182
 183         * make-hosted: Make sure to reset test list after every test suite.
 184
 185 2010-05-07  Adam Roben  <aroben@apple.com>
 186
 187         Fix typo in sunspider-compare-results.js that caused incorrect
 188         two-sample t statistics to be computed
 189
 190         Fixes <http://webkit.org/b/38768> SunSpider computes incorrect
 191         two-sample t statistics
 192
 193         Reviewed by Darin Adler.
 194
 195         * resources/sunspider-compare-results.js:
 196         (sunspiderCompareResults.resultLine): Fixed the calculation of the t
 197         variable to match the definition of a two-sample t statistic. Looks
 198         like this was just a simple typo.
 199
 200 2010-04-05  Darin Adler  <darin@apple.com>
 201
 202         Updated the ignore property for changes to the contents here.
 203
 204         * .: Modified property svn:ignore.
 205
 206 2010-02-20  Maciej Stachowiak  <mjs@apple.com>
 207
 208         Reviewed by Adam Barth.
 209
 210         Add sunspider-0.9.1 to the site in preparation for announcing it.
 211         https://bugs.webkit.org/show_bug.cgi?id=35206
 212
 213         * hosted/sunspider.css: Remove tabs, which I noticed where present while
 214         copying to the WebKitSite directory.
 215
 216 2010-01-11  Oliver Hunt  <oliver@apple.com>
 217
 218         Reviewed by Darin Adler.
 219
 220         Sunspider parse-only tests don't run
 221         https://bugs.webkit.org/show_bug.cgi?id=33489
 222
 223         Fix parse-only test list
 224
 225         * tests/parse-only/LIST:
 226
 227 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 228
 229         Reviewed by Adele Peterson.
 230
 231         Make SunSpider version more prominent in the title
 232         https://bugs.webkit.org/show_bug.cgi?id=32574
 233
 234         * hosted/sunspider.html:
 235         * hosted/versions.html:
 236         * resources/driver-TEMPLATE.html:
 237         * resources/results-TEMPLATE.html:
 238
 239 2009-12-15  Maciej Stachowiak  <mjs@apple.com>
 240
 241         Reviewed by Darin Adler.
 242
 243         Make SunSpider refuse to compare results across content versions
 244         https://bugs.webkit.org/show_bug.cgi?id=32573
 245
 246         The results URL now includes a version indicator, and compare mode
 247         will refuse to compare between versions.
 248
 249         * resources/driver-TEMPLATE.html:
 250         * resources/results-TEMPLATE.html:
 251
 252 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 253
 254         Reviewed by Darin Adler.
 255
 256         Some Browser-hosted SunSpider files are not valid HTML5
 257         https://bugs.webkit.org/show_bug.cgi?id=32536
 258
 259         Made various fixes to validate as HTML5 with no warnings (mainly removing bogus
 260         close tags and adding a meta charset declaration).
 261
 262         * hosted/sunspider.html:
 263         * hosted/versions.html:
 264         * resources/TEMPLATE.html:
 265         * resources/driver-TEMPLATE.html:
 266         * resources/results-TEMPLATE.html:
 267
 268 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 269
 270         Reviewed by Eric Seidel.
 271
 272         Make sunspider-0.9.1 the default content set (both command-line and hosted)
 273         https://bugs.webkit.org/show_bug.cgi?id=32537
 274
 275         * hosted/sunspider.html: Change default suite link.
 276         * sunspider: Change default suite.
 277         * sunspider-compare-results: Likewise.
 278
 279 2009-12-14  Maciej Stachowiak  <mjs@apple.com>
 280
 281         Reviewed by Sam Weinig.
 282
 283         in-browser SunSpider suffers excessive penalty under power management
 284         https://bugs.webkit.org/show_bug.cgi?id=32505
 285
 286         I have made a few changes to address this:
 287
 288         1) Change how browser-hosted SunSpider loads tests - preload the content and write it in with
 289         document.write to reduce triggering of progress UI.
 290         2) Reduce time between tests to 10ms from 500ms, so that power management doesn't keep the CPU
 291         stepped all the way down the whole time.
 292         3) Run the test cycle an extra time for warmup before the runs that count.
 293
 294         * hosted/sunspider-record-result.js: Removed.
 295         * make-hosted:
 296         * resources/TEMPLATE.html:
 297         * resources/driver-TEMPLATE.html:
 298
 299 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 300
 301         Reviewed by Gavin Barraclough.
 302
 303         SunSpider/tests/string-base64.js does not compute a valid base64 encoded string
 304         https://bugs.webkit.org/show_bug.cgi?id=16806
 305
 306         Based on a patch by Eric Seidel.
 307
 308         Fix the base64 computation to actually compute correct results. The impact on runtime of
 309         the test is pretty small, but noticeable for some browsers. But at least it's not
 310         doing a wrong and meaningless computation any more.
 311
 312         * tests/sunspider-0.9.1/string-base64.js:
 313         ():
 314         (base64ToString):
 315
 316 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 317
 318         Fixing commit error...
 319
 320         I accidentally committed my last patch in a form that broke Web-hosted SunSpider. Fixing.
 321
 322         * resources/driver-TEMPLATE.html:
 323
 324 2009-12-13  Maciej Stachowiak  <mjs@apple.com>
 325
 326         Reviewed by Sam Weinig.
 327
 328         Use JSON.parse instead of eval for Web-hosted SunSpider results processing
 329         https://bugs.webkit.org/show_bug.cgi?id=32490
 330
 331         * hosted/json2.js: Added. Incorporated from json.org
 332         * resources/driver-TEMPLATE.html: Fix an HTML validation bug I noticed.
 333         * resources/results-TEMPLATE.html: Use JSON.parse instead of eval to process
 334         results.
 335
 336 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 337
 338         Reviewed by Oliver Hunt.
 339
 340         Enable Web-hosted version of SunSpider to handle multiple versions
 341         https://bugs.webkit.org/show_bug.cgi?id=32478
 342
 343         * make-hosted: Now generate the hosted version to be able to run both the
 344         0.9 and the 0.9.1 test suites.
 345         * hosted: Modified property svn:ignore.
 346         * hosted/sunspider-driver.html: Removed.
 347         * hosted/sunspider-results.html: Removed.
 348         * hosted/sunspider.html:
 349         * hosted/versions.html: Added.
 350         * resources/TEMPLATE.html:
 351         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-driver.html.
 352         * resources/driver-TEMPLATE.html: Copied from hosted/sunspider-results.html.
 353
 354 2009-12-12  Maciej Stachowiak  <mjs@apple.com>
 355
 356         Reviewed by Oliver Hunt.
 357
 358         Give command-line sunspider the ability to handle multiple suites and versions
 359         https://bugs.webkit.org/show_bug.cgi?id=32477
 360
 361         Each suite/version's set of tests are now in a separate subdirectory of tests/, with a file named LIST
 362         enumerating the tests. Current suites are:
 363
 364         sunspider-0.9 - riginal SunSpider content
 365         sunspider-0.9.1 - SunSpider suite with a few tests fixed (and a few more to come)
 366         ubench - "ubench" microbenchmark suite
 367         v8-v4 - version 4 of the v8 benchmark
 368         parse-only - some parser tests
 369
 370         Sunspider can now be run with the --suite= parameter to select a
 371         suite. The default is --suite=sunspider-0.9.
 372
 373         * .: Modified property svn:ignore to ignore new results directories.
 374         * make-hosted: Generate based on sunspider-0.9 suite.
 375         * resources/TEMPLATE.html: Fix an HTML compliance problem.
 376         * resources/sunspider-standalone-driver.js: Become aware of suite names.
 377         * sunspider: Modified to handle multiple suites as described above.
 378         * sunspider-compare-results: ditto
 379         * tests/LIST: Moved to tests/sunspider-0.9/ and tests/sunspider-0.9.1/
 380         * tests/LIST-PARSE-ONLY: Moved to parse-only/LIST
 381         * tests/LIST-UBENCH: Removed.
 382         * tests/LIST-V8: Removed.
 383         * tests/3d-cube.js: Moved to tests/sunspider-0.9 and tests/sunspider-0.9.1
 384         * tests/3d-morph.js: ditto
 385         * tests/3d-raytrace.js: ditto
 386         * tests/access-binary-trees.js: ditto
 387         * tests/access-fannkuch.js: ditto
 388         * tests/access-nbody.js: ditto
 389         * tests/access-nsieve.js: ditto
 390         * tests/bitops-3bit-bits-in-byte.js: ditto
 391         * tests/bitops-bits-in-byte.js: ditto
 392         * tests/bitops-bitwise-and.js: ditto
 393         * tests/bitops-nsieve-bits.js: ditto
 394         * tests/controlflow-recursive.js: ditto
 395         * tests/crypto-aes.js: ditto
 396         * tests/crypto-md5.js: ditto
 397         * tests/crypto-sha1.js: ditto
 398         * tests/date-format-tofte.js: ditto
 399         * tests/date-format-xparb.js: ditto
 400         * tests/math-cordic.js: ditto
 401         * tests/math-partial-sums.js: ditto
 402         * tests/math-spectral-norm.js: ditto
 403         * tests/regexp-dna.js: ditto
 404         * tests/string-base64.js: ditto
 405         * tests/string-fasta.js: ditto
 406         * tests/string-tagcloud.js: ditto
 407         * tests/string-unpack-code.js: ditto
 408         * tests/string-validate-input.js: ditto
 409         * tests/sunspider-0.9/: Added
 410         * tests/sunspider-0.9/*.js: Moved from one level up
 411         * tests/sunspider-0.9.1: Added.
 412         * tests/sunspider-0.9.1/*.js: Moved from one level up.
 413         * tests/v8-crypto.js: Moved to v8-v4/ directory.
 414         * tests/v8-deltablue.js: ditto
 415         * tests/v8-earley-boyer.js: ditto
 416         * tests/v8-raytrace.js: ditto
 417         * tests/v8-regexp.js: ditto
 418         * tests/v8-richards.js: ditto
 419         * tests/v8-splay.js: ditto
 420         * tests/v8-v4: Added.
 421
 422 2009-09-30  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 423
 424         Reviewed by Darin Adler.
 425
 426         Platform specific null device replaced with a platform independent.
 427         https://bugs.webkit.org/show_bug.cgi?id=29544
 428
 429         * sunspider: Using File::Spec->devnull() instead of /dev/null and NUL.
 430
 431 2009-09-21  Csaba Osztrogonac  <oszi@inf.u-szeged.hu>
 432
 433         Reviewed by Maciej Stachowiak.
 434
 435         [Qt] Make sunspider script work on Windows platform.
 436         https://bugs.webkit.org/show_bug.cgi?id=29544
 437
 438         * sunspider:
 439         Using platform specific null device instead of /dev/null.
 440
 441 2009-06-19  Adam Treat  <adam.treat@torchmobile.com>
 442
 443         Reviewed by Oliver Hunt.
 444
 445         https://bugs.webkit.org/show_bug.cgi?id=26540
 446         Currently the SunSpider test driver lacks an option to run a test suite that
 447         will test JavaScriptCore parsing performance only.  This patch adds just such
 448         a test suite and option to SunSpider as well as the jsc test shell.  I've included
 449         three large javascript source files found in the wild: jquery, mootools and prototype.
 450         Combined with the concatenation of all three, these form a new testsuite to measure
 451         and test pure JavaScriptCore parsing performance.
 452
 453         * resources/sunspider-standalone-driver.js:
 454           Load the js with parse-only flag if it is part of the parse-only suite.
 455         * sunspider:
 456           Add the parse-only suite to the list of options.
 457         * sunspider-compare-results:
 458           Add the parse-only suite to the list of options.
 459         * tests/LIST-PARSE-ONLY: Added.
 460         * tests/parse-only/concat-jquery-mootools-prototype.js: Added.
 461           A very large concatenation of the scripts found below.  This js takes ~30ms to parse on my machine.
 462         * tests/parse-only/jquery-1.3.2.js: Added.
 463           Latest version from jquery.com.
 464         * tests/parse-only/mootools-1.2.2-core-nc.js: Added.
 465           Latest version from mootools.net.
 466         * tests/parse-only/prototype-1.6.0.3.js: Added.
 467           Latest version from prototypejs.org.
 468
 469 2009-06-13  Steve Falkenburg  <sfalken@apple.com>
 470
 471         Update V8 benchmark to version 4.
 472
 473         Reviewed by Cameron Zwarich.
 474
 475         * tests/LIST-V8:
 476         * tests/v8-crypto.js:
 477         * tests/v8-deltablue.js:
 478         * tests/v8-earley-boyer.js:
 479         * tests/v8-raytrace.js:
 480         * tests/v8-regexp.js: Added.
 481         * tests/v8-richards.js:
 482         * tests/v8-splay.js: Added.
 483
 484 2009-05-26  Olivier DOLE  <odole@pleyo.com>
 485
 486         Reviewed by Maciej.
 487
 488         Store sunspider test suite results in a specific directory for each
 489         different type of benchmark (sunspider, V8 or ubench).
 490
 491         * sunspider:
 492         * sunspider-compare-results:
 493
 494 2009-05-12  Steve Falkenburg  <sfalken@apple.com>
 495
 496         Build fix.
 497
 498         * SunSpider.make:
 499
 500 2009-05-11  Steve Falkenburg  <sfalken@apple.com>
 501
 502         Include SunSpider tests in production builds.
 503
 504         Reviewed by Adam Roben.
 505
 506         * SunSpider.make: Added.
 507         * xcopy.excludes: Added.
 508
 509 2009-03-04  Oliver Hunt  <oliver@apple.com>
 510
 511         Added a few more tests to ubench
 512
 513         RS=Gavin Barraclough
 514
 515         Adding a couple of test to cover too few and too many arguments.  This
 516         is needed to prevent regressions in all call paths from my current refactoring.
 517
 518         * tests/LIST-UBENCH:
 519         * tests/ubench/function-correct-args.js: Added.
 520         (f):
 521         * tests/ubench/function-excess-args.js: Added.
 522         (f):
 523
 524 2008-10-30  Zoltan Horvath  <Horvath.Zoltan.6@stud.u-szeged.hu>
 525
 526         Fix a bug where when V8's or UBENCH's test suite list file is missing
 527         the error message is suggesting that SunSpider's list file is missing.
 528
 529         Reviewed by Sam Weinig.
 530
 531         * sunspider:
 532         (loadTestsList): Fix the error message to use the right list name.
 533
 534 2008-10-07  David Hyatt  <hyatt@apple.com>
 535
 536         https://bugs.webkit.org/show_bug.cgi?id=18989
 537
 538         Remove global substitution from SunSpider regexp test, since only Firefox was honoring it and
 539         we want all browsers to be doing the same thing on the test.
 540
 541         Reviewed by Maciej
 542
 543         * tests/regexp-dna.js:
 544
 545 2008-09-09  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 546
 547         Rubber-stamped by Maciej Stachowiak.
 548
 549         The bleeding edge branch of V8 now supports passing fileanmes after
 550         '-f', so the SunSpider script should be changed to do this again.
 551         This allows all 3 major JavaScript engines to be used with the
 552         unmodified SunSpider script.
 553
 554         * sunspider:
 555
 556 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 557
 558         Reviewed by Maciej Stachowiak.
 559
 560         Add a '--args' option to SunSpider, so that we can pass '-j' to TraceMonkey
 561         and '--expose-gc' to V8. Also, rename the '--v8' option to '--v8-suite' and
 562         correct the punctuation in its documentation.
 563
 564         * sunspider:
 565
 566 2008-09-04  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 567
 568         Reviewed by Oliver Hunt.
 569
 570         Bug 20616: Incorporate V8 benchmarks in testing
 571         <https://bugs.webkit.org/show_bug.cgi?id=20616>
 572
 573         This is a first attempt at running the V8 benchmarks within SunSpider,
 574         with the --v8 option. The number of iterations is fixed for each
 575         benchmark, and was chosen to make each of the benchmarks run between
 576         500 ms and 600 ms in V8 on my machine, a 2.16 GHz MacBook Pro.
 577
 578         * sunspider:
 579         * tests/LIST-V8: Added.
 580         * tests/v8-crypto.js: Added.
 581         * tests/v8-deltablue.js: Added.
 582         * tests/v8-earley-boyer.js: Added.
 583         * tests/v8-raytrace.js: Added.
 584         * tests/v8-richards.js: Added.
 585
 586 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 587
 588         Not reviewed.
 589
 590         Remove a local change that was accidentally committed in r36034.
 591
 592         * resources/sunspider-standalone-driver.js:
 593
 594 2008-09-02  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 595
 596         Rubber-stamped by Oliver Hunt.
 597
 598         Make SunSpider work with JavaScript shells that don't take filenames
 599         after a '-f' argument.
 600
 601         * sunspider:
 602
 603 2008-06-07  Cameron Zwarich  <cwzwarich@uwaterloo.ca>
 604
 605         Reviewed by Timothy.
 606
 607         Remove the --squirrelfish option from SunSpider, as it is no longer needed.
 608
 609         * sunspider:
 610         * tests/LIST-SQUIRRELFISH: Removed.
 611
 612 2008-06-03  Alexey Proskuryakov  <ap@webkit.org>
 613
 614         Rubber-stamped by Mark Rowe.
 615
 616         Roll out r31215 - with Mac OS X 10.5.3, there is no need to pause update daemon any more.
 617
 618         * sunspider:
 619
 620 2008-04-15  Maciej Stachowiak  <mjs@apple.com>
 621
 622         Rubber stamped by Oliver.
 623
 624         - add newly working tests to squirrelfish list
 625
 626         * tests/LIST-SQUIRRELFISH: Add 3d-raytrace and string-unpack-code
 627
 628 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 629
 630         Rubber stamped by Oliver.
 631
 632         - move ubench tests properly.
 633
 634         * tests/ubench: Added.
 635         * tests/ubench/function-closure.js: Added.
 636         * tests/ubench/function-empty.js: Added.
 637         * tests/ubench/function-missing-args.js: Added.
 638         * tests/ubench/function-sum.js: Added.
 639         * tests/ubench/loop-empty-resolve.js: Added.
 640         * tests/ubench/loop-empty.js: Added.
 641         * tests/ubench/loop-sum.js: Added.
 642
 643 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 644
 645         Rubber stamped by Oliver.
 646
 647         * sunspider: Don't force --runs=1 in squirrelfish mode any more.
 648
 649 2008-04-14  Maciej Stachowiak  <mjs@apple.com>
 650
 651         Reviewed by Oliver.
 652
 653         - make --squirrelfish mode use the real harness, but a reduced set of tests
 654
 655         Also add a new ubench mode which runs the older squirrelfish microbenchmarks.
 656
 657         * sunspider:
 658         * tests/LIST-SQUIRRELFISH:
 659         * tests/LIST-UBENCH: Added.
 660         * tests/squirrelfish: Moved to tests/ubench
 661
 662 2008-04-14  Geoffrey Garen  <ggaren@apple.com>
 663
 664         Reviewed by Maciej Stachowiak.
 665
 666         Removed an unused test.
 667
 668         * tests/squirrelfish/loop-resolve.js: Removed.
 669
 670 2008-04-10  Maciej Stachowiak  <mjs@apple.com>
 671
 672         Reviewed by Oliver.
 673
 674         Add newly runnable tests to --squirrelfish mode.
 675
 676         * tests/LIST-SQUIRRELFISH:
 677
 678 2008-04-06  Geoffrey Garen  <ggaren@apple.com>
 679
 680         Reviewed by Maciej Stachowiak.
 681
 682         A little more love for --squirrelfish mode:
 683
 684         Fixed a misplaced sort. Results properly sort now.
 685
 686         Added a test list just for squirrelfish, and updated it to include
 687         all currently passing SunSpider tests.
 688
 689         Fixed the pruning regexp to match 3d-morph.js.
 690
 691         * sunspider:
 692
 693 2008-03-26  Geoffrey Garen  <ggaren@apple.com>
 694
 695         Reviewed by Oliver Hunt.
 696
 697         --squirrelfish mode: pared down tests for squirrelfish to chew on.
 698
 699         * sunspider:
 700         * tests/LIST:
 701         * tests/squirrelfish: Added.
 702         * tests/squirrelfish/function-closure.js: Added.
 703         * tests/squirrelfish/function-empty.js: Added.
 704         * tests/squirrelfish/function-missing-args.js: Added.
 705         * tests/squirrelfish/function-sum.js: Added.
 706         * tests/squirrelfish/loop-empty-resolve.js: Added.
 707         * tests/squirrelfish/loop-empty.js: Added.
 708         * tests/squirrelfish/loop-resolve.js: Added.
 709         * tests/squirrelfish/loop-sum.js: Added.
 710
 711 2008-03-21  Alexey Proskuryakov  <ap@webkit.org>
 712
 713         Reviewed by Geoff.
 714
 715         * sunspider: Pause update daemon on Darwin to make results reliable
 716         again (workaround for <rdar://problem/5811127>).
 717
 718 2008-02-07  Eric Seidel  <eric@webkit.org>
 719
 720         Reviewed by Sam.
 721
 722         * sunspider: add an error message when ./tests/LIST is not found
 723
 724 2007-12-29  Alp Toker  <alp@atoker.com>
 725
 726         Fix typo.
 727
 728         * hosted/sunspider.html:
 729
 730 2007-12-29  Alp Toker  <alp@atoker.com>
 731
 732         Reviewed by Maciej.
 733
 734         Provide the current URL in the SunSpider results page. Useful for
 735         copying benchmark results out of browsers without a location bar or
 736         where the location bar can't handle long URLs.
 737
 738         * hosted/sunspider-results.html:
 739
 740 2007-12-20  Eric Seidel  <eric@webkit.org>
 741
 742         Reviewed by mjs.
 743
 744         * sunspider: increase default number of runs for more accurate numbers
 745
 746 2007-12-18  Mark Rowe  <mrowe@apple.com>
 747
 748         Fix typo.
 749
 750         * hosted/sunspider.html:
 751
 752 2007-12-18  Maciej Stachowiak  <mjs@apple.com>
 753
 754         Reviewed by Eric.
 755
 756         - give an extra digit of precision for small differences.
 757
 758         * resources/sunspider-compare-results.js:
 759
 760 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 761
 762         Reviewed by Eric.
 763
 764         - work around IE layout bugs
 765
 766         * hosted/sunspider.css:
 767
 768 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 769
 770         Reviewed by Mark.
 771
 772         - fix compare feature
 773
 774         * hosted/sunspider-results.html:
 775
 776 2007-12-17  Maciej Stachowiak  <mjs@apple.com>
 777
 778         Reviewed by Mark.
 779
 780         - Change the format of the output URL to be shorter, to meet IE constraints
 781
 782         * hosted/sunspider-driver.html:
 783         * hosted/sunspider-results.html:
 784
 785 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 786
 787         Reviewed by Oliver.
 788
 789         - Add some explanatory test and a dumb logo to SunSpider web hosted version
 790
 791         * hosted/sunspider-driver.html:
 792         * hosted/sunspider-results.html:
 793         * hosted/sunspider.css:
 794         * hosted/sunspider.html:
 795
 796 2007-12-13  Maciej Stachowiak  <mjs@apple.com>
 797
 798         Reviewed by Oliver.
 799
 800         - Make the web-hosted version of SunSpider look a little bit less drab.
 801
 802         * hosted/sunspider-driver.html:
 803         * hosted/sunspider-results.html:
 804         * hosted/sunspider.css: Added.
 805         * hosted/sunspider.html:
 806         * resources/TEMPLATE.html:
 807
 808 2007-12-12  Maciej Stachowiak  <mjs@apple.com>
 809
 810         Reviewed by Sam.
 811
 812         - Change sunspider-compare-results output from showing "% faster" values to "x as fast" values.
 813
 814         A sample of what new-style results would look like:
 815
 816         TEST                   COMPARISON            FROM                 TO             DETAILS
 817         ...
 818         ** TOTAL **:           1.41x as fast     6613.6ms +/- 0.1%   4703.0ms +/- 0.1%     significant
 819         ...
 820             3bit-bits-in-byte: *1.36x as slow*     88.8ms +/- 0.6%    121.1ms +/- 0.6%     significant
 821
 822         * resources/sunspider-compare-results.js:
 823
 824 2007-11-14  Eric Seidel  <eric@webkit.org>
 825
 826         Reviewed by Sam.
 827
 828         * sunspider: add --shark-cache for L2 Cache Miss profiling
 829
 830 2007-11-09  Alexey Proskuryakov  <ap@webkit.org>
 831
 832         Reviewed by Darin Adler.
 833
 834         http://bugs.webkit.org/show_bug.cgi?id=15925
 835         SunSpider should check for Shark being installed
 836
 837         * sunspider: Explicitly test that shark command line tool is installed.
 838
 839 2007-11-04  Darin Adler  <darin@apple.com>
 840
 841         * hosted/sunspider.html: Tweaked formatting a bit.
 842
 843 2007-11-03  Maciej Stachowiak  <mjs@apple.com>
 844
 845         Reviewed by Darin Adler.
 846
 847         - Add compare mode and a description of the benchmark to the hosted version.
 848
 849         * hosted/sunspider-results.html: Add compare mode - you can just paste in another results URL.
 850         * hosted/sunspider.html: Added intro text.
 851         * make-hosted: Copy new file.
 852         * resources/sunspider-compare-results.js: Made more reusable by putting the implementation in
 853         a function.
 854         * resources/sunspider-standalone-compare.js: Added. Calls the function in the way standalone mode
 855         expects.
 856         * sunspider-compare-results: Use sunspider-standalone-compare.js.
 857
 858 2007-11-02  Darin Adler  <darin@apple.com>
 859
 860         * sunspider: Changed "--base" to "--set-baseline".
 861
 862 2007-11-02  Darin Adler  <darin@apple.com>
 863
 864         Reviewed by Maciej.
 865
 866         * sunspider: Added a "--base" option that records the current test as a baseline.
 867         Moved the code to find the newest .mshark file into a function (since I needed to use
 868         it in the other script and I prefer to copy/paste a function).
 869
 870         * sunspider-compare-results: Added logic to compare the baseline with the last
 871         run when you don't pass any parameters.
 872
 873 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 874
 875         Reviewed by Eric.
 876
 877         - Added list of ECMAScript3 features that SunSpider does not cover currently.
 878
 879         * UNCOVERED: Added.
 880
 881 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 882
 883         Reviewed by Oliver.
 884
 885         - Give better error messages when sunspider-compare-results fails
 886
 887         * sunspider-compare-results:
 888
 889 2007-10-29  Maciej Stachowiak  <mjs@apple.com>
 890
 891         Reviewed by Darin Adler.
 892
 893         - fix Date tests to work in command-line SpiderMonkey
 894
 895         * tests/date-format-tofte.js:
 896         * tests/date-format-xparb.js:
 897
 898 2007-10-29  Darin Adler  <darin@apple.com>
 899
 900         Reviewed by Maciej.
 901
 902         * sunspider: Added --shark20 option, to run Shark at its highest sample
 903         resolution instead of its default.
 904
 905 2007-10-28  Adam Roben  <aroben@apple.com>
 906
 907         Fix SunSpider on Windows
 908
 909         Reviewed by Eric.
 910
 911         * sunspider: Don't use colons in filenames.
 912
 913 2007-10-28  Darin Adler  <darin@apple.com>
 914
 915         Reviewed by Adam.
 916
 917         * resources/sunspider-analyze-results.js: Tweak the output format a little.
 918         Change so that when there's only one run we don't write out confidence
 919         intervals at all rather than writing "NaN" over and over again.
 920
 921         * sunspider: Use the time and date as a suffix on the results file, that way
 922         all the results are left behind in the tmp directory. This will make it easy
 923         for us to add features that compare past results in the future. Also rename
 924         the .mshark files using the same scheme.
 925
 926         * sunspider-compare-results: Relax the parsing rules so we can parse the
 927         results file exactly as it's emitted from sunspider.
 928
 929 2007-10-25  Adam Roben  <aroben@apple.com>
 930
 931         Strip carriage returns from results
 932
 933         Carriage returns were screwing up sunspider-compare-results on
 934         Windows.
 935
 936         Rubberstamped by Oliver.
 937
 938         * sunspider:
 939
 940 2007-10-24  Darin Adler  <darin@apple.com>
 941
 942         Reviewed by Eric.
 943
 944         * sunspider: Open the profile in shark after completing a run.
 945         * sunspider-compare-results: Tiny perl tweaks.
 946
 947 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 948
 949         Reviewed by Darin Adler.
 950
 951         - Don't hardcode my path to testkjs
 952
 953         * sunspider-compare-results:
 954
 955 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 956
 957         Reviewed by Darin Adler.
 958
 959         - Add a compare script to compare results, and improve formatting of standard results a bit.
 960
 961         * resources/sunspider-analyze-results.js:
 962         * resources/sunspider-compare-results.js: Added.
 963         * sunspider-compare-results: Added.
 964
 965 2007-10-23  Maciej Stachowiak  <mjs@apple.com>
 966
 967         Reviewed by Darin Adler.
 968
 969         * resources/sunspider-analyze-results.js: Do the error range
 970         properly, using the t-distribution instead of 1.96 (which was
 971         based on the normal distribution).
 972
 973         * sunspider: Print results in a way that will be friendlier to the
 974         soon-to-come compare mode.
 975
 976 2007-10-22  Maciej Stachowiak  <mjs@apple.com>
 977
 978         Reviewed by Review.
 979
 980         - add more tests (probably the final set for now)
 981
 982         Already balanced. Added date, regexp, control flow, and a few
 983         more string and object/array access tests.
 984
 985         * tests/LIST:
 986         * tests/access-fannkuch.js: Added.
 987         * tests/access-nbody.js: Added.
 988         * tests/controlflow-recursive.js: Added.
 989         * tests/date-format-tofte.js: Added.
 990         * tests/date-format-xparb.js: Added.
 991         * tests/regexp-dna.js: Added.
 992         * tests/string-validate-input.js: Added.
 993
 994 2007-10-21  Eric Seidel  <eric@webkit.org>
 995
 996         Reviewed by mjs.
 997
 998         * sunspider: make --shark only sample from testkjs
 999
1000 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1001
1002         Rubber stamped by Eric.
1003
1004         - added crypto tests (already balanced)
1005
1006         * tests/LIST:
1007         * tests/crypto-aes.js: Added.
1008         * tests/crypto-md5.js: Added.
1009         * tests/crypto-sha1.js: Added.
1010
1011 2007-10-21  Maciej Stachowiak  <mjs@apple.com>
1012
1013         Reviewed by Adam.
1014
1015         - rebalance new tests
1016
1017         Scaled the new tests so they take 400-600ms on a reasonably normal user setup:
1018
1019         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1020
1021         This balances them with the existing tests.
1022
1023         * tests/access-binary-trees.js:
1024         * tests/access-nsieve.js:
1025         * tests/bitops-nsieve-bits.js:
1026         * tests/math-partial-sums.js:
1027         * tests/math-spectral-norm.js:
1028         * tests/string-fasta.js:
1029
1030 2007-10-20  Eric Seidel  <eric@webkit.org>
1031
1032         Reviewed by darin.
1033
1034         Add improved argument handling to sunspider, including
1035         --runs=<number>, --tests=<pattern, --shark, and --help
1036         Also re-factor code into subroutines
1037
1038         * sunspider: improved argument handling
1039
1040 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1041
1042         Reviewed by Mark.
1043
1044         - Add more new tests, mostly from the computer language shootout. Not normalized yet.
1045
1046         * TODO:
1047         * tests/LIST:
1048         * tests/access-binary-trees.js: Added.
1049         * tests/access-nsieve.js: Added.
1050         * tests/bitops-nsieve-bits.js: Added.
1051         * tests/math-partial-sums.js: Added.
1052         * tests/math-spectral-norm.js: Added.
1053         * tests/string-fasta.js: Added.
1054
1055 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1056
1057         Reviewed by Darin Adler.
1058
1059         * TODO: Updated to note areas that are now well-covered.
1060
1061         - Change the tests and the driver to avoid leaving large object
1062         graphs hanging around, since that throws off the subsequent tests.
1063
1064         * resources/sunspider-standalone-driver.js:
1065         * tests/3d-cube.js:
1066         * tests/3d-morph.js:
1067         * tests/3d-raytrace.js:
1068         * tests/bitops-bitwise-and.js:
1069         * tests/string-base64.js:
1070         * tests/string-tagcloud.js:
1071
1072 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1073
1074         Reviewed by Nikolas.
1075
1076         - Rebalance test complexity.
1077
1078         Scaled all the tests so they take 400-600ms on a reasonably normal user setup:
1079
1080         Internet Explorer 7, Window XP, Dell Latitude D810 with a 2.13GHz Pentium M
1081
1082         However, for some tests, IE7 was a huge outlier compared to other
1083         major browsers. For those tests, I used Firefox 2.0.0.8 on the
1084         same OS and hardware as a normalization basis.
1085
1086         * tests/3d-morph.js:
1087         * tests/3d-raytrace.js:
1088         * tests/bitops-3bit-bits-in-byte.js:
1089         * tests/bitops-bits-in-byte.js:
1090         * tests/bitops-bitwise-and.js:
1091         * tests/math-cordic.js:
1092         * tests/string-base64.js:
1093         * tests/string-tagcloud.js:
1094         * tests/string-unpack-code.js:
1095
1096 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1097
1098         Reviewed by Oliver.
1099
1100         - Fix the browser-hosted driver to be compatible with Safari 2
1101
1102         * hosted/sunspider-driver.html:
1103
1104 2007-10-20  Maciej Stachowiak  <mjs@apple.com>
1105
1106         Rubber stamped by Hyatt.
1107
1108         - Add BSD license notice to cordic test, the original author said to do so.
1109
1110         * tests/math-cordic.js:
1111
1112 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1113
1114         Reviewed by Eric.
1115
1116         - Support Windows and Windows browsers.
1117
1118         * hosted/sunspider-results.html: Display results correctly in IE.
1119         * sunspider: tweaks to work on Windows, and with spaces in the path.
1120
1121 2007-10-18  Maciej Stachowiak  <mjs@apple.com>
1122
1123         Rubber stamped by Oliver.
1124
1125         * tests/string-tagcloud.js: Remove stray print()
1126
1127 2007-10-17  Maciej Stachowiak  <mjs@apple.com>
1128
1129         Reviewed by Oliver.
1130
1131         Add some string test cases.
1132
1133         * tests/string-base64.js: Added. Does base64 encoding and decoding.
1134         * tests/string-tagcloud.js: Added. Parses a JSON string and
1135         generates tagcloud HTML markup based on the data.
1136         * tests/string-unpack-code.js: Added. Unpack compressed versions of
1137         four of the most popular JavaScript libraries.
1138         * tests/LIST: Added the new tests.
1139
1140 2007-10-16  Maciej Stachowiak  <mjs@apple.com>
1141
1142         Reviewed by Anders.
1143
1144         * TODO: Updated with info on how to do the stats properly.
1145
1146         * hosted/sunspider-driver.html: Reduce timeout from 500 to 200 to
1147         avoid overly slowing down the benchmark.
1148
1149         Added some 3D-related test cases.
1150
1151         * tests/3d-cube.js: Added.
1152         * tests/3d-morph.js: Added.
1153         * tests/3d-raytrace.js: Added.
1154         * tests/LIST:
1155
1156 2007-10-14  Maciej Stachowiak  <mjs@apple.com>
1157
1158         Reviewed by Darin Adler.
1159
1160         - New JavaScript benchmark
1161         http://bugs.webkit.org/show_bug.cgi?id=15515
1162
1163         A start on a new JavaScript benchmark, with standalone and
1164         browser-hosted drivers.
1165
1166         The test content is still incomplete and the drivers could use
1167         some more features but there is enough here to make a good start.
1168
1169         The drivers compute a 95% confidence interval on the mean for the
1170         whole test, each category, and each individual test to make it
1171         easier to tell whether differences are statistically
1172         significant. The confidence interval can be narrowed by running
1173         the test more times.
1174
1175         * TODO: Added. Remaining things that need to be done.
1176         * sunspider: Added. Perl script that acts as the standalone test driver.
1177         * resources/sunspider-standalone-driver.js: Added. JavaScript part
1178         of standalone test driver.
1179         * resources/sunspider-analyze-results.js: Added. JavaScript statistical
1180         analysis code.
1181         * resources/TEMPLATE.html: Added. Template for browser-hosted tests.
1182         * make-hosted: Added. Script to generate browser-hosted tests.
1183         * hosted/sunspider.html: Added. Start page for browser-hosted test.
1184         * hosted/sunspider-driver.html: Added. Driver for browser-hosted tests.
1185         * hosted/sunspider-results.html: Added. Results page for browser-hosted
1186         tests.
1187         * hosted/sunspider-record-result.js: Added. Helper file for browser-hosted
1188         tests.
1189         * tests/LIST: Added. List of tests to use.
1190         * tests/bitops-3bit-bits-in-byte.js: Added. Some initial test content.
1191         * tests/bitops-bits-in-byte.js: Added. Ditto.
1192         * tests/bitops-bitwise-and.js: Added. Ditto.
1193         * tests/math-cordic.js: Added. Ditto.
1194