[wasm] Add Wasm.Build tests, for testing wasm app builds (#47683)
authorAnkit Jain <radical@gmail.com>
Tue, 9 Mar 2021 19:01:38 +0000 (14:01 -0500)
committerGitHub <noreply@github.com>
Tue, 9 Mar 2021 19:01:38 +0000 (14:01 -0500)
commit8c64e3073e9c32e081ebd9df8141fbb2438b06d7
tree3ee782d9f9a0f452674852d58fd5c25c44c09fbb
parente0bbfa0f555cf5b9d8b8749e1a91a78924c9d927
[wasm] Add Wasm.Build tests, for testing wasm app builds (#47683)

* [wasm] Add Wasm.Build tests, for testing wasm app builds

These tests will build wasm test projects, as part of each test method,
and run them.

Other library tests are run with `xharness`, and the test assembly is run
under wasm.  But here we want to run them with xunit, outside wasm. So,
this has different requirements for the helix payload, eg, the sdk,
xunit console runner etc.

To make it work, a new Scenario - `BuildWasmApps` is added, which emits
it's archives in a `buildwasmapps/` folder, which makes it easy to pick
up for the helix test run.

The tests are added under `src/tests/BuildWasmApps/Wasm.Build.Tests`, but
they use `Directory.Build*` from `src/libraries`, similar to how
FunctionalTests do it.

Another use case of this kinda scenario are the wasm debugger tests,
in which the individual test methods launch wasm apps, and then debug
them. (TBD)

Tests:

- The initial set of tests are just proof-of-concept, and more will be
  added once this is merged.

Note: The individual tests build test projects, and then run them with
`xharness`, under `v8`, and Chrome.

* [wasm] Disable il stripping completely

This uses `mono-cil-strip` from a mono installation. And in it's current
form it can cause issues, so disabling it for now.

* Bump helix timeout for tests from 30m to 60m

* [wasm] Cleanup builing RunScriptCommand

.. this allows supporting other properties from xunit*targets, eg. to
run a particular test `$(XUnitMethodName)`, which adds `-method foobar`
to the command line.

* [wasm] Fix timeout string, 00:60:00 to 01:00:00

* [wasm] Fix path to build support dir

* cleanup

* [wasm] fix InvariantGlobalization test

* [wasm] cleanup Wasm.Build.Tests.csproj

* [wasm] Add `include_aot` param for the test data

* [wasm] Enable verbose output for xunit

Instead of writing all the output to stdout also, use `-verbose` which
gives output like:

```
      Wasm.Build.Tests.WasmBuildAppTest.InvariantGlobalization(config: "Debug", aot: False, invariantGlobalization: null) [STARTING]
  ============== wasm test =============
  ============== wasm test-browser =============
      Wasm.Build.Tests.WasmBuildAppTest.InvariantGlobalization(config: "Debug", aot: False, invariantGlobalization: null) [FINISHED] Time: 8.6357275s
```

We log the detailed output to files anyway.

* [wasm] fix tests

* [wasm] Really enable verbose output for xunit, this time

* [wasm] Update tests to track the xharness fix for expected-exit-code

* [wasm] Bump browser job's timeout from 120 to 180 mins

* Improve comment

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
* Update eng/testing/tests.mobile.targets

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
* Remove unrelated commit

Instead, this is moved to a different AOT PR.

Revert "[wasm] Disable il stripping completely"

This reverts commit 25c2340a636be7d8973c09b6808a20466fdcd296.

* Revert "Remove unrelated commit"

This is needed because `mono-cil-strip` isn't available on helix. And we
want to disable cil stripping anyway.

This reverts commit ead13ee3d9c6d53a22b3c3051542057373c77b31.

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
14 files changed:
eng/pipelines/runtime.yml
eng/testing/tests.mobile.targets
eng/testing/xunit/xunit.console.targets
src/libraries/Directory.Build.props
src/libraries/sendtohelixhelp.proj
src/libraries/tests.proj
src/mono/wasm/Makefile
src/mono/wasm/build/WasmApp.LocalBuild.props
src/mono/wasm/build/WasmApp.targets
src/tests/BuildWasmApps/Directory.Build.props [new file with mode: 0644]
src/tests/BuildWasmApps/Directory.Build.targets [new file with mode: 0644]
src/tests/BuildWasmApps/Wasm.Build.Tests/Wasm.Build.Tests.csproj [new file with mode: 0644]
src/tests/BuildWasmApps/Wasm.Build.Tests/WasmBuildAppTest.cs [new file with mode: 0644]
src/tests/Common/dirs.proj