[wasm] Add Wasm.Build tests, for testing wasm app builds (#47683)
authorAnkit Jain <radical@gmail.com>
Tue, 9 Mar 2021 19:01:38 +0000 (14:01 -0500)
committerGitHub <noreply@github.com>
Tue, 9 Mar 2021 19:01:38 +0000 (14:01 -0500)
* [wasm] Add Wasm.Build tests, for testing wasm app builds

These tests will build wasm test projects, as part of each test method,
and run them.

Other library tests are run with `xharness`, and the test assembly is run
under wasm.  But here we want to run them with xunit, outside wasm. So,
this has different requirements for the helix payload, eg, the sdk,
xunit console runner etc.

To make it work, a new Scenario - `BuildWasmApps` is added, which emits
it's archives in a `buildwasmapps/` folder, which makes it easy to pick
up for the helix test run.

The tests are added under `src/tests/BuildWasmApps/Wasm.Build.Tests`, but
they use `Directory.Build*` from `src/libraries`, similar to how
FunctionalTests do it.

Another use case of this kinda scenario are the wasm debugger tests,
in which the individual test methods launch wasm apps, and then debug
them. (TBD)

Tests:

- The initial set of tests are just proof-of-concept, and more will be
  added once this is merged.

Note: The individual tests build test projects, and then run them with
`xharness`, under `v8`, and Chrome.

* [wasm] Disable il stripping completely

This uses `mono-cil-strip` from a mono installation. And in it's current
form it can cause issues, so disabling it for now.

* Bump helix timeout for tests from 30m to 60m

* [wasm] Cleanup builing RunScriptCommand

.. this allows supporting other properties from xunit*targets, eg. to
run a particular test `$(XUnitMethodName)`, which adds `-method foobar`
to the command line.

* [wasm] Fix timeout string, 00:60:00 to 01:00:00

* [wasm] Fix path to build support dir

* cleanup

* [wasm] fix InvariantGlobalization test

* [wasm] cleanup Wasm.Build.Tests.csproj

* [wasm] Add `include_aot` param for the test data

* [wasm] Enable verbose output for xunit

Instead of writing all the output to stdout also, use `-verbose` which
gives output like:

```
      Wasm.Build.Tests.WasmBuildAppTest.InvariantGlobalization(config: "Debug", aot: False, invariantGlobalization: null) [STARTING]
  ============== wasm test =============
  ============== wasm test-browser =============
      Wasm.Build.Tests.WasmBuildAppTest.InvariantGlobalization(config: "Debug", aot: False, invariantGlobalization: null) [FINISHED] Time: 8.6357275s
```

We log the detailed output to files anyway.

* [wasm] fix tests

* [wasm] Really enable verbose output for xunit, this time

* [wasm] Update tests to track the xharness fix for expected-exit-code

* [wasm] Bump browser job's timeout from 120 to 180 mins

* Improve comment

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
* Update eng/testing/tests.mobile.targets

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
* Remove unrelated commit

Instead, this is moved to a different AOT PR.

Revert "[wasm] Disable il stripping completely"

This reverts commit 25c2340a636be7d8973c09b6808a20466fdcd296.

* Revert "Remove unrelated commit"

This is needed because `mono-cil-strip` isn't available on helix. And we
want to disable cil stripping anyway.

This reverts commit ead13ee3d9c6d53a22b3c3051542057373c77b31.

Co-authored-by: Mitchell Hwang <mitchhwang1418@gmail.com>
14 files changed:
eng/pipelines/runtime.yml
eng/testing/tests.mobile.targets
eng/testing/xunit/xunit.console.targets
src/libraries/Directory.Build.props
src/libraries/sendtohelixhelp.proj
src/libraries/tests.proj
src/mono/wasm/Makefile
src/mono/wasm/build/WasmApp.LocalBuild.props
src/mono/wasm/build/WasmApp.targets
src/tests/BuildWasmApps/Directory.Build.props [new file with mode: 0644]
src/tests/BuildWasmApps/Directory.Build.targets [new file with mode: 0644]
src/tests/BuildWasmApps/Wasm.Build.Tests/Wasm.Build.Tests.csproj [new file with mode: 0644]
src/tests/BuildWasmApps/Wasm.Build.Tests/WasmBuildAppTest.cs [new file with mode: 0644]
src/tests/Common/dirs.proj

index 564cbdf..1b5b41d 100644 (file)
@@ -282,7 +282,7 @@ jobs:
       testGroup: innerloop
       nameSuffix: AllSubsets_Mono
       buildArgs: -s mono+libs+host+packs+libs.tests -c $(_BuildConfig) /p:ArchiveTests=true
-      timeoutInMinutes: 120
+      timeoutInMinutes: 180
       condition: >-
         or(
           eq(dependencies.evaluate_paths.outputs['SetPathVars_libraries.containsChange'], true),
@@ -297,6 +297,7 @@ jobs:
         scenarios:
         - normal
         - wasmtestonbrowser
+        - buildwasmapps
         condition: >-
           or(
           eq(variables['librariesContainsChange'], true),
index 56a9db8..18fb245 100644 (file)
@@ -12,7 +12,7 @@
 
   <PropertyGroup Condition="'$(TargetOS)' == 'Browser' And '$(UseDefaultBlazorWASMFeatureSwitches)' != 'false'">
     <!-- We need to set this in order to get extensibility on xunit category traits and other arguments we pass down to xunit via MSBuild properties -->
-    <RunScriptCommand Condition="'$(IsFunctionalTest)' != 'true'">$HARNESS_RUNNER wasm $XHARNESS_COMMAND  --app=. --engine=$(JSEngine) $(JSEngineArgs) --js-file=runtime.js --output-directory=$XHARNESS_OUT --  $(RunTestsJSArguments) --run WasmTestRunner.dll $(AssemblyName).dll</RunScriptCommand>
+    <RunScriptCommand Condition="'$(IsFunctionalTest)' != 'true' and '$(Scenario)' != 'BuildWasmApps'">$HARNESS_RUNNER wasm $XHARNESS_COMMAND  --app=. --engine=$(JSEngine) $(JSEngineArgs) --js-file=runtime.js --output-directory=$XHARNESS_OUT --  $(RunTestsJSArguments) --run WasmTestRunner.dll $(AssemblyName).dll</RunScriptCommand>
     <RunScriptCommand Condition="'$(IsFunctionalTest)' == 'true'">$HARNESS_RUNNER wasm $XHARNESS_COMMAND  --app=. --engine=$(JSEngine) $(JSEngineArgs) --js-file=runtime.js --output-directory=$XHARNESS_OUT --expected-exit-code=$(ExpectedExitCode) --  $(RunTestsJSArguments) --run $(AssemblyName).dll --testing</RunScriptCommand>
     <EventSourceSupport>false</EventSourceSupport>
     <UseSystemResourceKeys>true</UseSystemResourceKeys>
   <Import Project="$(MonoProjectRoot)\wasm\build\WasmApp.InTree.targets" Condition="'$(TargetOS)' == 'Browser'" />
   <PropertyGroup>
       <WasmBuildAppDependsOn>PrepareForWasmBuildApp;$(WasmBuildAppDependsOn)</WasmBuildAppDependsOn>
+      <EmSdkDir>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'src', 'mono', 'wasm', 'emsdk'))</EmSdkDir>
   </PropertyGroup>
 
-  <Target Condition="'$(TargetOS)' == 'Browser'" Name="BundleTestWasmApp" DependsOnTargets="WasmBuildApp" />
+  <Target Condition="'$(TargetOS)' == 'Browser'" Name="BundleTestWasmApp" DependsOnTargets="WasmBuildApp;StageEmSdkForHelix" />
+
+  <!-- CI has emscripten provisioned in $(EMSDK_PATH) as `/usr/local/emscripten`. Because helix tasks will
+   attempt to write a .payload file, we cannot use $(EMSDK_PATH) to package emsdk as a helix correlation 
+   payload. Instead, we copy over the files to a new directory `src/mono/wasm/emsdk` and use that. -->
+  <Target Name="StageEmSdkForHelix" Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' == 'BuildWasmApps' and '$(ContinuousIntegrationBuild)' == 'true' and !Exists($(EmSdkDir))">
+    <Error Condition="!Exists($(EMSDK_PATH))" Text="Could not find emscripten sdk in $(EmSdkDir) or in EMSDK_PATH=$(EMSDK_PATH)" />
+
+    <ItemGroup>
+      <EmSdkFiles Include="$(EMSDK_PATH)\**\*" Exclude="$(EMSDK_PATH)\.git\**\*" />
+    </ItemGroup>
+
+    <MakeDir Directories="$(EmSdkDir)" />
+    <Copy SourceFiles="@(EmSdkFiles)" DestinationFolder="$(EmSdkDir)\%(RecursiveDir)" />
+  </Target>
 
   <Target Condition="'$(TargetOS)' == 'Browser'" Name="PrepareForWasmBuildApp">
     <PropertyGroup>
index ee70055..a3e7d26 100644 (file)
@@ -62,7 +62,7 @@
   </Target>
 
   <!-- ResolveAssemblyReferences is the target that populates ReferenceCopyLocalPaths which is what is copied to output directory. -->
-  <Target Name="CopyRunnerToOutputDirectory" BeforeTargets="ResolveAssemblyReferences" Condition="'$(TargetsMobile)' != 'true'">
+  <Target Name="CopyRunnerToOutputDirectory" BeforeTargets="ResolveAssemblyReferences" Condition="'$(TargetsMobile)' != 'true' or '$(BundleXunitRunner)' == 'true'">
     <ItemGroup>
       <!-- Copy test runner to output directory -->
       <None Include="$([System.IO.Path]::GetDirectoryName('$(XunitConsole472Path)'))\*"
index ad7f4e3..7803ca3 100644 (file)
     <TestArchiveRoot>$(ArtifactsDir)helix/</TestArchiveRoot>
     <TestArchiveTestsRoot Condition="$(IsFunctionalTest) != true">$(TestArchiveRoot)tests/</TestArchiveTestsRoot>
     <TestArchiveTestsRoot Condition="$(IsFunctionalTest) == true">$(TestArchiveRoot)runonly/</TestArchiveTestsRoot>
+    <TestArchiveTestsRoot Condition="'$(Scenario)' == 'BuildWasmApps'">$(TestArchiveRoot)buildwasmapps/</TestArchiveTestsRoot>
     <TestArchiveTestsDir>$(TestArchiveTestsRoot)$(OSPlatformConfig)/</TestArchiveTestsDir>
     <TestArchiveRuntimeRoot>$(TestArchiveRoot)runtime/</TestArchiveRuntimeRoot>
 
     <EnableDefaultItems>false</EnableDefaultItems>
   </PropertyGroup>
 
-  <ItemGroup Condition="'$(IsTestProject)' == 'true'">
+  <ItemGroup Condition="'$(IsTestProject)' == 'true' and '$(SkipTestUtilitiesReference)' != 'true'">
     <ProjectReference Include="$(CommonTestPath)TestUtilities\TestUtilities.csproj" />
   </ItemGroup>
 
index 3269b7c..4efa708 100644 (file)
@@ -28,6 +28,7 @@
     <_workItemTimeout Condition="'$(_workItemTimeout)' == '' and ('$(TargetOS)' == 'iOS' or '$(TargetOS)' == 'tvOS' or '$(TargetOS)' == 'Android')">00:30:00</_workItemTimeout>
     <_workItemTimeout Condition="'$(Scenario)' == '' and '$(_workItemTimeout)' == '' and ('$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm')">00:45:00</_workItemTimeout>
     <_workItemTimeout Condition="'$(Scenario)' != '' and '$(_workItemTimeout)' == '' and ('$(TargetArchitecture)' == 'arm64' or '$(TargetArchitecture)' == 'arm')">01:00:00</_workItemTimeout>
+    <_workItemTimeout Condition="'$(Scenario)' == 'BuildWasmApps' and '$(_workItemTimeout)' == ''">01:00:00</_workItemTimeout>
     <_workItemTimeout Condition="'$(Scenario)' == '' and '$(_workItemTimeout)' == ''">00:15:00</_workItemTimeout>
     <_workItemTimeout Condition="'$(Scenario)' != '' and '$(_workItemTimeout)' == ''">00:30:00</_workItemTimeout>
 
     <HelixPreCommand Include="export XHARNESS_LOG_WITH_TIMESTAMPS=true" />
   </ItemGroup>
 
-  <ItemGroup Condition="'$(Scenario)' == 'WasmTestOnBrowser'">
+  <ItemGroup Condition="'$(Scenario)' == 'WasmTestOnBrowser' or '$(Scenario)' == 'BuildWasmApps'">
     <HelixPreCommand Include="export PATH=$HELIX_CORRELATION_PAYLOAD/chromedriver_linux64:$PATH" />
     <HelixPreCommand Include="export PATH=$HELIX_CORRELATION_PAYLOAD/chrome-linux:$PATH" />
   </ItemGroup>
 
+  <PropertyGroup Condition="'$(Scenario)' == 'BuildWasmApps'">
+    <IncludeXHarnessCli>true</IncludeXHarnessCli>
+    <IncludeDotNetCli>true</IncludeDotNetCli>
+    <DotNetCliPackageType>sdk</DotNetCliPackageType>
+    <GlobalJsonContent>$([System.IO.File]::ReadAllText('$(RepoRoot)global.json'))</GlobalJsonContent>
+    <DotNetCliVersion>$([System.Text.RegularExpressions.Regex]::Match($(GlobalJsonContent), '(%3F&lt;="dotnet": ").*(%3F=")'))</DotNetCliVersion>
+  </PropertyGroup>
+
   <!-- HelixPreCommands is a set of commands run before the work item command. We use it here to inject
        setting up the per-scenario environment.
   -->
       <ChromiumRevision>768968</ChromiumRevision>
       <ChromiumUrl>https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/$(ChromiumRevision)/chrome-linux.zip</ChromiumUrl>
       <SeleniumUrl>https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/$(ChromiumRevision)/chromedriver_linux64.zip</SeleniumUrl>
+      <EmSdkDir>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'src', 'mono', 'wasm', 'emsdk'))</EmSdkDir>
+      <WasmBuildTargetsDir>$([MSBuild]::NormalizeDirectory('$(RepoRoot)', 'src', 'mono', 'wasm', 'build'))</WasmBuildTargetsDir>
     </PropertyGroup>
 
     <PropertyGroup Condition="'$(RuntimeFlavor)' == 'CoreCLR' and '$(BUILD_BUILDID)' != ''">
       </HelixPostCommands>
     </PropertyGroup>
 
+    <ItemGroup Condition="'$(Scenario)' == 'BuildWasmApps'">
+      <HelixCorrelationPayload Include="$(EmSdkDir)"                             Destination="build/emsdk" />
+      <HelixCorrelationPayload Include="$(WasmAppBuilderDir)"                    Destination="build/WasmAppBuilder" />
+      <HelixCorrelationPayload Include="$(MonoAOTCompilerDir)"                   Destination="build/MonoAOTCompiler" />
+      <HelixCorrelationPayload Include="$(MicrosoftNetCoreAppRuntimePackDir)"    Destination="build/microsoft.netcore.app.runtime.browser-wasm" />
+      <HelixCorrelationPayload Include="$(WasmBuildTargetsDir)"                  Destination="build/wasm" />
+    </ItemGroup>
+
     <ItemGroup Condition="'$(TargetOS)' != 'Android' and '$(TargetOS)' != 'iOS' and '$(TargetOS)' != 'tvOS'">
       <HelixCorrelationPayload Include="$(HelixCorrelationPayload)"
                                Condition="'$(IncludeHelixCorrelationPayload)' == 'true' and '$(TargetOS)' != 'Browser'" />
       <HelixCorrelationPayload Include="chromedriver" Uri="$(SeleniumUrl)" Condition="'$(TargetOS)' == 'Browser'" />
 
       <_WorkItem Include="$(WorkItemArchiveWildCard)" Exclude="$(HelixCorrelationPayload)" />
-      <_WorkItem Include="$(TestArchiveRoot)runonly/**/WebAssembly.Console.*.Test.zip" Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' != 'WasmTestOnBrowser'" />
+      <_WorkItem Include="$(TestArchiveRoot)runonly/**/WebAssembly.Console.*.Test.zip" Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' != 'WasmTestOnBrowser' and '$(Scenario)' != 'BuildWasmApps'" />
       <_WorkItem Include="$(TestArchiveRoot)runonly/**/WebAssembly.Browser.*.Test.zip" Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' == 'WasmTestOnBrowser'" />
 
       <HelixWorkItem Include="@(_WorkItem -> '%(FileName)')">
       </HelixWorkItem>
     </ItemGroup>
 
-    <ItemGroup Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' != 'WasmTestOnBrowser'">
+    <ItemGroup Condition="'$(TargetOS)' == 'Browser' and '$(Scenario)' != 'WasmTestOnBrowser' and '$(Scenario)' != 'BuildWasmApps'">
       <!-- Create a work item for run-only WASM console app  -->
       <_RunOnlyWorkItem Include="$(TestArchiveRoot)runonly/**/*.Console.Sample.zip" />
       <HelixWorkItem Include="@(_RunOnlyWorkItem -> '%(FileName)')" >
index e2c0a92..a46f287 100644 (file)
                       Condition="'$(TestTrimming)' == 'true'"
                       AdditionalProperties="%(AdditionalProperties);SkipTrimmingProjectsRestore=true" />
     <ProjectReference Include="@(TrimmingTestProjects)" />
+    <ProjectReference Include="$(RepoRoot)\src\tests\BuildWasmApps\**\*.Tests.csproj"
+                      Exclude="@(ProjectExclusions)"
+                      Condition="'$(TargetOS)' == 'Browser' and '$(RunAOTCompilation)' != 'true'"
+                      BuildInParallel="false" />
   </ItemGroup>
 
   <ItemGroup Condition="'$(ArchiveTests)' == 'true' and '$(TargetOS)' == 'iOS'">
index a3e15f8..cbb7e16 100644 (file)
@@ -182,6 +182,9 @@ run-tests-jsc-%:
 run-tests-%:
        PATH="$(JSVU):$(PATH)" $(DOTNET) build $(TOP)/src/libraries/$*/tests/ /t:Test $(_MSBUILD_WASM_BUILD_ARGS) $(MSBUILD_ARGS)
 
+run-build-tests:
+       PATH="$(JSVU):$(PATH)" $(DOTNET) build $(TOP)/src/tests/BuildWasmApps/Wasm.Build.Tests/ /t:Test $(_MSBUILD_WASM_BUILD_ARGS) $(MSBUILD_ARGS)
+
 run-browser-tests-%:
        PATH="$(GECKODRIVER):$(CHROMEDRIVER):$(PATH)" XHARNESS_COMMAND="test-browser --browser=$(XHARNESS_BROWSER)" $(DOTNET) build $(TOP)/src/libraries/$*/tests/ /t:Test $(_MSBUILD_WASM_BUILD_ARGS) $(MSBUILD_ARGS)
 
index b545e57..a56d713 100644 (file)
         $(WasmBuildSupportDir) - directory which has all the tasks, targets, and runtimepack
 -->
 <Project>
+  <Import Project="$(MSBuildThisFileDirectory)WasmApp.props" />
+
   <PropertyGroup>
-    <_NetCoreAppToolCurrent>net5.0</_NetCoreAppToolCurrent>
+    <_NetCoreAppToolCurrent>net6.0</_NetCoreAppToolCurrent>
   </PropertyGroup>
 
   <PropertyGroup Condition="'$(RuntimeSrcDir)' != '' and '$(WasmBuildSupportDir)' == ''">
@@ -52,6 +54,4 @@
     <WasmBuildTasksAssemblyPath>$([MSBuild]::NormalizePath('$(WasmBuildTasksDir)', 'WasmBuildTasks.dll'))</WasmBuildTasksAssemblyPath>
     <MonoAOTCompilerTasksAssemblyPath>$([MSBuild]::NormalizePath('$(MonoAOTCompilerDir)', 'MonoAOTCompiler.dll'))</MonoAOTCompilerTasksAssemblyPath>
   </PropertyGroup>
-
-  <Import Project="$(MSBuildThisFileDirectory)WasmApp.props" />
 </Project>
index 0bd700e..a903819 100644 (file)
   -->
 
   <PropertyGroup>
-    <WasmStripAOTAssemblies Condition="'$(AOTMode)' == 'AotInterp'">false</WasmStripAOTAssemblies>
-    <WasmStripAOTAssemblies Condition="'$(WasmStripAOTAssemblies)' == ''">$(RunAOTCompilation)</WasmStripAOTAssemblies>
+    <WasmStripAOTAssemblies>false</WasmStripAOTAssemblies>
+
+    <!--<WasmStripAOTAssemblies Condition="'$(AOTMode)' == 'AotInterp'">false</WasmStripAOTAssemblies>-->
+    <!--<WasmStripAOTAssemblies Condition="'$(WasmStripAOTAssemblies)' == ''">$(RunAOTCompilation)</WasmStripAOTAssemblies>-->
     <_ExeExt Condition="$([MSBuild]::IsOSPlatform('WINDOWS'))">.exe</_ExeExt>
   </PropertyGroup>
 
@@ -69,6 +71,7 @@
   <Target Name="_WasmAotCompileApp" Condition="'$(RunAOTCompilation)' == 'true'">
     <Error Condition="'@(_WasmAssemblies)' == ''" Text="Item _WasmAssemblies is empty" />
     <Error Condition="'$(EMSDK_PATH)' == ''" Text="%24(EMSDK_PATH) should be set to emscripten sdk" />
+    <Error Condition="!Exists($(EMSDK_PATH))" Text="Cannot find EMSDK_PATH=$(EMSDK_PATH)" />
 
     <ItemGroup>
       <MonoAOTCompilerDefaultAotArguments Include="no-opt" />
@@ -323,7 +326,7 @@ EMSCRIPTEN_KEEPALIVE void mono_wasm_load_profiler_aot (const char *desc) { mono_
     <Error Condition="'$(WasmMainAssemblyFileName)' == ''" Text="%24(WasmMainAssemblyFileName) property needs to be set for generating $(WasmRunV8ScriptPath)." />
     <WriteLinesToFile
       File="$(WasmRunV8ScriptPath)"
-      Lines="v8 --expose_wasm runtime.js -- --run $(WasmMainAssemblyFileName) $*"
+      Lines="v8 --expose_wasm runtime.js -- ${RUNTIME_ARGS} --run $(WasmMainAssemblyFileName) $*"
       Overwrite="true">
     </WriteLinesToFile>
 
diff --git a/src/tests/BuildWasmApps/Directory.Build.props b/src/tests/BuildWasmApps/Directory.Build.props
new file mode 100644 (file)
index 0000000..68eaef4
--- /dev/null
@@ -0,0 +1,6 @@
+<Project>
+  <PropertyGroup>
+    <Scenario>BuildWasmApps</Scenario>
+  </PropertyGroup>
+  <Import Project="..\..\libraries\Directory.Build.props" />
+</Project>
diff --git a/src/tests/BuildWasmApps/Directory.Build.targets b/src/tests/BuildWasmApps/Directory.Build.targets
new file mode 100644 (file)
index 0000000..fa2efe4
--- /dev/null
@@ -0,0 +1,8 @@
+<Project>
+  <Import Project="..\..\libraries\Directory.Build.targets" />
+
+  <PropertyGroup>
+    <BundleDir>$(OutDir)</BundleDir>
+    <RunScriptOutputPath>$(OutDir)\RunTests.sh</RunScriptOutputPath>
+  </PropertyGroup>
+</Project>
diff --git a/src/tests/BuildWasmApps/Wasm.Build.Tests/Wasm.Build.Tests.csproj b/src/tests/BuildWasmApps/Wasm.Build.Tests/Wasm.Build.Tests.csproj
new file mode 100644 (file)
index 0000000..a6fe3e4
--- /dev/null
@@ -0,0 +1,34 @@
+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <TargetFrameworks>$(NetCoreAppToolCurrent)</TargetFrameworks>
+    <SkipTestUtilitiesReference>true</SkipTestUtilitiesReference>
+    <IsTestProject>true</IsTestProject>
+    <BundleXunitRunner>true</BundleXunitRunner>
+    <CLRTestKind>BuildAndRun</CLRTestKind>
+    <TestFramework>xunit</TestFramework>
+    <WasmGenerateAppBundle>false</WasmGenerateAppBundle>
+
+    <!-- don't run any wasm build steps -->
+    <WasmBuildAppAfterThisTarget />
+  </PropertyGroup>
+
+  <PropertyGroup Condition="'$(ContinuousIntegrationBuild)' == 'true'">
+    <_PreCommand>WasmBuildSupportDir=%24{HELIX_CORRELATION_PAYLOAD}/build</_PreCommand>
+    <_PreCommand>$(_PreCommand) DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1</_PreCommand>
+  </PropertyGroup>
+
+  <PropertyGroup>
+    <_PreCommand>$(_PreCommand) TEST_LOG_PATH=%24{XHARNESS_OUT}/logs</_PreCommand>
+    <_PreCommand>$(_PreCommand) HARNESS_RUNNER=%24{HARNESS_RUNNER}</_PreCommand>
+
+    <RunScriptCommand>$(_PreCommand) dotnet exec xunit.console.dll $(AssemblyName).dll -xml %24XHARNESS_OUT/testResults.xml</RunScriptCommand>
+    <RunScriptCommand Condition="'$(ContinuousIntegrationBuild)' == 'true'">$(RunScriptCommand) -nocolor</RunScriptCommand>
+    <RunScriptCommand Condition="'$(ContinuousIntegrationBuild)' == 'true' or '$(XUnitShowProgress)' == 'true'">$(RunScriptCommand) -verbose</RunScriptCommand>
+  </PropertyGroup>
+
+  <ItemGroup>
+    <Compile Include="WasmBuildAppTest.cs" />
+
+    <None Include="$(RepoRoot)\src\mono\wasm\runtime-test.js" CopyToOutputDirectory="PreserveNewest" />
+  </ItemGroup>
+</Project>
diff --git a/src/tests/BuildWasmApps/Wasm.Build.Tests/WasmBuildAppTest.cs b/src/tests/BuildWasmApps/Wasm.Build.Tests/WasmBuildAppTest.cs
new file mode 100644 (file)
index 0000000..c6ff9aa
--- /dev/null
@@ -0,0 +1,663 @@
+// Licensed to the .NET Foundation under one or more agreements.
+// The .NET Foundation licenses this file to you under the MIT license.
+
+using System;
+using System.Collections.Generic;
+using System.Diagnostics;
+using System.IO;
+using System.Linq;
+using System.Text;
+using Xunit;
+using Xunit.Abstractions;
+
+#nullable enable
+
+namespace Wasm.Build.Tests
+{
+    public class WasmBuildAppTest : IDisposable
+    {
+        private const string TestLogPathEnvVar = "TEST_LOG_PATH";
+        private const string SkipProjectCleanupEnvVar = "SKIP_PROJECT_CLEANUP";
+        private const string XHarnessRunnerCommandEnvVar = "XHARNESS_CLI_PATH";
+
+        private readonly string _tempDir;
+        private readonly ITestOutputHelper _testOutput;
+        private readonly string _id;
+        private readonly string _logPath;
+
+        private const string s_targetFramework = "net5.0";
+        private static string s_runtimeConfig = "Release";
+        private static string s_runtimePackDir;
+        private static string s_defaultBuildArgs;
+        private static readonly string s_logRoot;
+        private static readonly string s_emsdkPath;
+        private static readonly bool s_skipProjectCleanup;
+        private static readonly string s_xharnessRunnerCommand;
+
+        static WasmBuildAppTest()
+        {
+            DirectoryInfo? solutionRoot = new (AppContext.BaseDirectory);
+            while (solutionRoot != null)
+            {
+                if (File.Exists(Path.Combine(solutionRoot.FullName, "NuGet.config")))
+                {
+                    break;
+                }
+
+                solutionRoot = solutionRoot.Parent;
+            }
+
+            if (solutionRoot == null)
+            {
+                string? buildDir = Environment.GetEnvironmentVariable("WasmBuildSupportDir");
+
+                if (buildDir == null || !Directory.Exists(buildDir))
+                    throw new Exception($"Could not find the solution root, or a build dir: {buildDir}");
+
+                s_emsdkPath = Path.Combine(buildDir, "emsdk");
+                s_runtimePackDir = Path.Combine(buildDir, "microsoft.netcore.app.runtime.browser-wasm");
+                s_defaultBuildArgs = $" /p:WasmBuildSupportDir={buildDir} /p:EMSDK_PATH={s_emsdkPath} ";
+            }
+            else
+            {
+                string artifactsBinDir = Path.Combine(solutionRoot.FullName, "artifacts", "bin");
+                s_runtimePackDir = Path.Combine(artifactsBinDir, "microsoft.netcore.app.runtime.browser-wasm", s_runtimeConfig);
+
+                string? emsdk = Environment.GetEnvironmentVariable("EMSDK_PATH");
+                if (string.IsNullOrEmpty(emsdk))
+                    emsdk = Path.Combine(solutionRoot.FullName, "src", "mono", "wasm", "emsdk");
+                s_emsdkPath = emsdk;
+
+                s_defaultBuildArgs = $" /p:RuntimeSrcDir={solutionRoot.FullName} /p:RuntimeConfig={s_runtimeConfig} /p:EMSDK_PATH={s_emsdkPath} ";
+            }
+
+            string? logPathEnvVar = Environment.GetEnvironmentVariable(TestLogPathEnvVar);
+            if (!string.IsNullOrEmpty(logPathEnvVar))
+            {
+                s_logRoot = logPathEnvVar;
+                if (!Directory.Exists(s_logRoot))
+                {
+                    Directory.CreateDirectory(s_logRoot);
+                }
+            }
+            else
+            {
+                s_logRoot = Environment.CurrentDirectory;
+            }
+
+            string? cleanupVar = Environment.GetEnvironmentVariable(SkipProjectCleanupEnvVar);
+            s_skipProjectCleanup = !string.IsNullOrEmpty(cleanupVar) && cleanupVar == "1";
+
+            string? harnessVar = Environment.GetEnvironmentVariable(XHarnessRunnerCommandEnvVar);
+            if (string.IsNullOrEmpty(harnessVar))
+            {
+                throw new Exception($"{XHarnessRunnerCommandEnvVar} not set");
+            }
+
+            s_xharnessRunnerCommand = harnessVar;
+        }
+
+        public WasmBuildAppTest(ITestOutputHelper output)
+        {
+            _testOutput = output;
+            _id = Path.GetRandomFileName();
+            _tempDir = Path.Combine(AppContext.BaseDirectory, _id);
+            Directory.CreateDirectory(_tempDir);
+
+            _logPath = Path.Combine(s_logRoot, _id);
+            Directory.CreateDirectory(_logPath);
+
+            _testOutput.WriteLine($"Test Id: {_id}");
+        }
+
+
+        /*
+         * TODO:
+            - AOT modes
+                - llvmonly
+                - aotinterp
+                    - skipped assemblies should get have their pinvoke/icall stuff scanned
+
+            - only buildNative
+            - aot but no wrapper - check that AppBundle wasn't generated
+        */
+
+
+        public static TheoryData<string, bool> ConfigWithAOTData(bool include_aot=true)
+        {
+            TheoryData<string, bool> data = new()
+            {
+                { "Debug", false },
+                { "Release", false }
+            };
+
+            if (include_aot)
+            {
+                data.Add("Debug", true);
+                data.Add("Release", true);
+            }
+
+            return data;
+        }
+
+        public static TheoryData<string, bool, bool?> InvariantGlobalizationTestData()
+        {
+            var data = new TheoryData<string, bool, bool?>();
+            foreach (var configData in ConfigWithAOTData())
+            {
+                data.Add((string)configData[0], (bool)configData[1], null);
+                data.Add((string)configData[0], (bool)configData[1], true);
+                data.Add((string)configData[0], (bool)configData[1], false);
+            }
+            return data;
+        }
+
+        // TODO: check that icu bits have been linked out
+        [Theory]
+        [MemberData(nameof(InvariantGlobalizationTestData))]
+        public void InvariantGlobalization(string config, bool aot, bool? invariantGlobalization)
+        {
+            File.WriteAllText(Path.Combine(_tempDir, "Program.cs"), @"
+            using System;
+            using System.Threading.Tasks;
+
+            public class TestClass {
+                public static int Main()
+                {
+                    Console.WriteLine(""Hello, World!"");
+                    return 42;
+                }
+            }
+            ");
+
+            string? extraProperties = null;
+            if (invariantGlobalization != null)
+                extraProperties = $"<InvariantGlobalization>{invariantGlobalization}</InvariantGlobalization>";
+
+            string projectName = $"invariant_{invariantGlobalization?.ToString() ?? "unset"}";
+            BuildProject(projectName, config, aot: aot, extraProperties: extraProperties,
+                        hasIcudt: invariantGlobalization == null || invariantGlobalization.Value == false);
+
+            RunAndTestWasmApp(projectName, config, isAOT: aot, expectedExitCode: 42,
+                                test: output => Assert.Contains("Hello, World!", output));
+        }
+
+        [Theory]
+        [MemberData(nameof(ConfigWithAOTData), parameters: /*aot*/ true)]
+        public void TopLevelMain(string config, bool aot)
+            => TestMain("top_level",
+                    @"System.Console.WriteLine(""Hello, World!""); return await System.Threading.Tasks.Task.FromResult(42);",
+                    config, aot);
+
+        [Theory]
+        [MemberData(nameof(ConfigWithAOTData), parameters: /*aot*/ true)]
+        public void AsyncMain(string config, bool aot)
+            => TestMain("async_main", @"
+            using System;
+            using System.Threading.Tasks;
+
+            public class TestClass {
+                public static async Task<int> Main()
+                {
+                    Console.WriteLine(""Hello, World!"");
+                    return await Task.FromResult(42);
+                }
+            }", config, aot);
+
+        [Theory]
+        [MemberData(nameof(ConfigWithAOTData), parameters: /*aot*/ true)]
+        public void NonAsyncMain(string config, bool aot)
+            => TestMain("non_async_main", @"
+                using System;
+                using System.Threading.Tasks;
+
+                public class TestClass {
+                    public static int Main()
+                    {
+                        Console.WriteLine(""Hello, World!"");
+                        return 42;
+                    }
+                }", config, aot);
+
+        public static TheoryData<string, bool, string[]> MainWithArgsTestData()
+        {
+            var data = new TheoryData<string, bool, string[]>();
+            foreach (var configData in ConfigWithAOTData())
+            {
+                data.Add((string)configData[0], (bool)configData[1], new string[] { "abc", "foobar" });
+                data.Add((string)configData[0], (bool)configData[1], new string[0]);
+            }
+
+            return data;
+        }
+
+        [Theory]
+        [MemberData(nameof(MainWithArgsTestData))]
+        public void NonAsyncMainWithArgs(string config, bool aot, string[] args)
+            => TestMainWithArgs("non_async_main_args", @"
+                public class TestClass {
+                    public static int Main(string[] args)
+                    {
+                        ##CODE##
+                        return 42 + count;
+                    }
+                }", config, aot, args);
+
+        [Theory]
+        [MemberData(nameof(MainWithArgsTestData))]
+        public void AsyncMainWithArgs(string config, bool aot, string[] args)
+            => TestMainWithArgs("async_main_args", @"
+                public class TestClass {
+                    public static async System.Threading.Tasks.Task<int> Main(string[] args)
+                    {
+                        ##CODE##
+                        return await System.Threading.Tasks.Task.FromResult(42 + count);
+                    }
+                }", config, aot, args);
+
+        [Theory]
+        [MemberData(nameof(MainWithArgsTestData))]
+        public void TopLevelWithArgs(string config, bool aot, string[] args)
+            => TestMainWithArgs("top_level_args",
+                                @"##CODE## return await System.Threading.Tasks.Task.FromResult(42 + count);",
+                                config, aot, args);
+
+        void TestMain(string projectName, string programText, string config, bool aot)
+        {
+            File.WriteAllText(Path.Combine(_tempDir, "Program.cs"), programText);
+            BuildProject(projectName, config, aot: aot);
+            RunAndTestWasmApp(projectName, config, isAOT: aot, expectedExitCode: 42,
+                                test: output => Assert.Contains("Hello, World!", output));
+        }
+
+        void TestMainWithArgs(string projectName, string programFormatString, string config, bool aot, string[] args)
+        {
+            string code = @"
+                    int count = args == null ? 0 : args.Length;
+                    System.Console.WriteLine($""args#: {args?.Length}"");
+                    foreach (var arg in args ?? System.Array.Empty<string>())
+                        System.Console.WriteLine($""arg: {arg}"");
+                    ";
+            string programText = programFormatString.Replace("##CODE##", code);
+
+            File.WriteAllText(Path.Combine(_tempDir, "Program.cs"), programText);
+            BuildProject(projectName, config, aot: aot);
+            RunAndTestWasmApp(projectName, config, isAOT: aot, expectedExitCode: 42 + args.Length, args: string.Join(' ', args),
+                test: output =>
+                {
+                    Assert.Contains($"args#: {args.Length}", output);
+                    foreach (var arg in args)
+                        Assert.Contains($"arg: {arg}", output);
+                });
+        }
+
+        private void RunAndTestWasmApp(string projectName, string config, bool isAOT, Action<string> test, int expectedExitCode=0, string? args=null)
+        {
+            Dictionary<string, string>? envVars = new();
+            envVars["XHARNESS_DISABLE_COLORED_OUTPUT"] = "true";
+            if (isAOT)
+            {
+                envVars["EMSDK_PATH"] = s_emsdkPath;
+                envVars["MONO_LOG_LEVEL"] = "debug";
+                envVars["MONO_LOG_MASK"] = "aot";
+            }
+
+            string bundleDir = Path.Combine(GetBinDir(config: config), "AppBundle");
+            string v8output = RunWasmTest(projectName, bundleDir, envVars, expectedExitCode, appArgs: args);
+            Test(v8output);
+
+            string browserOutput = RunWasmTestBrowser(projectName, bundleDir, envVars, expectedExitCode, appArgs: args);
+            Test(browserOutput);
+
+            void Test(string output)
+            {
+                if (isAOT)
+                {
+                    Assert.Contains("AOT: image 'System.Private.CoreLib' found.", output);
+                    Assert.Contains($"AOT: image '{projectName}' found.", output);
+                }
+                else
+                {
+                    Assert.DoesNotContain("AOT: image 'System.Private.CoreLib' found.", output);
+                    Assert.DoesNotContain($"AOT: image '{projectName}' found.", output);
+                }
+            }
+        }
+
+        private string RunWithXHarness(string testCommand, string relativeLogPath, string projectName, string bundleDir, IDictionary<string, string>? envVars=null,
+                                        int expectedAppExitCode=0, int xharnessExitCode=0, string? extraXHarnessArgs=null, string? appArgs=null)
+        {
+            _testOutput.WriteLine($"============== {testCommand} =============");
+            Console.WriteLine($"============== {testCommand} =============");
+            string testLogPath = Path.Combine(_logPath, relativeLogPath);
+
+            StringBuilder args = new();
+            args.Append($"exec {s_xharnessRunnerCommand}");
+            args.Append($" {testCommand}");
+            args.Append($" --app=.");
+            args.Append($" --output-directory={testLogPath}");
+            args.Append($" --expected-exit-code={expectedAppExitCode}");
+            args.Append($" {extraXHarnessArgs ?? string.Empty}");
+
+            args.Append(" -- ");
+            // App arguments
+
+            if (envVars != null)
+            {
+                var setenv = string.Join(' ', envVars.Select(kvp => $"--setenv={kvp.Key}={kvp.Value}").ToArray());
+                args.Append($" {setenv}");
+            }
+
+            args.Append($" --run {projectName}.dll");
+            args.Append($" {appArgs ?? string.Empty}");
+
+            var (exitCode, output) = RunProcess("dotnet",
+                                        args: args.ToString(),
+                                        workingDir: bundleDir,
+                                        envVars: envVars,
+                                        label: testCommand);
+
+            File.WriteAllText(Path.Combine(testLogPath, $"xharness.log"), output);
+
+            if (exitCode != xharnessExitCode)
+            {
+                _testOutput.WriteLine($"Exit code: {exitCode}");
+                Assert.True(exitCode == expectedAppExitCode, $"[{testCommand}] Exit code, expected {expectedAppExitCode} but got {exitCode}");
+            }
+
+            return output;
+        }
+        private string RunWasmTest(string projectName, string bundleDir, IDictionary<string, string>? envVars=null, int expectedAppExitCode=0, int xharnessExitCode=0, string? appArgs=null)
+            => RunWithXHarness("wasm test", "wasm-test", projectName, bundleDir,
+                                    envVars: envVars,
+                                    expectedAppExitCode: expectedAppExitCode,
+                                    extraXHarnessArgs: "--js-file=runtime.js --engine=V8 -v trace",
+                                    appArgs: appArgs);
+
+        private string RunWasmTestBrowser(string projectName, string bundleDir, IDictionary<string, string>? envVars=null, int expectedAppExitCode=0, int xharnessExitCode=0, string? appArgs=null)
+            => RunWithXHarness("wasm test-browser", "wasm-test-browser", projectName, bundleDir,
+                                    envVars: envVars,
+                                    expectedAppExitCode: expectedAppExitCode,
+                                    extraXHarnessArgs: "-v trace", // needed to get messages like those for AOT loading
+                                    appArgs: appArgs);
+
+        private static void InitProjectDir(string dir)
+        {
+            File.WriteAllText(Path.Combine(dir, "Directory.Build.props"), s_directoryBuildProps);
+            File.WriteAllText(Path.Combine(dir, "Directory.Build.targets"), s_directoryBuildTargets);
+        }
+
+        private void BuildProject(string projectName,
+                                  string config,
+                                  string? extraBuildArgs = null,
+                                  string? extraProperties = null,
+                                  bool aot = false,
+                                  bool? dotnetWasmFromRuntimePack = null,
+                                  bool hasIcudt = true)
+        {
+            if (aot)
+                extraProperties = $"{extraProperties}\n<RunAOTCompilation>true</RunAOTCompilation>\n";
+
+            InitProjectDir(_tempDir);
+
+            File.WriteAllText(Path.Combine(_tempDir, $"{projectName}.csproj"),
+@$"<Project Sdk=""Microsoft.NET.Sdk"">
+  <PropertyGroup>
+    <TargetFramework>{s_targetFramework}</TargetFramework>
+    <OutputType>Exe</OutputType>
+    <WasmGenerateRunV8Script>true</WasmGenerateRunV8Script>
+    <WasmMainJSPath>runtime-test.js</WasmMainJSPath>
+    {extraProperties ?? string.Empty}
+  </PropertyGroup>
+</Project>");
+
+            File.Copy(Path.Combine(AppContext.BaseDirectory, "runtime-test.js"), Path.Combine(_tempDir, "runtime-test.js"));
+
+            StringBuilder sb = new();
+            sb.Append("publish");
+            sb.Append(s_defaultBuildArgs);
+
+            sb.Append($" /p:Configuration={config}");
+
+            string logFilePath = Path.Combine(_logPath, $"{projectName}.binlog");
+            _testOutput.WriteLine($"Binlog path: {logFilePath}");
+            sb.Append($" /bl:\"{logFilePath}\" /v:minimal /nologo");
+            if (extraBuildArgs != null)
+                sb.Append($" {extraBuildArgs} ");
+
+            AssertBuild(sb.ToString());
+
+            string bundleDir = Path.Combine(GetBinDir(config: config), "AppBundle");
+            AssertBasicAppBundle(bundleDir, projectName, config, hasIcudt);
+
+            dotnetWasmFromRuntimePack ??= !aot;
+            AssertDotNetWasmJs(bundleDir, fromRuntimePack: dotnetWasmFromRuntimePack.Value);
+        }
+
+        private static void AssertBasicAppBundle(string bundleDir, string projectName, string config, bool hasIcudt=true)
+        {
+            AssertFilesExist(bundleDir, new []
+            {
+                "index.html",
+                "runtime.js",
+                "dotnet.timezones.blat",
+                "dotnet.wasm",
+                "mono-config.js",
+                "dotnet.js",
+                "run-v8.sh"
+            });
+
+            AssertFilesExist(bundleDir, new[] { "icudt.dat" }, expectToExist: hasIcudt);
+
+            string managedDir = Path.Combine(bundleDir, "managed");
+            AssertFilesExist(managedDir, new[] { $"{projectName}.dll" });
+
+            bool is_debug = config == "Debug";
+            if (is_debug)
+            {
+                // Use cecil to check embedded pdb?
+                // AssertFilesExist(managedDir, new[] { $"{projectName}.pdb" });
+
+                //FIXME: um.. what about these? embedded? why is linker omitting them?
+                //foreach (string file in Directory.EnumerateFiles(managedDir, "*.dll"))
+                //{
+                    //string pdb = Path.ChangeExtension(file, ".pdb");
+                    //Assert.True(File.Exists(pdb), $"Could not find {pdb} for {file}");
+                //}
+            }
+        }
+
+        private void AssertDotNetWasmJs(string bundleDir, bool fromRuntimePack)
+        {
+            string nativeDir = GetRuntimeNativeDir();
+
+            AssertFile(Path.Combine(nativeDir, "dotnet.wasm"), Path.Combine(bundleDir, "dotnet.wasm"), "Expected dotnet.wasm to be same as the runtime pack", same: fromRuntimePack);
+            AssertFile(Path.Combine(nativeDir, "dotnet.js"), Path.Combine(bundleDir, "dotnet.js"), "Expected dotnet.js to be same as the runtime pack", same: fromRuntimePack);
+        }
+
+        private static void AssertFilesDontExist(string dir, string[] filenames, string? label = null)
+            => AssertFilesExist(dir, filenames, label, expectToExist: false);
+
+        private static void AssertFilesExist(string dir, string[] filenames, string? label = null, bool expectToExist=true)
+        {
+            Assert.True(Directory.Exists(dir), $"[{label}] {dir} not found");
+            foreach (string filename in filenames)
+            {
+                string path = Path.Combine(dir, filename);
+
+                if (expectToExist)
+                {
+                    Assert.True(File.Exists(path),
+                            label != null
+                                ? $"{label}: {path} doesn't exist"
+                                : $"{path} doesn't exist");
+                }
+                else
+                {
+                    Assert.False(File.Exists(path),
+                            label != null
+                                ? $"{label}: {path} should not exist"
+                                : $"{path} should not exist");
+                }
+            }
+        }
+
+        private static void AssertSameFile(string file0, string file1, string? label=null) => AssertFile(file0, file1, label, same: true);
+        private static void AssertNotSameFile(string file0, string file1, string? label=null) => AssertFile(file0, file1, label, same: false);
+
+        private static void AssertFile(string file0, string file1, string? label=null, bool same=true)
+        {
+            Assert.True(File.Exists(file0), $"{label}: Expected to find {file0}");
+            Assert.True(File.Exists(file1), $"{label}: Expected to find {file1}");
+
+            FileInfo finfo0 = new(file0);
+            FileInfo finfo1 = new(file1);
+
+            if (same)
+                Assert.True(finfo0.Length == finfo1.Length, $"{label}: File sizes don't match for {file0} ({finfo0.Length}), and {file1} ({finfo1.Length})");
+            else
+                Assert.True(finfo0.Length != finfo1.Length, $"{label}: File sizes should not match for {file0} ({finfo0.Length}), and {file1} ({finfo1.Length})");
+        }
+
+        private void AssertBuild(string args)
+        {
+            (int exitCode, _) = RunProcess("dotnet", args, workingDir: _tempDir, label: "build");
+            Assert.True(0 == exitCode, $"Build process exited with non-zero exit code: {exitCode}");
+        }
+
+        private string GetObjDir(string targetFramework=s_targetFramework, string? baseDir=null, string config="Debug")
+            => Path.Combine(baseDir ?? _tempDir, "obj", config, targetFramework, "browser-wasm", "wasm");
+
+        private string GetBinDir(string targetFramework=s_targetFramework, string? baseDir=null, string config="Debug")
+            => Path.Combine(baseDir ?? _tempDir, "bin", config, targetFramework, "browser-wasm");
+
+        private string GetRuntimePackDir() => s_runtimePackDir;
+
+        private string GetRuntimeNativeDir()
+            => Path.Combine(GetRuntimePackDir(), "runtimes", "browser-wasm", "native");
+
+        public void Dispose()
+        {
+            if (s_skipProjectCleanup)
+                return;
+
+            try
+            {
+                Directory.Delete(_tempDir, recursive: true);
+            }
+            catch
+            {
+                Console.Error.WriteLine($"Failed to delete '{_tempDir}' during test cleanup");
+            }
+        }
+
+        private (int, string) RunProcess(string path,
+                                         string args = "",
+                                         IDictionary<string, string>? envVars = null,
+                                         string? workingDir = null,
+                                         string? label = null,
+                                         bool logToXUnit = true)
+        {
+            _testOutput.WriteLine($"Running: {path} {args}");
+            StringBuilder outputBuilder = new ();
+            var processStartInfo = new ProcessStartInfo
+            {
+                FileName = path,
+                UseShellExecute = false,
+                CreateNoWindow = true,
+                RedirectStandardError = true,
+                RedirectStandardOutput = true,
+                Arguments = args,
+            };
+
+            if (workingDir != null)
+                processStartInfo.WorkingDirectory = workingDir;
+
+            if (envVars != null)
+            {
+                if (envVars.Count > 0)
+                    _testOutput.WriteLine("Setting environment variables for execution:");
+
+                foreach (KeyValuePair<string, string> envVar in envVars)
+                {
+                    processStartInfo.EnvironmentVariables[envVar.Key] = envVar.Value;
+                    _testOutput.WriteLine($"\t{envVar.Key} = {envVar.Value}");
+                }
+            }
+
+            Process? process = Process.Start(processStartInfo);
+            if (process == null)
+                throw new ArgumentException($"Process.Start({path} {args}) returned null process");
+
+            process.ErrorDataReceived += (sender, e) => LogData("[stderr]", e.Data);
+            process.OutputDataReceived += (sender, e) => LogData("[stdout]", e.Data);
+
+            try
+            {
+                process.BeginOutputReadLine();
+                process.BeginErrorReadLine();
+                process.WaitForExit();
+
+                return (process.ExitCode, outputBuilder.ToString().Trim('\r', '\n'));
+            }
+            catch
+            {
+                Console.WriteLine(outputBuilder.ToString());
+                throw;
+            }
+
+            void LogData(string label, string? message)
+            {
+                if (logToXUnit && message != null)
+                {
+                    _testOutput.WriteLine($"{label} {message}");
+                }
+                outputBuilder.AppendLine($"{label} {message}");
+            }
+        }
+
+        private static string s_directoryBuildProps = @"<Project>
+  <PropertyGroup>
+    <_WasmTargetsDir Condition=""'$(RuntimeSrcDir)' != ''"">$(RuntimeSrcDir)\src\mono\wasm\build\</_WasmTargetsDir>
+    <_WasmTargetsDir Condition=""'$(WasmBuildSupportDir)' != ''"">$(WasmBuildSupportDir)\wasm\</_WasmTargetsDir>
+    <EMSDK_PATH Condition=""'$(WasmBuildSupportDir)' != ''"">$(WasmBuildSupportDir)\emsdk\</EMSDK_PATH>
+
+  </PropertyGroup>
+
+  <Import Project=""$(_WasmTargetsDir)WasmApp.LocalBuild.props"" Condition=""Exists('$(_WasmTargetsDir)WasmApp.LocalBuild.props')"" />
+
+  <PropertyGroup>
+    <WasmBuildAppDependsOn>PrepareForWasmBuild;$(WasmBuildAppDependsOn)</WasmBuildAppDependsOn>
+  </PropertyGroup>
+</Project>";
+
+        private static string s_directoryBuildTargets = @"<Project>
+  <Target Name=""CheckWasmLocalBuildInputs"" BeforeTargets=""Build"">
+    <Error Condition=""'$(RuntimeSrcDir)' == '' and '$(WasmBuildSupportDir)' == ''""
+           Text=""Both %24(RuntimeSrcDir) and %24(WasmBuildSupportDir) are not set. Either one of them needs to be set to use local runtime builds"" />
+
+    <Error Condition=""'$(RuntimeSrcDir)' != '' and '$(WasmBuildSupportDir)' != ''""
+           Text=""Both %24(RuntimeSrcDir) and %24(WasmBuildSupportDir) are set. "" />
+
+    <Error Condition=""!Exists('$(_WasmTargetsDir)WasmApp.LocalBuild.props')""
+           Text=""Could not find WasmApp.LocalBuild.props in $(_WasmTargetsDir)"" />
+    <Error Condition=""!Exists('$(_WasmTargetsDir)WasmApp.LocalBuild.targets')""
+           Text=""Could not find WasmApp.LocalBuild.targets in $(_WasmTargetsDir)"" />
+
+    <Warning
+      Condition=""'$(WasmMainJS)' != '' and '$(WasmGenerateAppBundle)' != 'true'""
+      Text=""%24(WasmMainJS) is set when %24(WasmGenerateAppBundle) is not true: it won't be used because an app bundle is not being generated. Possible build authoring error"" />
+  </Target>
+
+  <Target Name=""PrepareForWasmBuild"">
+    <ItemGroup>
+      <WasmAssembliesToBundle Include=""$(TargetDir)publish\*.dll"" />
+    </ItemGroup>
+  </Target>
+
+  <Import Project=""$(_WasmTargetsDir)WasmApp.LocalBuild.targets"" Condition=""Exists('$(_WasmTargetsDir)WasmApp.LocalBuild.targets')"" />
+</Project>";
+
+    }
+
+ }
index e12d882..53a064e 100644 (file)
@@ -10,6 +10,7 @@
       <DisabledProjects Include="$(TestRoot)*\**\cs_template.csproj" />
       <DisabledProjects Include="$(TestRoot)Common\**\*.*proj" />
       <DisabledProjects Include="$(TestRoot)FunctionalTests\**\*.csproj" /> <!-- They need to be isolated from the existing setup -->
+      <DisabledProjects Include="$(TestRoot)BuildWasmApps\**\*.csproj" /> <!-- built and run with libraries -->
       <DisabledProjects Include="$(TestRoot)GC\Performance\Framework\GCPerfTestFramework.csproj" />
       <DisabledProjects Include="$(TestRoot)Loader\classloader\generics\regressions\DD117522\Test.csproj" />
       <DisabledProjects Include="$(TestRoot)Loader\classloader\generics\GenericMethods\VSW491668.csproj" /> <!-- issue 5501 -->