Fix the Windows private runtime build testing problems (#807)
authorMike McLaughlin <mikem@microsoft.com>
Sat, 8 Feb 2020 20:03:08 +0000 (12:03 -0800)
committerGitHub <noreply@github.com>
Sat, 8 Feb 2020 20:03:08 +0000 (12:03 -0800)
Fix the Windows private runtime build testing problems

Enable the "setclrpath" for Windows.

Change the SOS test harness and scripts to set the runtime path for triage dumps.

Add -privatebuildpath option to build scripts that installs the test runtimes, copies
the private runtime build and on Windows regedit's the necessary keys to create dumps.

Add private build registry cleanup.

18 files changed:
README.md
documentation/privatebuildtesting.md [new file with mode: 0644]
documentation/sos.md [new file with mode: 0644]
eng/CleanupPrivateBuild.csproj [new file with mode: 0644]
eng/InstallRuntimes.proj
eng/build.ps1
eng/build.sh
src/SOS/SOS.UnitTests/ConfigFiles/Windows/Debugger.Tests.Config.txt
src/SOS/SOS.UnitTests/SOSRunner.cs
src/SOS/SOS.UnitTests/Scripts/DualRuntimes.script
src/SOS/Strike/runtime.cpp
src/SOS/Strike/runtime.h
src/SOS/Strike/sos.def
src/SOS/Strike/sosdocs.txt
src/SOS/Strike/sosdocsunix.txt
src/SOS/Strike/strike.cpp
src/SOS/lldbplugin/setclrpathcommand.cpp
src/Tools/dotnet-dump/Commands/SOSCommand.cs

index 980c1b649a238b512452f3773b79d3f16b34faea..066162d2f77141aa7294b9edf3b3e316ac9c4b8d 100644 (file)
--- a/README.md
+++ b/README.md
@@ -27,62 +27,22 @@ To install the platform's prerequisites and build:
  * [MacOS Instructions](documentation/building/osx-instructions.md)
  * [FreeBSD Instructions](documentation/building/freebsd-instructions.md)
  * [NetBSD Instructions](documentation/building/netbsd-instructions.md)
+ * [Testing on private runtime builds](documentation/privatebuildtesting.md)
 
-## Getting lldb
-
-Getting a version of lldb that works for your platform can be a problem sometimes. The version has to be at least 3.9 or greater because of a bug running SOS on a core dump that was fixed. Some Linux distros like Ubuntu it is easy as `sudo apt-get install lldb-3.9 python-lldb-3.9`. On other distros, you will need to build lldb. The directions below should give you some guidance.
-
-* [Linux Instructions](documentation/lldb/linux-instructions.md)
-* [MacOS Instructions](documentation/lldb/osx-instructions.md)
-* [FreeBSD Instructions](documentation/lldb/freebsd-instructions.md)
-* [NetBSD Instructions](documentation/lldb/netbsd-instructions.md)
-
-## Installing SOS
-
-* [Linux and MacOS Instructions](documentation/installing-sos-instructions.md)
-* [Windows Instructions](documentation/installing-sos-windows-instructions.md)
-
-## Using SOS
-
-* [SOS debugging for Linux/MacOS](documentation/sos-debugging-extension.md)
-* [SOS debugging for Windows](documentation/sos-debugging-extension-windows.md)
-* [Debugging a core dump](documentation/debugging-coredump.md)
-
-## Tools
+## SOS and Other Diagnostic Tools
 
+* [SOS](documentation/sos.md) - About the SOS debugger extension.
 * [dotnet-dump](documentation/dotnet-dump-instructions.md) - Dump collection and analysis utility.
 * [dotnet-trace](documentation/dotnet-trace-instructions.md) - Enable the collection of events for a running .NET Core Application to a local trace file.
 * [dotnet-counters](documentation/dotnet-counters-instructions.md) - Monitor performance counters of a .NET Core application in real time. 
 
-## New Features
-
-The `bpmd` command can now be used before the runtime is loaded. You can load SOS or the sos plugin on Linux and execute bpmd. Always add the module extension for the first parameter.
-
-    bpmd SymbolTestApp.dll SymbolTestApp.Program.Main
-
-You can set a source file/line number breakpoint like this (the fully qualified source file path is usually not necessary):
-
-    bpmd SymbolTestApp.cs:24
-
-Symbol server support - The `setsymbolserver` command enables downloading the symbol files (portable PDBs) for managed assemblies during commands like `clrstack`, etc. See `soshelp setsymbolserver` for more details.
-
-    (lldb) setsymbolserver -ms
-
-Before executing the "bt" command to dump native frames to load the native symbols (for live debugging only):
-
-    (lldb) loadsymbols
-
-To add a local directory to search for symbols:
-
-    (lldb) setsymbolserver -directory /tmp/symbols
-
 ## Useful Links
 
 * [FAQ](documentation/FAQ.md) - Frequently asked questions.
 * [The LLDB Debugger](http://lldb.llvm.org/index.html) - More information about lldb.
 * [SOS](https://msdn.microsoft.com/en-us/library/bb190764(v=vs.110).aspx) - More information about SOS.
-* [Debugging CoreCLR](https://github.com/dotnet/coreclr/blob/master/Documentation/building/debugging-instructions.md) - Instructions for debugging .NET Core and the CoreCLR runtime.
-* [dotnet/coreclr](https://github.com/dotnet/coreclr) - Source for the .NET Core runtime.
+* [Debugging CoreCLR](https://github.com/dotnet/runtime/blob/master/docs/workflow/debugging/coreclr/debugging.md) - Instructions for debugging .NET Core and the CoreCLR runtime.
+* [dotnet/runtime](https://github.com/dotnet/runtime) - Source for the .NET Core runtime.
 * [Official Build Instructions](documentation/building/official-build-instructions.md) - Internal official build instructions.
 
 [//]: # (Begin current test results)
diff --git a/documentation/privatebuildtesting.md b/documentation/privatebuildtesting.md
new file mode 100644 (file)
index 0000000..4a389c4
--- /dev/null
@@ -0,0 +1,39 @@
+Private runtime build testing
+=============================
+
+Here are some instructions on how to run the diagnostics repo's tests against a locally build private .NET Core runtime. These directions will work on Windows, Linux and MacOS. 
+
+1. Build the runtime repo (see [Workflow Guide](https://github.com/dotnet/runtime/blob/master/docs/workflow/README.md)).
+2. Build the diagnostics repo (see [Building the Repository](../README.md)).
+3. Run the diagnostics repo tests with the -privatebuildpath option.
+
+On Windows:
+```
+C:\diagnostics> test -privatebuildpath c:\runtime\artifacts\bin\coreclr\Windows_NT.x64.Debug
+```
+
+When you are all done with the private runtime testing, run this command to remove the Windows registry entries added in the above steps.
+
+```
+C:\diagnostics> test -cleanupprivatebuild
+```
+
+There will be some popups from regedit asking for administrator permission to edit the registry (press Yes), warning about adding registry keys from AddPrivateTesting.reg (press Yes) and that the edit was successful (press OK).
+
+On Linux/MacOS:
+```
+~/diagnostics$ ./test.sh --privatebuildpath /home/user/runtime/artifacts/bin/coreclr/Linux.x64.Debug
+```
+The private runtime will be copied to the diagnostics repo and the tests started. It can be just the coreclr binaries but this assumes that the private build is close to the latest published master build. If not, you can pass the runtime's testhost directory containing all the shared runtime bits i.e. `c:\runtime\artifacts\bin\testhost\netcoreapp5.0-Windows_NT-Debug-x64\shared\Microsoft.NETCore.App\5.0.0` or `/home/user/runtime/artifacts/bin/testhost/netcoreapp5.0-Linux-Release-x64/shared/Microsoft.NETCore.App/5.0.0`
+
+On Linux/MacOS it is recommended to test against Release runtime builds because of a benign assert in DAC (tracked by issue #[31897](https://github.com/dotnet/runtime/issues/31897)) that causes the tests to fail.
+
+On Windows the DAC is not properly signed for a private runtime build so there are a couple of registry keys that need to be added so Windows will load the DAC and the tests can create proper mini-dumps. An example of the registry key values added are:
+
+```
+[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\KnownManagedDebuggingDlls]
+"C:\diagnostics\.dotnet\shared\Microsoft.NETCore.App\5.0.0-alpha.1.20102.3\mscordaccore.dll"=dword:0
+
+[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\MiniDumpAuxiliaryDlls]
+"C:\diagnostics\.dotnet\shared\Microsoft.NETCore.App\5.0.0-alpha.1.20102.3\coreclr.dll"="C:\diagnostics\.dotnet\shared\Microsoft.NETCore.App\5.0.0-alpha.1.20102.3\mscordaccore.dll"
+```
\ No newline at end of file
diff --git a/documentation/sos.md b/documentation/sos.md
new file mode 100644 (file)
index 0000000..fe4518d
--- /dev/null
@@ -0,0 +1,46 @@
+SOS 
+===
+
+SOS is a debugger extension that allows a developer to inspect the managed state of a .NET Core and desktop runtime process. SOS can be loaded by WinDbg/cdb debuggers on Windows and lldb on Linux and MacOS.
+
+## Getting lldb
+
+Getting a version of lldb that works for your platform can be a problem sometimes. The version has to be at least 3.9 or greater because of a bug running SOS on a core dump that was fixed. Some Linux distros like Ubuntu it is easy as `sudo apt-get install lldb-3.9 python-lldb-3.9`. On other distros, you will need to build lldb. The directions below should give you some guidance.
+
+* [Linux Instructions](lldb/linux-instructions.md)
+* [MacOS Instructions](lldb/osx-instructions.md)
+* [FreeBSD Instructions](lldb/freebsd-instructions.md)
+* [NetBSD Instructions](lldb/netbsd-instructions.md)
+
+## Installing SOS
+
+* [Linux and MacOS Instructions](installing-sos-instructions.md)
+* [Windows Instructions](installing-sos-windows-instructions.md)
+
+## Using SOS
+
+* [SOS debugging for Linux/MacOS](sos-debugging-extension.md)
+* [SOS debugging for Windows](sos-debugging-extension-windows.md)
+* [Debugging a core dump](debugging-coredump.md)
+
+## New SOS Features
+
+The `bpmd` command can now be used before the runtime is loaded. You can load SOS or the sos plugin on Linux and execute bpmd. Always add the module extension for the first parameter.
+
+    bpmd SymbolTestApp.dll SymbolTestApp.Program.Main
+
+You can set a source file/line number breakpoint like this (the fully qualified source file path is usually not necessary):
+
+    bpmd SymbolTestApp.cs:24
+
+Symbol server support - The `setsymbolserver` command enables downloading the symbol files (portable PDBs) for managed assemblies during commands like `clrstack`, etc. See `soshelp setsymbolserver` for more details.
+
+    (lldb) setsymbolserver -ms
+
+Before executing the "bt" command to dump native frames to load the native symbols (for live debugging only):
+
+    (lldb) loadsymbols
+
+To add a local directory to search for symbols:
+
+    (lldb) setsymbolserver -directory /tmp/symbols
diff --git a/eng/CleanupPrivateBuild.csproj b/eng/CleanupPrivateBuild.csproj
new file mode 100644 (file)
index 0000000..83118ce
--- /dev/null
@@ -0,0 +1,8 @@
+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <TargetFramework>netcoreapp2.0</TargetFramework>
+  </PropertyGroup>
+  
+  <Import Project="$(RepositoryEngineeringDir)\InstallRuntimes.proj" />
+</Project>
+
index 0e86ebdb38363aef24579ad9026f941f47fdbeee..4b4b6255ff74e3dc2bf3d3137692910c776d3e25 100644 (file)
@@ -1,7 +1,9 @@
+<!-- All Rights Reserved. Licensed to the .NET Foundation under one or more agreements. The .NET Foundation licenses this file to you under the MIT license. See the LICENSE file in the project root for more information. -->
 <Project>
   <!--
      $(BuildArch) - architecture to test (x64, x86, arm, arm64). Defaults to x64.
      $(DailyTest) - if true, only install/test the latest (master branch) runtime
+     $(PrivateBuildPath) - if non-empty, path to private runtime build to copy/test
      
      From Versions.props:
 
     <DailyTest Condition="'$(DailyTest)' == ''">false</DailyTest>
     <BuildArch Condition="'$(BuildArch)' == ''">$(Platform)</BuildArch>
     <BuildArch Condition="'$(BuildArch)' == ''">x64</BuildArch>
+  </PropertyGroup>
+
+  <PropertyGroup Condition="'$(BuildArch)' != 'x86'">
+    <DotNetInstallRoot>$(DotNetRoot)</DotNetInstallRoot>
+    <RegistryRoot>HKEY_LOCAL_MACHINE\SOFTWARE</RegistryRoot>
+  </PropertyGroup>
+
+  <PropertyGroup Condition="'$(BuildArch)' == 'x86'">
+    <DotNetInstallRoot>$(DotNetRoot)x86\</DotNetInstallRoot>
+    <RegistryRoot>HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node</RegistryRoot>
+  </PropertyGroup>
+
+  <PropertyGroup>
     <CommonInstallArgs>-architecture $(BuildArch)</CommonInstallArgs>
-    <TestConfigFileName Condition="'$(BuildArch)' != 'x86'">$(DotNetRoot)Debugger.Tests.Versions.txt</TestConfigFileName>
-    <TestConfigFileName Condition="'$(BuildArch)' == 'x86'">$(DotNetRoot)x86\Debugger.Tests.Versions.txt</TestConfigFileName>
+    <DotNetInstallDir>$(DotNetInstallRoot.Replace("\", "\\"))shared\\Microsoft.NETCore.App\\$(MicrosoftNETCoreAppVersion)\\</DotNetInstallDir>
+    <TestConfigFileName>$(DotNetInstallRoot)Debugger.Tests.Versions.txt</TestConfigFileName>
+    <AddRegeditFileName>$(DotNetInstallRoot)AddPrivateTesting.reg</AddRegeditFileName>
+    <RemoveRegeditFileName>$(DotNetInstallRoot)RemovePrivateTesting.reg</RemoveRegeditFileName>
+    <RegeditCommand>regedit.exe</RegeditCommand>
   </PropertyGroup>
 
   <Choose>
     <TestVersions Include="Latest" RuntimeVersion="$(MicrosoftNETCoreAppVersion)" AspNetVersion="$(MicrosoftAspNetCoreAppRefVersion)" Install="true" />
     <TestVersions Include="31" RuntimeVersion="$(MicrosoftNETCoreApp31Version)" AspNetVersion="$(MicrosoftAspNetCoreApp31Version)" Install="!$(DailyTest)" />
     <TestVersions Include="30" RuntimeVersion="$(MicrosoftNETCoreApp30Version)" AspNetVersion="$(MicrosoftAspNetCoreApp30Version)" Install="!$(DailyTest)" />
-    <TestVersions Include="21" RuntimeVersion="$(MicrosoftNETCoreApp21Version)" AspNetVersion="$(MicrosoftAspNetCoreApp21Version)" Install="!$(DailyTest)" />
+    <TestVersions Include="21" RuntimeVersion="$(MicrosoftNETCoreApp21Version)" AspNetVersion="$(MicrosoftAspNetCoreApp21Version)" Install="true" />
   </ItemGroup>
 
+<!--
+    Installs the runtimes for the SOS tests, handles private runtime build support or cleans up the private build registry keys
+-->
+
   <Target Name="InstallTestRuntimes" 
           BeforeTargets="RunTests"
-          DependsOnTargets="InstallRuntimesWindows;InstallRuntimesUnix;WriteTestVersionManifest" />
+          DependsOnTargets="InstallRuntimesWindows;InstallRuntimesUnix;WriteTestVersionManifest;TestPrivateBuild" />
+
+<!--
+    Installs the test runtimes on Windows
+-->
 
   <Target Name="InstallRuntimesWindows"
           Condition="$([MSBuild]::IsOsPlatform(Windows))"
           Condition="%(TestVersions.Install)" />
   </Target>
 
+<!--
+    Installs the test runtimes on Linux/MacOS
+-->
+
   <Target Name="InstallRuntimesUnix"
           Condition="!$([MSBuild]::IsOsPlatform(Windows))"
           Inputs="$(VersionsPropsPath)" Outputs="$(TestConfigFileName)">
           Condition="%(TestVersions.Install)" />
    </Target>
 
-  <Target Name="WriteTestVersionManifest" Inputs="$(VersionsPropsPath)" Outputs="$(TestConfigFileName)">
+<!--
+    Writes the Debugger.Tests.Versions.txt file used by the SOS test harness
+-->
+
+  <Target Name="WriteTestVersionManifest" 
+          Inputs="$(VersionsPropsPath)" 
+          Outputs="$(TestConfigFileName)">
+
     <PropertyGroup>
       <TestConfigFileLines>
 <![CDATA[
   <AspNetCoreVersionLatest>$(MicrosoftAspNetCoreAppRefVersion)</AspNetCoreVersionLatest>
 </Configuration>
 ]]>
-     </TestConfigFileLines>
+      </TestConfigFileLines>
     </PropertyGroup>
 
     <WriteLinesToFile File="$(TestConfigFileName)" Lines="$(TestConfigFileLines)" Overwrite="true" WriteOnlyWhenDifferent="true" />
+
     <Message Importance="High" Text="Created config file $(TestConfigFileName)" />
 
     <ItemGroup>
       <FileWrites Include="$(TestConfigFileName)" />
     </ItemGroup>
   </Target>
+
+<!--
+    Copies the private runtime build binaries and on Windows adds registry keys
+-->
+
+  <Target Name="TestPrivateBuild"
+          Condition="'$(PrivateBuildPath)' != ''"
+          DependsOnTargets="ModifyRegistry">
+
+    <ItemGroup>
+      <PrivateBuildFiles Include="$(PrivateBuildPath)\*" />
+    </ItemGroup>
+
+    <Message Importance="High" Text="Copying private build binaries from $(PrivateBuildPath) to $(DotNetInstallDir.Replace('\\', '\'))" />
+
+    <Copy SourceFiles="@(PrivateBuildFiles)" DestinationFolder="$(DotNetInstallDir.Replace('\\', '\'))" />
+  </Target>
+
+<!--
+    Removes the private build registry keys
+-->
+
+  <Target Name="CleanupPrivateBuild"
+          Condition="Exists($(RemoveRegeditFileName))">
+
+    <Exec Command="$(RegeditCommand) $(RemoveRegeditFileName)" />
+    <!-- 
+        Delete only the AddRegeditFileName so the target ModifyRegistry will run on next 
+        build. Leaving the remove reg key file so this target can be run multiple times.
+    -->
+    <Delete Files="$(AddRegeditFileName)" />
+  </Target>
+
+<!--
+    On Windows adds the registry keys to allow the unsigned private build DAC to generate dumps
+-->
+
+  <Target Name="ModifyRegistry"
+          Condition="$([MSBuild]::IsOsPlatform(Windows))"
+          DependsOnTargets="CreateRemoveRegFile"
+          Inputs="$(VersionsPropsPath)" 
+          Outputs="$(AddRegeditFileName)">
+
+    <PropertyGroup>
+      <AddRegeditFileLines>
+<![CDATA[
+Windows Registry Editor Version 5.00
+
+[$(RegistryRoot)\Microsoft\Windows NT\CurrentVersion\KnownManagedDebuggingDlls]
+"$(DotNetInstallDir)mscordaccore.dll"=dword:0
+
+[$(RegistryRoot)\Microsoft\Windows NT\CurrentVersion\MiniDumpAuxiliaryDlls]
+"$(DotNetInstallDir)coreclr.dll"="$(DotNetInstallDir)mscordaccore.dll"
+]]>
+      </AddRegeditFileLines>
+    </PropertyGroup>
+
+    <WriteLinesToFile File="$(AddRegeditFileName)" Lines="$(AddRegeditFileLines)" Overwrite="true" WriteOnlyWhenDifferent="true" />
+
+    <ItemGroup>
+      <FileWrites Include="$(AddRegeditFileName)" />
+    </ItemGroup>
+
+    <Exec Command="$(RegeditCommand) $(AddRegeditFileName)" />
+  </Target>
+
+<!--
+    Creates the reg file to remove the registry keys added in ModifyRegistry
+-->
+
+  <Target Name="CreateRemoveRegFile">
+    <PropertyGroup>
+      <RemoveRegeditFileLines>
+<![CDATA[
+Windows Registry Editor Version 5.00
+
+[$(RegistryRoot)\Microsoft\Windows NT\CurrentVersion\KnownManagedDebuggingDlls]
+"$(DotNetInstallDir)mscordaccore.dll"=-
+
+[$(RegistryRoot)\Microsoft\Windows NT\CurrentVersion\MiniDumpAuxiliaryDlls]
+"$(DotNetInstallDir)coreclr.dll"=-
+]]>
+      </RemoveRegeditFileLines>
+    </PropertyGroup>
+
+    <WriteLinesToFile File="$(RemoveRegeditFileName)" Lines="$(RemoveRegeditFileLines)" Overwrite="true" WriteOnlyWhenDifferent="true" />
+
+    <ItemGroup>
+      <FileWrites Include="$(RemoveRegeditFileName)" />
+    </ItemGroup>
+  </Target>
+
 </Project>
index 8f32731eba9cb78aa0fbeb201f1f93312d772319..820720139bfacc95c5e9f92a54eafd0ef120d4cf 100644 (file)
@@ -8,6 +8,8 @@ Param(
   [switch] $skipmanaged,
   [switch] $skipnative,
   [switch] $dailytest,
+  [string] $privatebuildpath = "",
+  [switch] $cleanupprivatebuild,
   [Parameter(ValueFromRemainingArguments=$true)][String[]] $remainingargs
 )
 
@@ -45,11 +47,22 @@ $engroot = Join-Path $reporoot "eng"
 $artifactsdir = Join-Path $reporoot "artifacts"
 $logdir = Join-Path $artifactsdir "log"
 $logdir = Join-Path $logdir Windows_NT.$architecture.$configuration
+$dailytestproperty = "false"
 
 if ($ci) {
     $remainingargs = "-ci " + $remainingargs
 }
 
+if ($dailytest -or $privatebuildpath -ne "") {
+    $dailytestproperty = "true"
+}
+
+# Remove the private build registry keys
+if ($cleanupprivatebuild) {
+    Invoke-Expression "& `"$engroot\common\msbuild.ps1`" $engroot\CleanupPrivateBuild.csproj /v:$verbosity /t:CleanupPrivateBuild /p:BuildArch=$architecture /p:TestArchitectures=$architecture"
+    exit $lastExitCode
+}
+
 # Install sdk for building, restore and build managed components.
 if (-not $skipmanaged) {
     Invoke-Expression "& `"$engroot\common\build.ps1`" -build -binaryLog -configuration $configuration -verbosity $verbosity /p:BuildArch=$architecture /p:TestArchitectures=$architecture $remainingargs"
@@ -69,7 +82,7 @@ if (-not $skipnative) {
 # Run the xunit tests
 if ($test -or $dailytest) {
     if (-not $crossbuild) {
-        & "$engroot\common\build.ps1" -test -configuration $configuration -verbosity $verbosity -ci:$ci /bl:$logdir\Test.binlog /p:BuildArch=$architecture /p:TestArchitectures=$architecture /p:DailyTest=$dailyTest
+        & "$engroot\common\build.ps1" -test -configuration $configuration -verbosity $verbosity -ci:$ci /bl:$logdir\Test.binlog /p:BuildArch=$architecture /p:TestArchitectures=$architecture /p:DailyTest=$dailytestproperty /p:PrivateBuildPath=$privatebuildpath
         if ($lastExitCode -ne 0) {
             exit $lastExitCode
         }
index 7a928e4366fa97c9dfa53e4f36676f9bc5700486..bf621c97916b3cefaed662696f3af32812b9839a 100755 (executable)
@@ -30,6 +30,7 @@ __NativeBuild=true
 __CrossBuild=false
 __Test=false
 __DailyTest=false
+__PrivateBuildPath=""
 __CI=false
 __Verbosity=minimal
 __ManagedBuildArgs=
@@ -43,6 +44,7 @@ usage()
     echo "--skipnative - Skip building native components"
     echo "--test - run xunit tests"
     echo "--dailytest - test components for daily build job"
+    echo "--privatebuildpath - path to local private runtime build to test"
     echo "--architecture <x64|x86|arm|armel|arm64>"
     echo "--configuration <debug|release>"
     echo "--rootfs <ROOTFS_DIR>"
@@ -177,6 +179,12 @@ while :; do
             __DailyTest=true
             ;;
 
+        -privatebuildpath)
+            __PrivateBuildPath=$2
+            __DailyTest=true
+            shift
+            ;;
+
         -ci)
             __CI=true
             __ManagedBuildArgs="$__ManagedBuildArgs $1"
@@ -498,7 +506,7 @@ if [ $__Test == true ]; then
 
       echo "lldb: '$LLDB_PATH' gdb: '$GDB_PATH'"
 
-      "$__ProjectRoot/eng/common/build.sh" --test --configuration "$__BuildType" --verbosity "$__Verbosity" /bl:$__LogDir/Test.binlog /p:BuildArch=$__BuildArch /p:DailyTest=$__DailyTest $__TestArgs
+      "$__ProjectRoot/eng/common/build.sh" --test --configuration "$__BuildType" --verbosity "$__Verbosity" /bl:$__LogDir/Test.binlog /p:BuildArch=$__BuildArch /p:DailyTest=$__DailyTest /p:PrivateBuildPath=$__PrivateBuildPath $__TestArgs
       if [ $? != 0 ]; then
           exit 1
       fi
index ef06736e0c8b579e3dfbf518a998cbeb9d51fbff..3ee2b5ededcf4d8ddf2f7b631e5c22326ff5cc2a 100644 (file)
@@ -22,6 +22,9 @@
   <DumpDir>$(RootBinDir)\tmp\$(TargetConfiguration)\dumps</DumpDir>
   <CDBPath>$(RootBinDir)\bin\SOS.UnitTests\$(TargetConfiguration)\netcoreapp2.0\publish\runtimes\win-$(TargetArchitecture)\native\cdb.exe</CDBPath>
   <CDBHelperExtension>$(InstallDir)\runcommand.dll</CDBHelperExtension>
+
+  <DesktopFrameworkPath Condition="$(TargetArchitecture) == x64">$(WinDir)\Microsoft.Net\Framework64\v4.0.30319\</DesktopFrameworkPath>
+  <DesktopFrameworkPath Condition="$(TargetArchitecture) != x64">$(WinDir)\Microsoft.Net\Framework\v4.0.30319\</DesktopFrameworkPath>
   <DesktopFramework>net462</DesktopFramework>
 
   <DebuggeeSourceRoot>$(RepoRootDir)\src\SOS\SOS.UnitTests\Debuggees</DebuggeeSourceRoot>
       <BuildProjectFramework>$(DesktopFramework)</BuildProjectFramework>
       <BuildProjectRuntime>win-$(TargetArchitecture)</BuildProjectRuntime>
       <DebugType>full</DebugType>
-      <FrameworkDirPath Condition="$(TargetArchitecture) == x64">$(WinDir)\Microsoft.Net\Framework64\v4.0.30319\</FrameworkDirPath>
-      <FrameworkDirPath Condition="$(TargetArchitecture) != x64">$(WinDir)\Microsoft.Net\Framework\v4.0.30319\</FrameworkDirPath>
-      <RuntimeSymbolsPath>$(FrameworkDirPath)</RuntimeSymbolsPath>
+      <RuntimeSymbolsPath>$(DesktopFrameworkPath)</RuntimeSymbolsPath>
       <SOSHostRuntime>$(DotNetRoot)\shared\Microsoft.NETCore.App\$(RuntimeVersion21)</SOSHostRuntime>
     </Option>
   </Options>
index 0c84c29966c5c8e1a7a5537547a7955a9b8d4bf3..fd63d8d95be7101492e17ad64bb5c76d5657a55a 100644 (file)
@@ -440,11 +440,14 @@ public class SOSRunner : IDisposable
                     initialCommands.Add(".sympath %DEBUG_ROOT%");
                     initialCommands.Add(".extpath " + Path.GetDirectoryName(config.SOSPath()));
 
-                    // Add the path to runtime so cdb/sos can find mscordbi.
-                    string runtimeSymbolsPath = config.RuntimeSymbolsPath;
-                    if (runtimeSymbolsPath != null)
+                    // Add the path to runtime so cdb/SOS can find DAC/DBI for triage dumps
+                    if (information.DumpType == DumpType.Triage)
                     {
-                        initialCommands.Add(".sympath+ " + runtimeSymbolsPath);
+                        string runtimeSymbolsPath = config.RuntimeSymbolsPath;
+                        if (runtimeSymbolsPath != null)
+                        {
+                            initialCommands.Add(".sympath+ " + runtimeSymbolsPath);
+                        }
                     }
                     // Turn off warnings that can happen in the middle of a command's output
                     initialCommands.Add(".outmask- 0x244");
@@ -543,6 +546,15 @@ public class SOSRunner : IDisposable
                     }
                     initialCommands.Add("setsymbolserver -directory %DEBUG_ROOT%");
 
+                    // Add the path to runtime so dotnet-dump/SOS can find DAC/DBI for triage dumps
+                    if (information.DumpType == DumpType.Triage)
+                    {
+                        string runtimeSymbolsPath = config.RuntimeSymbolsPath;
+                        if (runtimeSymbolsPath != null)
+                        {
+                            initialCommands.Add("setclrpath " + runtimeSymbolsPath);
+                        }
+                    }
                     arguments.Append(debuggerPath);
                     arguments.Append(@" analyze %DUMP_NAME%");
                     debuggerPath = config.DotNetDumpHost();
@@ -1175,6 +1187,7 @@ public class SOSRunner : IDisposable
         }
         vars.Add("%DEBUG_ROOT%", debuggeeConfig.BinaryDirPath);
         vars.Add("%SOS_PATH%", information.TestConfiguration.SOSPath());
+        vars.Add("%DESKTOP_RUNTIME_PATH%", information.TestConfiguration.DesktopRuntimePath());
 
         // Can be used in an RegEx expression
         vars.Add("<DEBUGGEE_EXE>", debuggeeExe.Replace(@"\", @"\\"));
@@ -1385,6 +1398,11 @@ public static class TestConfigurationExtensions
         return TestConfiguration.MakeCanonicalPath(config.GetValue("SOSHostRuntime"));
     }
 
+    public static string DesktopRuntimePath(this TestConfiguration config)
+    {
+        return TestConfiguration.MakeCanonicalPath(config.GetValue("DesktopRuntimePath"));
+    }
+
     public static bool GenerateDumpWithLLDB(this TestConfiguration config)
     {
         return config.GetValue("GenerateDumpWithLLDB")?.ToLowerInvariant() == "true";
index e54fa09c84b4bcc764e81b8d42871ff6ec9f30d0..954d5fea3d86aa8fb3204a4dd1ac5ea14c398d77 100644 (file)
@@ -71,6 +71,10 @@ VERIFY:\s*Total\s+<DECVAL>\s+objects\s+
 SOSCOMMAND:SOSStatus
 VERIFY:.*\.NET Core runtime at.*\s+
 
+IFDEF:TRIAGE_DUMP
+SOSCOMMAND:setclrpath %DESKTOP_RUNTIME_PATH%
+ENDIF:TRIAGE_DUMP
+
 SOSCOMMAND:SOSStatus -desktop
 VERIFY:\s*Switched to desktop CLR runtime successfully\s+
 
@@ -83,12 +87,7 @@ VERIFY:\s+<HEXVAL>\s+<HEXVAL>\s+SymbolTestDll\.TestClass\.Foo6\(.*\)\s+\[(?i:.*[
 VERIFY:\s+<HEXVAL>\s+<HEXVAL>\s+SymbolTestDll\.TestClass\.Foo5\(.*\)\s+\[(?i:.*[\\|/]TestClass\.cs) @ 16\]\s*
 VERIFY:\s+<HEXVAL>\s+<HEXVAL>\s+SymbolTestDll\.TestClass\.ThrowException\(.*\)\s+\[(?i:.*[\\|/]TestClass\.cs) @ 10\]\s*
 
-IFDEF:DOTNETDUMP
 SOSCOMMAND:clrthreads
-ENDIF:DOTNETDUMP
-!IFDEF:DOTNETDUMP
-SOSCOMMAND:Threads
-ENDIF:DOTNETDUMP
 VERIFY:\s*ThreadCount:\s+<DECVAL>\s+
 VERIFY:\s+UnstartedThread:\s+<DECVAL>\s+
 VERIFY:\s+BackgroundThread:\s+<DECVAL>\s+
index 76505769bcfa38a0deb0877fb8f8db53e966a621..94afc1fd0ab69a48108a014965ae615b94642bc3 100644 (file)
@@ -37,6 +37,9 @@ bool Runtime::s_isDesktop = false;
 LPCSTR Runtime::s_dacFilePath = nullptr;
 LPCSTR Runtime::s_dbiFilePath = nullptr;
 
+// The runtime module path set by the "setclrpath" command
+LPCSTR g_runtimeModulePath = nullptr;
+
 // Current runtime instance
 IRuntime* g_pRuntime = nullptr;
 
@@ -267,10 +270,17 @@ LPCSTR Runtime::GetRuntimeDirectory()
 {
     if (m_runtimeDirectory == nullptr)
     {
-        std::string runtimeDirectory;
-        if (SUCCEEDED(GetRuntimeDirectory(runtimeDirectory)))
+        if (g_runtimeModulePath != nullptr)
+        {
+            m_runtimeDirectory = _strdup(g_runtimeModulePath);
+        }
+        else 
         {
-            m_runtimeDirectory = _strdup(runtimeDirectory.c_str());
+            std::string runtimeDirectory;
+            if (SUCCEEDED(GetRuntimeDirectory(runtimeDirectory)))
+            {
+                m_runtimeDirectory = _strdup(runtimeDirectory.c_str());
+            }
         }
     }
     return m_runtimeDirectory;
index eef4893527a3f051a1c0c5fd819f852d407476f5..1693201982110b0188200f9cec7d58e0bbc7b0a5 100644 (file)
@@ -88,6 +88,7 @@ public:
     virtual void DisplayStatus() = 0;
 };
 
+extern LPCSTR g_runtimeModulePath;
 extern IRuntime* g_pRuntime;
 
 /**********************************************************************\
index 55e1003800a6a6fd695ffffa43b46aef966a7c71..ec85ef31c3e552ca3d487c39fe5598f00d1e7f47 100644 (file)
@@ -128,6 +128,8 @@ EXPORTS
     sethostruntime=SetHostRuntime
     SetSymbolServer
     setsymbolserver=SetSymbolServer
+    SetClrPath
+    setclrpath=SetClrPath
     SOSFlush
     sosflush=SOSFlush
     StopOnException
index 212ad00e350c3ecf694842d0bcd4e8b7c45277cb..f38283f3bbd6328483f80b57bb097d0a633ce189 100644 (file)
@@ -62,9 +62,10 @@ Examining the GC history           Other
 -----------------------------      -----------------------------
 HistInit                           SetHostRuntime (sethostruntime)
 HistRoot                           SetSymbolServer (setsymbolserver)
-HistObj                            FAQ
+HistObj                            SetClrPath (setclrpath)
 HistObjFind                        SOSFlush
 HistClear                          SOSStatus (sosstatus)
+                                   FAQ
                                    Help (soshelp)
 \\
 
@@ -2638,10 +2639,21 @@ COMMAND: sosstatus.
 Display internal SOS status, reset the internal cached state, or change between desktop or 
 netcore runtimes when both are loaded in the process or dump.
 
-0:000> !sosstatus
-Target platform: 8664 Context size 04d0
-.NET Core runtime at 00007FFEE7230000 (005c2000)
-DAC file path: C:\Users\mikem\AppData\Local\Temp\SymbolCache\mscordaccore.dll/5d0707425c2000/mscordaccore.dll
-DBI file path: C:\Users\mikem\AppData\Local\Temp\SymbolCache\mscordbi.dll/5d0707425c2000/mscordbi.dll
-Cache: C:\Users\mikem\AppData\Local\Temp\SymbolCache
-Server: http://msdl.microsoft.com/download/symbols/
\ No newline at end of file
+    0:000> !sosstatus
+    Target platform: 8664 Context size 04d0
+    .NET Core runtime at 00007FFEE7230000 (005c2000)
+    DAC file path: C:\Users\mikem\AppData\Local\Temp\SymbolCache\mscordaccore.dll/5d0707425c2000/mscordaccore.dll
+    DBI file path: C:\Users\mikem\AppData\Local\Temp\SymbolCache\mscordbi.dll/5d0707425c2000/mscordbi.dll
+    Cache: C:\Users\mikem\AppData\Local\Temp\SymbolCache
+    Server: http://msdl.microsoft.com/download/symbols/
+//
+
+COMMAND: setclrpath.
+!setclrpath <path-to-runtime>
+
+Sets the path to load the runtime DAC/DBI files. This command may be necessary for dumps that were created
+on another machine or for triage dumps because there are no module paths included in the dump to infer the
+runtime directory.
+
+    0:000> !setclrpath "C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.0"
+//
\ No newline at end of file
index 196a45be269934e849a9ac96f24edf87a028e98d..2ead3614d1b53f2b22058fcac49372549c3c9c8d 100644 (file)
@@ -55,26 +55,14 @@ Examining the GC history           Other
 -----------------------------      -----------------------------
 HistInit (histinit)                SetHostRuntime (sethostruntime)
 HistRoot (histroot)                SetSymbolServer (setsymbolserver, loadsymbols)
-HistObj  (histobj)                 FAQ
+HistObj  (histobj)                 SetClrPath (setclrpath)
 HistObjFind (histobjfind)          SOSFlush (sosflush)
 HistClear (histclear)              SOSStatus (sosstatus)
+                                   FAQ
                                    Help (soshelp)
 \\
 
 COMMAND: faq.
->> Where can I get the right version of SOS for my build?
-
-If you are running a xplat version of coreclr, the sos module (exact name
-is platform dependent) is installed in the same directory as the main coreclr
-module. There is also an lldb sos plugin command that allows the path where
-the sos, dac and dbi modules are loaded:
-
-    "setclrpath /home/user/coreclr/bin/Product/Linux.x64.Debug""
-
-If you are using a dump file created on another machine, it is a little bit
-more complex. You need to make sure the dac module that came with that install
-is in the directory set with the above command.
-
 >> I have a chicken and egg problem. I want to use SOS commands, but the CLR
    isn't loaded yet. What can I do?
 
@@ -1963,3 +1951,13 @@ SOSStatus [-reset]
 -reset - reset all the cached internal SOS state.
 
 Display internal SOS status or reset the internal cached state.
+\\
+
+COMMAND: setclrpath.
+!setclrpath <path-to-runtime>
+
+Sets the path to load the runtime DAC/DBI files. This command may be necessary for dumps that were created
+on another machine or for triage dumps.
+
+    (lldb) setclrpath /home/user/coreclr/bin/Product/Linux.x64.Debug
+\\
\ No newline at end of file
index 48eda6df2ccd71c2cdc7722b96757a04864e4417..8e18d42c25ac3464c252cef472fcc2f5c5d8b21d 100644 (file)
@@ -10470,6 +10470,9 @@ DECLARE_API(SOSStatus)
     if (g_targetMachine != nullptr) {
         ExtOut("Target platform: %04x Context size %04x\n", g_targetMachine->GetPlatform(), g_targetMachine->GetContextSize());
     }
+    if (g_runtimeModulePath != nullptr) {
+        ExtOut("Runtime module path: %s\n", g_runtimeModulePath);
+    }
     if (g_pRuntime != nullptr) {
         g_pRuntime->DisplayStatus();
     }
@@ -16078,6 +16081,38 @@ DECLARE_API(SetSymbolServer)
     return Status;
 }
 
+//
+// Sets the runtime module path
+//
+DECLARE_API(SetClrPath)
+{
+    INIT_API_EXT();
+
+    StringHolder runtimeModulePath;
+    CMDValue arg[] =
+    {
+        {&runtimeModulePath.data, COSTRING},
+    };
+    size_t narg;
+    if (!GetCMDOption(args, nullptr, 0, arg, _countof(arg), &narg))
+    {
+        return E_FAIL;
+    }
+    if (narg > 0)
+    {
+        if (g_runtimeModulePath != nullptr)
+        {
+            free((void*)g_runtimeModulePath);
+        }
+        g_runtimeModulePath = _strdup(runtimeModulePath.data);
+    }
+    if (g_runtimeModulePath != nullptr)
+    {
+        ExtOut("Runtime module path: %s\n", g_runtimeModulePath);
+    }
+    return S_OK;
+}
+
 void PrintHelp (__in_z LPCSTR pszCmdName)
 {
     static LPSTR pText = NULL;
index 0a964473ff7d6731b4d428b543f1d7e88c574916..cbe4773e8ae2a9f81723d762c04e0d634f24a0e8 100644 (file)
@@ -48,6 +48,6 @@ bool
 setclrpathCommandInitialize(lldb::SBDebugger debugger)
 {
     lldb::SBCommandInterpreter interpreter = debugger.GetCommandInterpreter();
-    lldb::SBCommand command = interpreter.AddCommand("setclrpath", new setclrpathCommand(), "Set the path to load coreclr dac/dbi files. setclrpath <path>");
+    lldb::SBCommand command = interpreter.AddCommand("setclrpath", new setclrpathCommand(), "Set the path to load the runtime DAC/DBI files. setclrpath <path>");
     return true;
 }
index 639693eb5dd4cc93f84d4f2bc364052fe7e2cbc5..2bb9c8d60ad54815eb75b8a3b9f510cecf21e9c3 100644 (file)
@@ -48,6 +48,7 @@ namespace Microsoft.Diagnostics.Tools.Dump
     [Command(Name = "histobjfind",      AliasExpansion = "HistObjFind",         Help = "Displays all the log entries that reference an object at the specified address.")]
     [Command(Name = "histroot",         AliasExpansion = "HistRoot",            Help = "Displays information related to both promotions and relocations of the specified root.")]
     [Command(Name = "setsymbolserver",  AliasExpansion = "SetSymbolServer",     Help = "Enables the symbol server support ")]
+    [Command(Name = "setclrpath",       AliasExpansion = "setclrpath",          Platform = CommandPlatform.Windows, Help = "Set the path to load the runtime DAC/DBI files.")]
     [Command(Name = "dumprcw",          AliasExpansion = "DumpRCW",             Platform = CommandPlatform.Windows, Help = "Displays information about a Runtime Callable Wrapper.")]
     [Command(Name = "dumpccw",          AliasExpansion = "DumpCCW",             Platform = CommandPlatform.Windows, Help = "Displays information about a COM Callable Wrapper.")]
     [Command(Name = "dumppermissionset",AliasExpansion = "DumpPermissionSet",   Platform = CommandPlatform.Windows, Help = "Displays a PermissionSet object (debug build only).")]