Nikola Milosavljevic [Fri, 2 Oct 2020 19:44:03 +0000 (12:44 -0700)]
Update THIRD-PARTY-NOTICES file (#41851)
* Introduce THIRD-PARTY-NOTICES file for deployment with shared host package
* Update of THIRD-PARTY-NOTICES for .NET 5.0 release
* Update src/installer/pkg/THIRD-PARTY-NOTICES.TXT
Co-authored-by: Adeel Mujahid <adeelbm@outlook.com>
* Update libunwind name in root TPN file to prevent regression in future updates
* Update libunwind license
* Updates based on review comments
Co-authored-by: Adeel Mujahid <adeelbm@outlook.com>
Brian Sullivan [Fri, 2 Oct 2020 19:13:01 +0000 (12:13 -0700)]
Fixes dotnet/runtime issue #12446 ILDASM - constants printed with the wrong endian (#42848)
When the Disassembler prints 4-byte and 8-byte values it should print them in little endian format
Maryam Ariyan [Fri, 2 Oct 2020 18:47:28 +0000 (11:47 -0700)]
Moves ConfigurationRootTest to common (#42788)
- used by two test projects
Aleksey Kliger (λgeek) [Fri, 2 Oct 2020 18:42:36 +0000 (14:42 -0400)]
[metadata] Getting the element size of array of fnptr types is ok (#42920)
Fixes https://github.com/dotnet/runtime/issues/42917
Tomáš Rylek [Fri, 2 Oct 2020 16:46:04 +0000 (18:46 +0200)]
Cleanly split BinDir to RuntimeBinDir and TestBinDir (#42829)
Jo Shields [Fri, 2 Oct 2020 14:14:12 +0000 (10:14 -0400)]
Add a few useful links, and move doc as requested (#42965)
Marek Safar [Fri, 2 Oct 2020 13:55:02 +0000 (15:55 +0200)]
Use constants instead of instance call for http handlers predefined values (#42941)
Sergey Andreenko [Fri, 2 Oct 2020 09:09:28 +0000 (02:09 -0700)]
Add `Crossgen2JitPath` to R2R test. (#42939)
Layomi Akinrinade [Fri, 2 Oct 2020 03:58:42 +0000 (23:58 -0400)]
Address JSON test failures on Mono + Windows (#42960)
Mateo Torres-Ruiz [Fri, 2 Oct 2020 00:44:36 +0000 (17:44 -0700)]
Add single-file app's dir to NATIVE_DLL_SEARCH_DIRECTORIES (#42876)
Add two directories to NATIVE_DLL_SEARCH_DIRECTORIES to single-file bundles:
1. The bundle exe directory
2. If the bundle extracts any files, the extraction directory
Fixes #42772
Nathan Ricci [Thu, 1 Oct 2020 21:55:29 +0000 (17:55 -0400)]
Disabled GC.TotalBytesAllocated test. (#42921)
Kenneth Pouncey [Thu, 1 Oct 2020 21:44:56 +0000 (23:44 +0200)]
[wasm][crypto] RandomNumberGenerator mapped to Web Crypto getRandomValues (#42728)
* [wasm][crypto] RandomNumberGenerator mapped to Web Crypto getRandomValues
- Uses Web Crypto API [`getRandomValues`](https://www.w3.org/TR/WebCryptoAPI/#Crypto-method-getRandomValues) if available.
- Falls back to `/dev/urandom` as default if the crypto library is missing.
* Remove the emscripten interface code from the driver.c.
* remove extraneous code comment
* Move emscripten definition around.
* Address review comment
* Add javascript bridge implementation library to Native source tree.
- Javascript checks for crypto interface and uses `crypto.getRandomValues`
- Add api bridge call when building for emscripten browser.
- separate out into browser subdirectory
- If we couldn't find a proper implementation, as Math.random() is not suitable we will abort.
```
ABORT: no cryptographic support found getRandomValues. Consider polyfilling it if you want to use something insecure like Math.random(), e.g. put this in a --pre-js: var crypto = { getRandomValues: function(array) { for (var i = 0; i < array.length; i++) array[i] = (Math.random()*256)|0 } };
```
* Change tests to set random values of the buffer instead of return a single value.
* Remove old test code
* Remove testing code
* Incorporate the PAL bridge layer into the `--js-library` build process
* Address review comments about directory structure and naming
* Update src/mono/wasm/runtime-test.js
Co-authored-by: Ryan Lucia <ryan@luciaonline.net>
* Add note about insecure code for testing purposes
* Formatting
* Return -1 if crypto does not exist instead of aborting from js. This allows the managed code exception flow to continue as normal.
Co-authored-by: Ryan Lucia <ryan@luciaonline.net>
Santiago Fernandez Madero [Thu, 1 Oct 2020 21:35:25 +0000 (16:35 -0500)]
Add server core testing (#41701)
* Add server core testing to Windows x64
* Add server core testing
* Use 20h1 queue instead
* Remove runtime testing until we get more capacity on 20H1 queue
* Use 2004 image
* Skip tests on Server core
* PR Feedback and fix build
* Disable more server core tests
* Disable more tests on Server Core
* Attempt fixing failing process test in server core
address feedback
* Fix PlatformDetection.IsInContainer for windows and feedback from Eirik's change
* PR Feedback
* Don't run server core on PRs yet until we get more capacity
Co-authored-by: Eirik Tsarpalis <eirik.tsarpalis@gmail.com>
Stephen Toub [Thu, 1 Oct 2020 19:30:32 +0000 (15:30 -0400)]
Some style clean-up in HttpClient.cs (#42956)
David Mason [Thu, 1 Oct 2020 19:06:14 +0000 (12:06 -0700)]
Fix race condition in GetAppdomainStaticAddress test (#42437)
Layomi Akinrinade [Thu, 1 Oct 2020 18:38:46 +0000 (14:38 -0400)]
Add log for ILLinkSharedFramework target running in libraries build (#42910)
* Add log for ILLinkSharedFramework target running in libraries build
* Address feedback -express that the operation is ongoing
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>
Bruce Forstall [Thu, 1 Oct 2020 18:19:11 +0000 (11:19 -0700)]
Create a SuperPMI collection pipeline (#42053)
* Create a SuperPMI collection pipeline
Start with a clone of jitstress.yml, and go from there.
* Only build Windows_NT_x64, disable test builds and runs
* TEMPORARY: disable normal PR triggered jobs
* some template files for superpmi
* Little trial error# 1
* fix the superpmi-setup.ps1
* Disable CoreClrTestBuildHost, fix the project path in superpmi-send-to-helix
* Skip build, download, create core_root
* Just copy superpmi scripts, import pipeline-logging-functions
* add a dummy step
* add missing space before message
* Replace message with script
* Temporarily disable dependsOn
* disable another dependsOn
* Pass superpmi.proj file
* logging around send-to-helix
* fix superpmi.proj to include ItemGroup
* have placeholder.txt in WorkItem folder
* fix workitem folder creation, script name
* Re-enable build coreclr, libraries, download, core_root
* fix superpmi.py path
* clone jitutils, build and copy pmi.dll
* Pass Core_Root as payload
* Use hardcoded checked
* Use .dotnet from runtime repo
- Build jitutils before and send the payload to helix
* Fix the pmi.dll location and add -output_mch_path
* fix the output path
* Try publishing SourcesDirectory
* Upload to HELIX_WORKITEM_UPLOAD_ROOT
* Include test in collection and partition core_root and test
* Enable CoreClrTestBuildHost
* Add dependsOn P1 test
* Add DownloadFilesFromResults
* Properly copy the Tests
* Workaround for https://github.com/dotnet/arcade/pull/6179
* Disable test run, SuperPMI just system.web.dll
* Fix the mch path
* Comment partition for test
* Remove commented code
* Increase timeout to 12 hours
* fast pass
- Comment build coreclr/libraries job
- Instead just download coreclr from some previous run, and tell that it is core_root
* Update the machine pool
* Add a dummy step
* Use SuperPMI_Min.zip that contains all the required files
* Another way of Extracting cab
* download in powershell, extract files separately
* fix the zip path
* Another logic of download zip
* fix path
* fix newdir path
* more fixes
* fix extract step
* add test python script and modify creator
* Map secret to env
* decrease timeout, fix superpmi path in proj, print unzipped contents
* Add a step to download and dir artifacts
* Attempt to call super_pmi upload
* list all contents
* Also dir SourcesDir\artifacts
* Use DownloadPipelineArtifact
* Use the right key
* Upload from helixresults
* cleanup run-superpmi-job.yml
* Re-enable superpmi for CORE_ROOT
* fix the dependsOn
* Replace checked with $(BuildConfig)
* Replace checked with $(BuildConfig)
Comment download artifacts
* change helix queue for windows and add for arm64
* Use FileSeparateChar in superpmi.proj
* Add Linux x64
* Fix the helix queue, property erroro
* logging in shell
* Fix PayloadDir and CorrelationPayloadDir
* Make timeout to 5 hours
* Print workitem contents
* fix WorkItemDirectory
* Just copy pmi.dll
* fix the pmi_assemblies path
* disable reporter
* pop put JitUtilsDirectory before deleting it
* chmod for superpmi-setup.sh
* just copy pmi.dll for unix
* fix the parameter to unix
* Add logging to understand -pmi_assemblies issue
* Add "binaries" folder where payload is present
fix unit script that initializes array
* remove precommand
* Pass Core_Root as CorrelationPayload
- Pass Core_Root as CorrelationPayload
- Pass tag to name .mch file with
- Changes to .ps1 and .sh to copy Core_Root directly
* Add back the workaround for https://github.com/dotnet/arcade/pull/6179
Also, rename 'checked' -> 'Checked'
* fix checked for Linux
* mkdir for linux
* fix path in HelixResultsDestinationDir
* Enable Windows x86/arm64
* fix the python path
* First version of superpmi-setup.py
* Attempt to use superpmi-setup.py
* Fix spelling of LibrariesArtifacts, set python path
* TestArtifacts commented
* superpmi-setup script for linux
* Enable Linux_x64
* Added pydocs
* Add option for exclude_directories
* Moved all the logic to superpmi-setup.py
* Replace superpmi-setup.ps1 and superpmi-setup.sh with superpmi-setup.py
* Fix up in superpmi-setup.py
* fix display name
* add missing spacE
* Use differnet conditional
* ignore PermissionError while cleaning up temp folder
* fix workitem copy for linux
* fix copy for unix
* Publish logs
* Print LibrariesArtifacts
* remove space in front of publish logs desc
* Add missing quotes for LibrariesArtifacts
* Download x64 coreclr to make upload work
* Enable Windows x86/arm64, Linux arm/arm64
* Delete superpmi-setup.ps1, superpmi-setup.sh
* Fix space around LibrariesArtifacts
* Download real coreclr x64
* ignore native dlls
* Make superpmi.py Python3.6 friendly
* Minor cleanup
* Cleanup superpmi-send-to-helix to use helix-inner-step
* Remove redundant steps
* Merge mch files
* Pass osGroup
* Make superpmi-setup script takes tests_directory
* Enable SPMI collection for P1
* General cleanup
* fix merge_mch pattern for OS
* Log the SPMI collection output and put it in upload_root folder
* Replace robocopy/rsync with shutil.copy2
Note that shutil.copy2 is very slow specially on Linux as compared to rsync.
But I think we are ok to have slowness in return of simpler code.
* fix path of pmi.dll to be copied
* Include linux .so and file_exe files in copy_directories
* Try publishing just .mch file
* Revert "Try publishing just .mch file"
This reverts commit
a5312c05f9502c7c2f2c5a28125f75f8d824b869.
* Disable tests SPMI collection on 42053
- Tests run times out on Windows
- Some of the partition's collection failed for tests because reply was not clean
- Here are the details when libraries/tests were run:
https://dev.azure.com/dnceng/internal/_build/results?buildId=832814&view=results
- Hence, disable tests run and just have libraries run for now.
- I didn't remove the test run part but commented the portion of tests collection with
TODO intentionally so when I have to enable it, I will know what to enable.
* Review feedback
Co-authored-by: Kunal Pathak <Kunal.Pathak@microsoft.com>
Tom Deseyn [Thu, 1 Oct 2020 16:31:40 +0000 (18:31 +0200)]
Use same code to DetectCiphersuiteConfiguration for portable and non-portable builds
Kevin Jones [Thu, 1 Oct 2020 14:58:25 +0000 (10:58 -0400)]
Update cross-platform-cryptography.md for CFB mode.
Jan Kotas [Thu, 1 Oct 2020 14:21:30 +0000 (07:21 -0700)]
Add Environment.ProcessPath (#42768)
Fixes #40862
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Ryan Lucia <ryan@luciaonline.net>
Stephen Toub [Thu, 1 Oct 2020 11:45:12 +0000 (07:45 -0400)]
Fix HttpClient.CancelAllPending/Timeout handling for GetString/ByteArrayAsync (#42346)
* Fix HttpClient.CancelAllPending/Timeout handling for GetString/ByteArrayAsync
When GetStringAsync and GetByteArrayAsync are reading the response body, they're only paying attention to the provided CancellationToken; they're not paying attention to the HttpClient's CancelAllPending or Timeout.
* Address PR feedback
Stephen Toub [Thu, 1 Oct 2020 10:42:58 +0000 (06:42 -0400)]
Add test to validate HttpResponseMessage.RequestMessage (#42844)
Marek Safar [Thu, 1 Oct 2020 09:55:57 +0000 (11:55 +0200)]
[System.Private.CoreLib] Methods with no instance access can be made static (#42805)
Tomas Weinfurt [Thu, 1 Oct 2020 05:57:04 +0000 (22:57 -0700)]
make CancelConnectAsync thread safe (#42935)
* make CancelConnectAsync thread safe
* replace var with MultipleConnectAsync
hev [Thu, 1 Oct 2020 05:28:58 +0000 (13:28 +0800)]
Fix build errors on GCC. (#42834)
* Fix build errors on GCC.
* Enable PGO on Clang.
* Fix build errors on GCC 10 and later.
Omair Majid [Thu, 1 Oct 2020 05:28:11 +0000 (01:28 -0400)]
Add an option to keep native debug symbols (#39203)
When packaging .NET for Linux distributions, the package builders
generally use a different workflow for shipping symbols to users:
1. The package maintainer builds code with the debug flags (such as
`-g`) to generate full native debug info and symbols.
2. Nothing is stripped from build by the package maintainer.
3. The build system (`rpmbuild`, `debuild`) removes the debug
info (or debug symbols) from the code and creates separate
`-debuginfo` or `-debug` packages that contain just the debug
symbols.
4. These debug packages are then distributed along with the normal
packages using the normal Linux distribution mechanisms. This lets
users install the exact set of debug symbols matching their other
package.
To support this workflow in dotnet/runtime, we need to add optional
support for not stripping debug symbols. I used it has follows:
CFLAGS=-g CXXFLAGS=-g ./build.sh --keepnativesymbols true
After this build, the built binaries include all debug symbols.
I can then rely on the distro package build system to identify, strip,
package and ship the debug info/symbols separately.
See https://github.com/dotnet/runtime/issues/3781 and
https://github.com/dotnet/source-build/issues/267 for more details on
the background and motivation.
For some related fixes, see:
- https://github.com/dotnet/coreclr/pull/3445
- https://github.com/dotnet/corefx/pull/24979
Aleksey Kliger (λgeek) [Thu, 1 Oct 2020 02:35:45 +0000 (22:35 -0400)]
[mono] Fix HelloWorld sample for OSX (#42916)
OSX `uname` doesn't have a `-o`
Sergey Andreenko [Thu, 1 Oct 2020 02:19:11 +0000 (19:19 -0700)]
Jit preparation for arm64 apple: Use bytes in `fgArgTabEntry`. (#42503)
* fix a dumping error introduced in my recent ref PR.
* add a pinvoke test with many small stack arguments.
It will fail on arm64 apple.
* Add get/set for lclVar::lvStkOffs.
* Change some types from `size_t` to `unsigned` when working with stack offsets.
* Rename `stackSize` to `GetStackSize`.
* Do not run stack level setter for arm64 apple.
* Use bytes in `fgArgTabEntry`.
* fix comments/rename vars.
* Add en explanation about `GetStackSlotsNumber`.
* Rename `DEBUG_NOT_OSX_ARM64` to `DEBUG_ARG_SLOTS`.
monojenkins [Thu, 1 Oct 2020 02:07:49 +0000 (22:07 -0400)]
[metadata] Use MONO_PROFILER_API on MonoClass getters in checked builds (#42819)
In normal builds, the getters are `static inline` functions, so the profiler doesn't reference the symbols - it just accesses the fields by offset.
In checked builds with `--enable-checked-build=private_types`, the getters are not inlined, so the profiler shared libraries need the symbols to be visible.
Co-authored-by: lambdageek <lambdageek@users.noreply.github.com>
Andrew Arnott [Thu, 1 Oct 2020 01:55:28 +0000 (19:55 -0600)]
Fix error in code comment (#42931)
The error led to the comment expressing the opposite of its otherwise obvious intent.
Eric Erhardt [Thu, 1 Oct 2020 01:49:09 +0000 (20:49 -0500)]
Update DataContractSerialization ILLink.Suppressions.xml (#42930)
Changes #40691 and #42824 conflicted. One added a new ILLink suppress warnings file, while the other added more warnings. This causes the build to break.
The fix is to regenerate the suppressions file with the latest code.
Fix #42926
Stephen Toub [Thu, 1 Oct 2020 00:58:04 +0000 (20:58 -0400)]
Fix nullable annotation on ISite.Container (#42918)
Mateo Torres-Ruiz [Wed, 30 Sep 2020 23:54:10 +0000 (16:54 -0700)]
Extract bundled files when IncludeAllContentForSelfExtract is set (#42435)
* Extract bundled files when IncludeAllContentForSelfExtract is set
* Don't pass bundle probe to the runtime for netcoreapp3 compat mode
* Bundle probe should not report extracted files
This also means we can still pass bundle probe to the runtime even in full-extract case as all files will be marked as extracted in that situation (so the probe should return false all the time).
This might be benefitial as the runtime will still "know" it's single-file (we don't really use that knowledge today though).
Improve the test to actually validate locally built binaries and add validation of TPA and framework assembly loading.
* CodeBaseThrows should fail
* Normalize app base path
* Search for CoreLib in TPA
* PR feedback
* Fix bundle tests to actually test single-file bundles
The tests were basically running everything using apphost.exe - the reason the tests worked is basically a weird coincidence (the bundle was always self-contained so it had everything managed in it and all the native dlls were beside the .exe) - so from host's point of view this looked like a self-contained app - and then hostpolicy/hostfxr noticed it's also a bundle and treated it as such.
This change introduces new test baseclass to reduce duplication and standardizes the tests to mostly use self-contained publish then bundled with singlefilehost.exe. Currently they do leave hostpolicy.dll and hostfxr.dll next to the exe (they're written in .deps.json so they must exist on disk) but at runtime they won't be used since the singlefilehost.exe has them bundled in itself.
* Don't use Microsoft.NETCore.App.Internal package
From 3.0+, Microsoft.NETCore.App just chains in the appropriate runtime
packs. The .Internal package was necessary for tests to use when we were
building with a older SDK, but targeting 3.0+. We should be able to get
rid of the workaround now.
This actually causes problems with the full extraction single-file mode
because pre-5.0 and in the .Internal package, System.Private.CoreLib.dll
was not listed as a runtime assembly (only under native). As a result,
it would not end up in the TPA for a single-file bundle.
* AppWithSubDirs: add back RuntimeIdentifier, don't use .Internal package
The test assets are all restored - passing in TestTargetRid - before
the test runs. Since we do a rid-specific publish later, it needs to
be restored with the approriate rid.
Stop using .Internal package. This resulted in host binaries getting
published even for framework-dependent publishes.
* Rework the single-file API tests for better coverage
The tests now validate behavior of all of the interesting APIs in both single-file and backcompat mode.
Added the AppContext.BaseDirectory to the tests.
* Fix the reported path to .deps.json for back compat mode
The path must point to the extraction folder and not use the "fake" in-bundle path.
* Fix a test bug on Linux
Actually we should be using path separator char in the APP_CONTEXT_DEPS_FILES, but that's something we won't be able to change really (too much chance of breaking something without too much value).
Co-authored-by: vitek-karas <vitek.karas@microsoft.com>
Co-authored-by: Elinor Fung <elfung@microsoft.com>
Andy Ayers [Wed, 30 Sep 2020 23:16:45 +0000 (16:16 -0700)]
JIT: show profile data for dot flowgraph dumps (#42657)
Also, tweak layout a bit, and update so that if a method is jitted
multiple times the files don't overwrite one another. Instead a
suffix is added to indicate the kind of jit codegen.
Tomas Weinfurt [Wed, 30 Sep 2020 22:37:23 +0000 (15:37 -0700)]
do not call RemoteCertificateValidationCallback on the same certificate again (#42836)
* do not call RemoteCertificateValidationCallback on the same certificate agin
* feedback from review
* feedback from review
* make sure we do not use Linq
Eric Erhardt [Wed, 30 Sep 2020 20:29:12 +0000 (15:29 -0500)]
DataContractSerialization doesn't work with TrimMode - link (#42824)
DataContractSerialization has some Reflection "shim" methods that the ILLinker can't see through. This causes critical methods to be trimmed and applications to fail. These methods were put in place in .NET Core 1.0 when the full Reflection API wasn't available.
The fix is to remove these "shim" Reflection APIs and use Reflection directly.
Fix #41525
Fix #42754
Lakshan Fernando [Wed, 30 Sep 2020 19:24:35 +0000 (12:24 -0700)]
Test fixes for casing (#42879)
Fixed #42334
Layomi Akinrinade [Wed, 30 Sep 2020 17:46:24 +0000 (13:46 -0400)]
Suppress linker analysis warnings on GetImmutableDictionaryCreateRangeMethod (#42905)
Alexander Köplinger [Wed, 30 Sep 2020 17:07:08 +0000 (19:07 +0200)]
Fix xml doc comment for SimpleConsoleFormatterOptions.SingleLine (#42896)
Setting it to `true` forces the single line, not `false` :)
Egor Bogatov [Wed, 30 Sep 2020 15:09:46 +0000 (18:09 +0300)]
Optimize Type.IsPrimitive (#42891)
Jan Vorlicek [Wed, 30 Sep 2020 14:36:25 +0000 (16:36 +0200)]
Fix test framework crossgening (#42874)
The recent change that replaced direct invocation of crossgen and
crossgen2 for crossgening framework assemblies for coreclr tests by
r2rtest tool usage has ommited the --release and --large-bubble
arguments. These correspond to the former -O and --inputbubble crossgen2
commands and should be present.
This also fixes failures in running tests with the crossgened runtime.
Many tests were failing with the error `Could not execute the method
because either the method itself or the containing type is not fully
instantiated.`
So while there is a problem to investigate related to debug flavor of
crossgening, this PR fixes the options.
Layomi Akinrinade [Wed, 30 Sep 2020 14:06:25 +0000 (10:06 -0400)]
Add granular suppressions for linker warnings (#40691)
* Add granular suppressions for linker warnings
* Add suppressions for mono SPC
* Misc additions
* Add suppressions for Debug config
* Fix Android xml file
* Include System.ComponentModel.Annotations suppressions
* Add suppressions for System.Security.Cryptography.X509Certificates on Linux
* Fix TargetOs exists check
* Add IL2077 to System.Security.Cryptography.X509Certificates on Android
* Generalize some System.Security.Cryptography.X509Certificates suppressions to 'NonWindows'
* Remove ref to catch-all System.Security.Cryptography.X509Certificates file
* Misc clean up and some feedback
* Use ILLinkDirectory in CreateRuntimeRootILLinkDescriptorFile.targets
* Update paths for directories
* Set TargetPath as part of include
* Delete bin-placing TODO
* Include more xml files during per-library run
* Include more xml files during per-library run, pt 2
* Include more xml files during per-library run, pt 3
* Try including System.ComponentModel.Annotations based on BuildingNETCoreAppVertical
* Switch to BuildAllConfigurations
* Include xml extension in file paths
* Remove version number from assembly suppression files
* Delete ILLink.Suppressions.xml
* Try removing IL2035 nowarn
* Remove version from Microsoft.VisualBasic.Core assemly name
Viktor Hofer [Wed, 30 Sep 2020 12:21:53 +0000 (14:21 +0200)]
Flow rid in yml jobs (#37652)
Egor Bogatov [Wed, 30 Sep 2020 10:53:40 +0000 (13:53 +0300)]
[mono] Retarget HelloWorld sample to in-tree runtime pack (it used to use ".dotnet-mono" hack) (#42851)
Egor Bogatov [Wed, 30 Sep 2020 09:44:55 +0000 (12:44 +0300)]
fix android sample (#42859)
Tomas Weinfurt [Wed, 30 Sep 2020 04:55:37 +0000 (21:55 -0700)]
make sure we process PhysicalAddress on Linux (#42878)
Layomi Akinrinade [Wed, 30 Sep 2020 03:42:36 +0000 (23:42 -0400)]
Fix InvalidCastException when deserializing some dictionary types (#42835)
Egor Bogatov [Tue, 29 Sep 2020 23:35:44 +0000 (02:35 +0300)]
[RyuJIT] Add "rorx" instruction (BMI2) and emit it instead of "rol" when possible (#41772)
* Use rorx instead of rol when possible
Taylor Southwick [Tue, 29 Sep 2020 20:59:26 +0000 (13:59 -0700)]
Check for null in OptionsFactory (#41047)
If an IValidateOptions<TOptions> returns null, then an error occurs that can be difficult to track down. This change checks for null before checking if it failed.
Ryan Lucia [Tue, 29 Sep 2020 20:27:01 +0000 (16:27 -0400)]
Add additional ALC assertions and test (#42765)
Jo Shields [Tue, 29 Sep 2020 19:34:39 +0000 (15:34 -0400)]
Add Wasm testing in Chromium, not just command-line `v8` (#40786)
This should give us a better idea of any cases where browser and JS engine behaviour diverge.
* Uses WebSockets to write pipe console messages to the test runner.
* Skip System.Net.Http FunctionalTests depending on X509 certificates, tests discovery fails on `System.Net.Http.Functional.Tests.HttpClientEKUTest..cctor()` which ends up throwing PNSE
* Added environment variable to help platform detect when we're running in the browser vs v8
* System.Net.Http LoopbackServer: fix NullReferenceException because we were trying to dispose a Socket instance that threw PNSE in the constructor
Co-authored-by: Ankit Jain <radical@gmail.com>
Co-authored-by: Steve Pfister <steve.pfister@microsoft.com>
Co-authored-by: Larry Ewing <lewing@microsoft.com>
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
Ryan Lucia [Tue, 29 Sep 2020 19:27:15 +0000 (15:27 -0400)]
Fix DefaultContext test on wasm (#42764)
monojenkins [Tue, 29 Sep 2020 18:29:09 +0000 (14:29 -0400)]
[runtime] sync parameter name in tramp-s390x.c (#42804)
s390x uses 'method' as parameter name in mono_arch_get_unbox_trampoline()
in mono/mini/tramp-s390x.c, rename to 'm' to be in sync with other arches.
Fixes: https://github.com/mono/mono/issues/20437
Co-authored-by: sharkcz <sharkcz@users.noreply.github.com>
campersau [Tue, 29 Sep 2020 18:00:40 +0000 (20:00 +0200)]
[browser][http] Set HttpResponseMessage.RequestMessage (#42822)
Egor Bogatov [Tue, 29 Sep 2020 16:00:16 +0000 (19:00 +0300)]
[Android] Make AndroidAppBuilder "dotnet build" friendly (#42597)
Kevin Jones [Tue, 29 Sep 2020 15:35:03 +0000 (11:35 -0400)]
Replace VerifyWithRevocation test with DynamicRevocationTests
* Test offline revocation checking as a DynamicRevocationTest, shutting down the responder between the cache populate and the offline build.
* Remove flaky VerifyWithRevocation test, as it no longer provides unique value.
Sergio Pedri [Tue, 29 Sep 2020 14:33:31 +0000 (16:33 +0200)]
Enable Unsafe.As NotNullIfNotNull on .NET Std. 2.0 (#42827)
Tomáš Rylek [Tue, 29 Sep 2020 13:57:45 +0000 (15:57 +0200)]
Physically remove the folder src/coreclr/tests (#42828)
This change removes the last few files from src/coreclr/tests
under src/tests and finally depletes the folder. One change of note
is rename of src/coreclr/tests/runtest.py to src/tests/run.py
in accordance with the previous renames.
I have put the runparallel.sh script to src/tests as it seems
to have general usability, I moved the rest of the scripts to
src/tests/Common/scripts. I haven't actually tried to make it
work. I managed to catch one reference from arm32_ci_script /
arm_ci_test.sh to bringup_runtest.sh that I fixed but again
without actually testing them.
I am not too familiar with these scripts, I haven't found
references to them anywhere in the infra, some of them seem
related to the JIT team, adding Bruce to the PR to confirm.
I have no idea whether they are used at all, any insights into
what obsolete scripts we can get rid of is more than welcome.
Thanks
Tomas
Avani Gupta [Tue, 29 Sep 2020 11:30:55 +0000 (04:30 -0700)]
Make ContentType(string) constructor throw FormatException instead of IndexOutOfRangeException (#39529)
Jan Vorlicek [Tue, 29 Sep 2020 10:31:44 +0000 (12:31 +0200)]
Fix bad configure tests (#42756)
There was an extra -c in the CMAKE_REQUIRED_FLAGS set for testing
HAVE_UNW_GET_ACCESSORS and HAVE_UNW_GET_SAVE_LOC that was breaking
build of coreclr under homebrew. The option was somehow making
these checks behave on ARM Linux, eveb though it is not clear to
me why, as it was just causing this option to be passed to the
compiler twice at different positions of the command line of
the cmake tests.
This change fixes it by using check_symbol_exists instead of
check_c_source_compiles, since just removing the duplicite -c
was resulting in the check failing on ARM / ARM64 Linux due
to a missing symbol from libunwind during linking.
Kunal Pathak [Tue, 29 Sep 2020 06:49:06 +0000 (23:49 -0700)]
Rename references of protononjit to clrjit_ (#42133)
* wip: rename protononjit
* Rename protononjit to clrjit_
* revert changes in run-pmi-diffs.py
* revert superpmi.py changes
* fix variable name
dotnet-maestro[bot] [Tue, 29 Sep 2020 01:23:14 +0000 (01:23 +0000)]
[master] Update dependencies from Microsoft/vstest dotnet/runtime-assets dotnet/llvm-project dotnet/xharness dotnet/icu dotnet/arcade mono/linker (#42619)
[master] Update dependencies from Microsoft/vstest dotnet/runtime-assets dotnet/llvm-project dotnet/xharness dotnet/icu dotnet/arcade mono/linker
Thays Grazia [Mon, 28 Sep 2020 23:35:28 +0000 (20:35 -0300)]
The test was already working on master. (#42818)
Adeel Mujahid [Mon, 28 Sep 2020 20:51:02 +0000 (23:51 +0300)]
Add a switch to compile dotnet.js as ES6 module (#41964)
* Add a switch to compile dotnet.js as ES6 module
With `./build.sh -os Browser /p:WasmEnableES6=True`, we can now compile
`dotnet.js` as an ES6 module, which can be easily be loaded in
environments like Deno. e.g.:
```js
// myApp.js
import jsdom from "https://dev.jspm.io/jsdom";
window.document = new jsdom.JSDOM(null, {url:'http://localhost'}).window.document;
const dotnetModule = await import('./artifacts//bin/native/net5.0-Browser-Debug-wasm/dotnet.js');
// this is needed since we cannot pass scriptDirectory to upstream module (yet: https://github.com/emscripten-core/emscripten/pull/12120)
const bytes = Deno.readFileSync('./artifacts//bin/native/net5.0-Browser-Debug-wasm/dotnet.wasm', 'utf8');
const dotnet = await dotnetModule.default({wasmBinary: bytes}); // currently it calls run() on load
// dotnet.run(); // we can _additionally_ invoke run() on demand
```
Also waiting on Deno PR to merge with utf-16le suppot, workaround is:
```sh
$ ./build.sh -os Browser /p:WasmEnableES6=True
$ sed -i '' 's/utf-16le/utf8/' artifacts/bin/native/net5.0-Browser-Debug-wasm/dotnet.js
```
and then
```sh
$ deno run --allow-read myApp.js
```
* Set default value to False explicitly
* Use lower-case false for check
Co-authored-by: Alexander Köplinger <alex.koeplinger@outlook.com>
Bruce Forstall [Mon, 28 Sep 2020 20:39:55 +0000 (13:39 -0700)]
Handle general SuperPMI fatal errors better (#42821)
Before, we weren't printing out an error message for fatal
errors, and the summary message indicated there were no problems.
rizi [Mon, 28 Sep 2020 17:54:42 +0000 (19:54 +0200)]
TraceSourceLogger now takes exception into account(adds it to the log… (#42571)
* TraceSourceLogger now takes exception into account(adds it to the log message) even if formatter is not null
* Fixed a logical error, that would have added the exception twice to the log message when the formatter would have been null.
* Added tests for TraceSourceLogger.
* Removed unnecessary white-spaces.
* Make the TraceLogger output more "natural" (one line message) and similar to SystemdConsoleFormatter.
* Update TraceSourceLoggerTest.cs
* Update TraceSourceLoggerTest.cs
Co-authored-by: Maryam Ariyan <maryam.ariyan@microsoft.com>
Eirik Tsarpalis [Mon, 28 Sep 2020 16:22:44 +0000 (17:22 +0100)]
Make Process.WaitForExitAsync wait for redirected output reads (#42585)
* Make Process.WaitForExitAsync wait for output reads
Addresses an issue where Process.WaitForExitAsync
doesn't wait for background redirected output reads,
a behaviour which diverges from the sync method
equivalent. Fixes #42556.
* address feedback
* pass cancellation token to AsyncStreamReader waiter
* address feedback
* use tcs.TrySetResult
* Update src/libraries/Common/src/System/Threading/Tasks/TaskTimeoutExtensions.cs
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
Viktor Hofer [Mon, 28 Sep 2020 16:15:17 +0000 (18:15 +0200)]
Add CompileDesignTime target to IL.Sdk (#42794)
Visual Studio requires the CompileDesignTime target to be present as it is called by the language service. This fixed building ilproj projects inside Visual Studio.
Aleksey Kliger (λgeek) [Mon, 28 Sep 2020 16:13:13 +0000 (12:13 -0400)]
[docs] Clarify that mixed configuration doesn't work in wasm (#42682)
Kevin Jones [Mon, 28 Sep 2020 15:48:39 +0000 (11:48 -0400)]
Support ECDiffieHellman on X509Certificate2
* Adds support for getting ECDH public/private key objects from an X509Certificate2.
* Makes it easier to build PublicKey objects, and makes them more useful.
monojenkins [Mon, 28 Sep 2020 08:44:05 +0000 (04:44 -0400)]
[interp] Throw an InvalidProgramException instead of asserting when trying to transform a method with too many locals. (#42797)
https://github.com/dotnet/runtime/issues/42628.
Co-authored-by: vargaz <vargaz@users.noreply.github.com>
Manish Godse [Mon, 28 Sep 2020 04:22:59 +0000 (21:22 -0700)]
Get builds working when paths contain whitespaces and utf8 chars (#42671)
* Fix to get builds working for paths with whitespaces
Currently builds clr+libs when space exists in repo folder. non-ascii should also be supported.
Validated only of windows
mono is still failing.
* fix midl to not emit full path in generated headers
* avoid full filenames in python source generator
* Fix ilasm target to support folders with spaces.
* update targets for ILDASM
Jan Kotas [Mon, 28 Sep 2020 01:52:38 +0000 (18:52 -0700)]
Use CryptographicallySecureRandomBytes for Guid generation (#42770)
Fixes #42752
Tomáš Rylek [Sun, 27 Sep 2020 22:37:45 +0000 (00:37 +0200)]
Move most of src/coreclr/tests under src/tests and related cleanups (#42386)
* Move stress_dependencies, external and scripts under src/tests/Common;
* Fix build scripts to download nuget packages under artifacts, not into the source tree;
* Replace src/tests/Directory.Build.props with src/coreclr/tests/Directory.Build.props;
* Inject src/coreclr/tests/Directory.Build.targets into src/tests/Directory.Build.targets.
Thanks
Tomas
RaymondHuy [Sun, 27 Sep 2020 22:16:58 +0000 (05:16 +0700)]
Support deserialization for Struct properties that implements IList<T> (#41223)
* Support deserialization for Struct properties that implements IList<T>
Add a new field into IListOfTConverter class to store the values to prevent unboxing when the TCollection is ValueType.
* Support deserialization for struct that implements IList<T>, ICollection<T>, IList
Use constraint type instead of TCollection to avoid unboxing when TCollection is ValueType.
* Add testcase Support deserialization for Struct that implements ISet<T>, IDictionary<TKey, TValue>, IDictionary
Use constraint type to cast to avoid unboxing when the target type is ValueType
* Add testcase for reading nullable struct and null json.
Modify the 'Add' method to use actual type casting.
* Add test cases for struct collection:
- Serializing a default instance.
- Deserializing a null json string.
Co-authored-by: HuyLuong <huy.luong@orientsoftware.com>
Adeel Mujahid [Sun, 27 Sep 2020 16:44:50 +0000 (19:44 +0300)]
Delete dead C/C++ code files (#42782)
Cleaned up some dead files, most of which are either not referenced
anywhere or referencing each other but not part of the overall build
plan.
The search heuristic used is not leveraging build system, but instead
lookup with multiple passes followed by manually excluding the files
which are either part of third-party libraries (libunwind, rapidjson),
a few shims cross-referencing libraries and coreclr or files that are
included in CMakeLists via globbing patterns:
* `src/coreclr/src/pal/prebuilt/idl/*.cpp`
* `src/coreclr/src/unwinder/{ARCH}/unwinder_{ARCH}.cpp`
Stephen Toub [Sun, 27 Sep 2020 12:35:47 +0000 (08:35 -0400)]
Remove dead DnsEndPoint allocation in HttpConnectionPool (#42778)
Stephen Toub [Sun, 27 Sep 2020 12:35:24 +0000 (08:35 -0400)]
Remove unnecessary byte[] allocation from CertificateRequest.CreateSelfSigned (#42776)
Levi Broderick [Sun, 27 Sep 2020 00:31:28 +0000 (17:31 -0700)]
Remove p/invoke to RtlMoveMemory, cleanup caller (#42749)
Bruce Forstall [Sat, 26 Sep 2020 20:10:39 +0000 (13:10 -0700)]
superpmi.py script improvements (#42773)
1. Make the script pylint and pycodestyle clean (with exclusions).
Add pylint/pycodestyle configuration files.
2. Simplify and remove duplication in asm diffs code for generating
asm and JIT dump files.
3. Add cases to automatically upload and download associated .MCT files
if one or more specific .MCH files are specified.
4. Fix a few bugs related to using VS Code to view diffs. (I'm inclined
to remove this complication entirely unless someone actually uses it.)
Jan Kotas [Sat, 26 Sep 2020 18:09:27 +0000 (11:09 -0700)]
Delete empty WarningControl.h (#42771)
Tanner Gooding [Sat, 26 Sep 2020 14:43:03 +0000 (07:43 -0700)]
Update the build script to support opening S.P.Corelib (#42700)
* Update the build script to support opening S.P.Corelib
* Adding a RuntimeFlavor switch to the build scripts
Mike McLaughlin [Sat, 26 Sep 2020 02:20:55 +0000 (19:20 -0700)]
Fix DAC thread context flags (#42744)
Fix DAC thread context flags
The CONTEXT_* flags doesn't have the proper architecture specific bit set for the cross-OS/cross-arch DAC/DBI. This causes the thread contexts not to be copied correctly because of the way CORDbgCopyThreadContext (in coreclr\src\debug\shared\arm64\ primitives.cpp) masks/checks the context flags.
Fix x86 build: missing DT_CONTEXT_ALL
monojenkins [Sat, 26 Sep 2020 01:50:59 +0000 (21:50 -0400)]
[aot] Prepend the assembly name to the names of gsharedvt wrappers to avoid duplicate symbol errors during static linking. (#42667)
Fixes https://github.com/mono/mono/issues/20417.
Co-authored-by: vargaz <vargaz@users.noreply.github.com>
Buyaa [Sat, 26 Sep 2020 00:42:24 +0000 (17:42 -0700)]
Fix additional warnings introduced after fixing inconsistencty (#42750)
Vladimir Sadov [Fri, 25 Sep 2020 23:42:54 +0000 (16:42 -0700)]
SIngle-pass `CopyValueClassUnchecked` (#42654)
* use memmoveGCRefs when copying structs with GC references
* no need for a full fence when copying struct without GC references
Maoni Stephens [Fri, 25 Sep 2020 22:07:26 +0000 (15:07 -0700)]
the other part of suspension fix (#42721)
monojenkins [Fri, 25 Sep 2020 21:41:50 +0000 (17:41 -0400)]
[runtime] Transition the trampoline code to use memory managers for memory allocation. (#42462)
Co-authored-by: vargaz <vargaz@users.noreply.github.com>
Ryan Lucia [Fri, 25 Sep 2020 21:36:34 +0000 (17:36 -0400)]
Change TARGETS_BROWSER define to TARGET_BROWSER (#42635)
This makes our define name consistent across the test suite. I noticed this while making unrelated wasm-specific changes to our tests.
Ryan Lucia [Fri, 25 Sep 2020 21:35:45 +0000 (17:35 -0400)]
Add Type.Namespace tests with array types, disable on Mono (#42634)
We probably have more issues lurking in here since a lot of these reflection APIs don't seem to test array types, but this particular one was in response to a customer bug report.
Drew Scoggins [Fri, 25 Sep 2020 20:46:32 +0000 (13:46 -0700)]
Add wildcard to release branch trigger (#42644)
* Add wildcard to release branch trigger
* Change wildcard to cover all release branches
Adeel Mujahid [Fri, 25 Sep 2020 19:28:24 +0000 (22:28 +0300)]
Enable clang 10,11 and gcc 10 detection (#42583)
Bruce Forstall [Fri, 25 Sep 2020 18:59:33 +0000 (11:59 -0700)]
Improve superpmi.py script output (#42715)
Create a log file that contains all output, and restrict
console output to a more concise subset, to avoid overwhelming
the user.
The `-log_level` option allows changing the level of console output.
The `-log_file` option allows changing the location of the generated
log file.
Also, create a better asm diffs summary at the end of the asm diffs run,
and similar for replay.
Kevin Jones [Fri, 25 Sep 2020 16:51:38 +0000 (12:51 -0400)]
Retry entire test for X.509 TimeoutTests instead of just X509Chain.Build
The TimeoutTests had functionality to retry the X509Chain.Build call if
it failed to improve the reliability of the tests. However, retrying
only the Build portion broke some assumptions about how long a chain
build would take.
If Build is invoked for a second time per the retry, however the
previous call to Build primed some of the AIA or CRL/OCSP caches, this
broke the assumption of how long a delayed fetch would take.
Instead, retry the entire test with a new chain every time so that
nothing is in the cache and the timing assumptions remain valid.
Sergey Andreenko [Fri, 25 Sep 2020 16:10:35 +0000 (09:10 -0700)]
Disable ArrayBufferWriterTests.Advance with JitStress. (#42674)
* Disable ArrayBufferWriterTests.Advance with JitStress.
Issue #42517
* Try to disable broader.
Julien Couvreur [Fri, 25 Sep 2020 15:27:10 +0000 (08:27 -0700)]
Adjust for roslyn change on ref parameters and NotNullIfNotNull (#42707)
* Adjust for roslyn change on ref parameters
* Adjust for roslyn change on NotNullIfNotNull
monojenkins [Fri, 25 Sep 2020 14:46:32 +0000 (10:46 -0400)]
Fix mono_aot_plt_trampoline GC unsafe assumption. (#42685)
managed->native wrapper does the following under coop:
enter gc safe
native call
exit gc safe
since enter/exit gc safe are icalls they will be called through PLT on AOT builds. First call through a PLT slot will resolve the function using mono_aot_plt_trampoline and it currently assumes GC safe mode, so when hit as part of resolving exit gc safe, it is called under GC safe mode.
This has not showed up to be an issue under coop AOT, but asserts when running under checked build.
Fix will make sure thread is in GC unsafe mode making sure assumption of GC unsafe mode holds.
Co-authored-by: lateralusX <lateralusX@users.noreply.github.com>
Tomáš Rylek [Fri, 25 Sep 2020 12:28:59 +0000 (14:28 +0200)]
Update docs and comments to reflect the renames around build-test and runtest scripts (#42708)
* Move the script runtesttilstable.sh under src/tests
* Update most references to build-test / runtest in the runtime repo
With this change I can't find any remaining orphaned references to
the old script names except for a few superpmi-related scripts
I need to follow up on separately as I'm not too familiar with
their functionality.
Thanks
Tomas
Stephen Toub [Fri, 25 Sep 2020 10:33:07 +0000 (06:33 -0400)]
Enable several more IDE analyzer warnings (#42692)
* Enable IDE0062: Make local functions static
* Enable IDE0079: Remove unnecessary suppression
* Enable IDE0082: Convert typeof to nameof
* Enable IDE0043: Validate format string
* Enable IDE0041: Use is null check
Miha Zupan [Fri, 25 Sep 2020 08:16:18 +0000 (10:16 +0200)]
Fix System.Net.Sockets telemetry (#42188)
* Correct NameResolutionTelemetry logic
Taken from
c4c9a2f99b7e339388199086d3014041abccc21e
* Enable listening to multiple sources with TestEventListener
* Workaround EventWrittenEventArgs bug when the EventArgs are stored
Workaround https://github.com/dotnet/runtime/issues/42128
* Correct System.Net.Sockets Telemetry
* Avoid using value tuple in TestEventListener
* Remove unnecessary argument to OnCompletedInternal
* Remove redundant Telemetry.IsEnabled check
* Log Connect/Accept start before the initial context capture
* Use SocketHelperBase in Accept tests
* Avoid duplicate events for BeginConnect without ConnextEx support
* Enable Sync Socket tests
* Revert unrelated SocketPal change
* Log the correct ErrorCode in case of socket disposal
* Add more info on TelemetryTest timeout
* Add PlatformSpecific attribute to ConnectFailure test
* Add missing BeginConnect AfterConnect call on sync failure
* Add comment around GetHelperBase
* Correct WaitForEventCountersAsync helper
* Assert against SocketError.TimedOut in ConnectFailed test
* Add EndConnect comment
* Log Failure around sync SocketPal exceptions
* Don't assert that the exception message is empty
* Dispose socket in a different Thread
* Disable Telemetry failure tests for Sync on RedHat7
* Use more descriptive names in MemberData generation
* Avoid using reflection for #42128 workaround
* Remove ConnectCanceled event