Jan Vorlicek [Wed, 11 Dec 2019 15:38:52 +0000 (16:38 +0100)]
Fix Emscripten build of libraries (#762)
Updating the libraries native build to use native cmake support for the
position independent code has broken emscripten build. This change
disables that for Emscripten.
Medeni Baykal [Wed, 11 Dec 2019 14:46:53 +0000 (15:46 +0100)]
Fixed a typo. (#763)
Andrew Arnott [Wed, 11 Dec 2019 14:46:29 +0000 (07:46 -0700)]
Document caution regarding AdvanceTo's side effect (#691)
After calling `PipeReader.AdvanceTo`, the `ReadResult.Buffer` property's backing data is recycled, leading to indeterministic changes to not only the content but its `Length` property. This isn't particular predictable as a consumer and the docs didn't make it clear either.
Tomáš Rylek [Wed, 11 Dec 2019 10:18:38 +0000 (11:18 +0100)]
Switch over stress runs to live-live mode (#697)
Anton Landor [Wed, 11 Dec 2019 07:59:32 +0000 (08:59 +0100)]
Added tests to cover all branches of JsonNodeOptions (#436)
* Added tests to cover all branches of JsonNodeOptions.
* Created a test class for JsonNodeOptions and moved relevant tests to it. Also implemented tests for CommentHandling and AllowTrailingCommas options.
* Added tests for default values. Moved JsonNodeOptionsTests to the correct item group.
* Added test to make sure default JsonNodeOptions have equal values to JsonNodeOptions created from default constructor.
* Added missed options arguments.
* Added assertions to make sure default JsonNodeOptions values are as expected.
Jeremy Koritzinsky [Wed, 11 Dec 2019 02:19:56 +0000 (18:19 -0800)]
Remove unused CMake define. (#744)
Remove unused CMAKE_USER_MAKE_RULES_OVERRIDE define. We used to use this define before 3.0 in coreclr. When we stopped using it, we had to define the variable in this command on each invocation to update it in the CMake cache, otherwise the old value might still be there and cause an error. As a result, I added in these "set to empty" lines to avoid issues checking out old branches that still used these files.
Now that we're in the dotnet/runtime repo and don't have any old servicing branches, we can remove this CMake define.
Andrew Au [Wed, 11 Dec 2019 01:18:00 +0000 (17:18 -0800)]
Merge pull request #719 from SrivastavaAnubhav/optional-mapfile
Disable generating a map file by default.
Davis Goodin [Wed, 11 Dec 2019 00:35:29 +0000 (18:35 -0600)]
Add installer build/test to the runtime.yml pipeline (#705)
* First pass to get an installer build integrated into runtime.yml with minimal changes to the installer jobs.
Fix paths to jobs.
Fix typos
Add platform to parameter lists.
Pass platform to all jobs.
Cleanup variable usages.
Don't pass buildConfig down to installer jobs. They use matrix strategy instead.
Fix variable usage in windows build.
Fix windows platform names.
Add missing platform parameter for Windows_NT_arm
Update last condition re parameters.dockerImage.
Rename dockerImage parameter to productBuildDockerImage to be more explicit.
Use pool and container parameters from platform-matrix in installer build.
Pass rootfs dir via crossrootfsDir to bash-build.yml.
Fix passing down container in runtime-installer pipeline.
Remove unused Linux_x64_raw config.
* Add support for clang 9 to corehost build.
* Skip tests on Linux_musl_arm64
* runtime.yml installer build with live artifacts
Includes refactoring the runtime legs to avoid duplicating artifact
transfer logic and preserve the runtime standalone build.
* Fix installer overrides: accept live CI artifacts
* Use clone-checkout-bundle-step.yml everywhere
* Try using isOfficialBuild var from template expr
* Switch from $[] to $() to fix clone template
$[] becomes empty string when a variable is undefined. $() sticks around literally if the variable doesn't exist, messing up the command. Unfortunately $[] doesn't seem to work in this context so we must use $() and define the variable as empty string.
David Wrighton [Wed, 11 Dec 2019 00:33:57 +0000 (16:33 -0800)]
Add porting guide (#716)
- General guidance for porting CoreCLR to a new architecture
Chris Ross [Tue, 10 Dec 2019 23:36:42 +0000 (15:36 -0800)]
Sync H2 namespace updates (#710)
Jan Vorlicek [Tue, 10 Dec 2019 22:48:07 +0000 (23:48 +0100)]
Fix relro, now and PIE for host and libraries (#685)
* Fix relro, now and PIE for host and libraries
The former core-setup and corefx native code build was missing the
-z,relro and -z,now options and also the position independent related
settings.
* Reflect PR feedback
Fadi Hanna [Tue, 10 Dec 2019 21:58:48 +0000 (13:58 -0800)]
Disable more tests from the crossgen2 run (#741)
Anubhav Srivastava [Tue, 10 Dec 2019 18:11:32 +0000 (10:11 -0800)]
Rename --generate-map-file to --map.
dotnet-maestro[bot] [Tue, 10 Dec 2019 16:06:41 +0000 (17:06 +0100)]
[master] Update dependencies from dotnet/arcade Microsoft/vstest (#614)
* Update dependencies from https://github.com/dotnet/arcade build
20191205.6
- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19605.6
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19605.6
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19605.6
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.19605.6
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19605.6
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19605.6
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19605.6
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19605.6
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19605.6
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19605.6
- Microsoft.DotNet.GenAPI - 5.0.0-beta.19605.6
- Microsoft.DotNet.GenFacades - 5.0.0-beta.19605.6
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19605.6
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19605.6
* Update dependencies from https://github.com/dotnet/arcade build
20191206.1
- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19606.1
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19606.1
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19606.1
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.19606.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19606.1
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19606.1
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19606.1
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19606.1
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19606.1
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19606.1
- Microsoft.DotNet.GenAPI - 5.0.0-beta.19606.1
- Microsoft.DotNet.GenFacades - 5.0.0-beta.19606.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19606.1
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19606.1
* Update dependencies from https://github.com/dotnet/arcade build
20191207.1
- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19607.1
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19607.1
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19607.1
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.19607.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19607.1
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19607.1
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19607.1
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19607.1
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19607.1
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19607.1
- Microsoft.DotNet.GenAPI - 5.0.0-beta.19607.1
- Microsoft.DotNet.GenFacades - 5.0.0-beta.19607.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19607.1
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19607.1
* Update dependencies from https://github.com/dotnet/arcade build
20191208.1
- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19608.1
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19608.1
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19608.1
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.19608.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19608.1
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19608.1
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19608.1
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19608.1
- Microsoft.DotNet.GenAPI - 5.0.0-beta.19608.1
- Microsoft.DotNet.GenFacades - 5.0.0-beta.19608.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19608.1
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19608.1
* Update dependencies from https://github.com/microsoft/vstest build
20191210-01
- Microsoft.NET.Test.Sdk - 16.5.0-preview-
20191210-01
Jan Vorlicek [Tue, 10 Dec 2019 11:40:49 +0000 (12:40 +0100)]
Fix DynamicMethodDesc::Destroy vs code heap enumeration race (#713)
There is a race between DynamicMethodDesc::Destroy called from
the finalizer thread and the MethodDescs enumeration called from
ETW::MethodLog::SendEventsForJitMethods at process exit.
DynamicMethodDesc::Destroy cleanos up its members m_pSig and
m_pszMethodName and then it calls GetLCGMethodResolver()->Destroy();
That calls EEJitManager::FreeCodeMemory, which tries to take the
m_CodeHeapCritSec lock. But this lock is already held by
the ETW::MethodLog::SendEventsForJitMethods.
So the iterator can see half-destroyed DynamicMethodDesc and
a crash happens when trying to get the dynamic method name
from the m_pszMethodName for the ETW event purposes.
The fix is to call the GetLCGMethodResolver()->Destroy() before
destroying the m_pSig and m_pszMethodName.
Anubhav Srivastava [Tue, 10 Dec 2019 03:46:44 +0000 (19:46 -0800)]
Disable generating a map file by default.
Swaroop Sridhar [Tue, 10 Dec 2019 02:50:44 +0000 (18:50 -0800)]
Update Multi-level SharedFX Lookup Doc (#503)
This document was written before .net core 3 was released, and
refers to certain features in .net core 2.1 and 3 as "proposed features."
This change integrates the sections describing (then) upcoming features
into the doc's mainstream.
The doc can be improved further to be more cohesive; this change
addresses the specific issue in #493
Fixes #493
Mateo Torres-Ruiz [Tue, 10 Dec 2019 02:09:41 +0000 (18:09 -0800)]
Merge pull request #692 from mateoatr/issue-353
Fix memory leaked caused by Marshal.GetFunctionPointerForDelegate
Bruce Forstall [Mon, 9 Dec 2019 23:45:42 +0000 (15:45 -0800)]
Improve superpmi collection steps (#652)
Rearrange the "clean" and "remove dups" phases of the collection
steps. This saves a lot of time by avoiding JIT compiling
all the functions that we will just throw out because the MCs
are considered duplicates.
Fixes #646
Tomáš Rylek [Mon, 9 Dec 2019 23:44:06 +0000 (00:44 +0100)]
Live-live CoreCLR ci.yml (#586)
Jarret Shook [Mon, 9 Dec 2019 23:21:19 +0000 (00:21 +0100)]
Re-enable arm32 windows testing (#642)
Cory Nelson [Mon, 9 Dec 2019 22:32:36 +0000 (14:32 -0800)]
Merge pull request #711 from dotnet/tratcher/codeowners
Add back CODEOWNERS for Http2 shared code
Chris R [Mon, 9 Dec 2019 22:16:25 +0000 (14:16 -0800)]
Add back CODEOWNERS for Http2 shared code
Nikolche Kolev [Mon, 9 Dec 2019 22:12:36 +0000 (14:12 -0800)]
Update repoRestore.targets (#451)
v-jizho2 [Mon, 9 Dec 2019 22:07:04 +0000 (14:07 -0800)]
Fix zero connection timeout issue (#400)
* Fix zero connection timeout issue
* Update based on comments
* remove trailing whitespace
* minor change to ctor
Jan Vorlicek [Mon, 9 Dec 2019 21:21:26 +0000 (22:21 +0100)]
Implement classMustBeLoadedBeforeCodeIsRun in crossgen2 (#608)
* Implement classMustBeLoadedBeforeCodeIsRun in crossgen2
This fixes missing eager fixups for method call arguments and return
value for methods going through PreStub.
It fixes the coreclr JIT\Regression\CLR-x86-JIT\V1.2-M01\b02345\b02345
test that needed Span<byte> type during GC stack walk with PreStub of a
method with Span<byte> argument on the stack and failed as it couldn't
be loaded during the GC.
The same failure also happened in many if not all tests when I have
attempted to run them with GCStress 3.
* Reflect PR feedback on member / property naming and method location
Sung Yoon Whang [Mon, 9 Dec 2019 21:07:05 +0000 (13:07 -0800)]
Turn back on some of the EventPipe tests that were disabled in GCStress (#533)
* Turn back on some of the EventPipe tests that were disabled in GCStress
* Mark more EventPipe tests as gcstress compatible
Next Turn [Mon, 9 Dec 2019 20:21:25 +0000 (04:21 +0800)]
Publish nethost static library (#296)
mateoatr [Mon, 9 Dec 2019 19:30:11 +0000 (19:30 +0000)]
Poison code before deleting the handle.
Keep code consistent.
mateoatr [Mon, 9 Dec 2019 19:05:19 +0000 (19:05 +0000)]
Delete handle before adding thunk to free list.
Set object handle to zero after destroying the handle.
Tomáš Rylek [Mon, 9 Dec 2019 17:53:14 +0000 (18:53 +0100)]
Add new YAML template for combined build of CoreCLR and libraries (#626)
As a common idiom in many live-live CoreCLR pipelines, we need
libraries to build a platform matrix equivalent to the CoreCLR
testing matrix. I have created a new template to capture that idiom.
I have initially switched over the Crossgen2 pipeline to use it.
Thanks
Tomas
mateoatr [Mon, 9 Dec 2019 17:11:22 +0000 (17:11 +0000)]
Destroy handle on UMEntryThunk::Terminate
Bruce Forstall [Mon, 9 Dec 2019 16:48:02 +0000 (08:48 -0800)]
Fix runtest.py for the dotnet/runtime repo (#669)
I simplified a lot of functions to just take the "args"
object, which contains a lot of paths and directories.
This makes it easier to centralize the directory computations
without spreading paths throughout the function signatures.
Viktor Hofer [Mon, 9 Dec 2019 15:56:35 +0000 (16:56 +0100)]
Add runtime-assets dependency (#671)
Adding a dependency to dotnet/runtime-assets to enable dependency flow
for (test) data packages.
Roman Marusyk [Mon, 9 Dec 2019 13:48:49 +0000 (14:48 +0100)]
Consolidate .netcoreapp.cs test files in System.Net.* (#664)
* Consolidate .netcoreapp.cs files because System.Net.* projects is no longer cross-compiled
* Fix build
* Add end-of-line
Anubhav Srivastava [Mon, 9 Dec 2019 10:47:40 +0000 (02:47 -0800)]
Fix logging initialization race (#616) (#634)
* Make logging initialization thread-safe. Fix broken defines for InterlockedCompareExchangePointer and InterlockedExchangePointer.
* Add InitLogging back to log.h. Revert change to the defines in utilcode.h. Move LogFileMutex initialization to InitializeLogging.
Jan Vorlicek [Mon, 9 Dec 2019 10:43:19 +0000 (11:43 +0100)]
Fix crossgen2 issue with generic delegates (#613)
The coreclr pri1 test is failing with the following error:
Expected typeof generic method Type Parameter for parameter, 'aaaa', to
be 'System.String', but found 'System.__Canon'
The issue was caused by the fact that we were not setting the
isInstantiatingStub for calls to generic delegates. The old crossgen
was doing that correctly.
Eugene Samoylov [Mon, 9 Dec 2019 07:56:59 +0000 (12:56 +0500)]
Add support for escaped characters in JsonString (for {Try}GetDateTime{Offset}) (#386)
* Add support for escaped characters in JsonString (for {Try}GetDateTime{Offset})
* Fix the review issues
* Fix a mistake after resolving a merge conflict
* Change `var` to explicit type
Jan Kotas [Sun, 8 Dec 2019 21:10:15 +0000 (13:10 -0800)]
Correct nullability annotations for CreateInstance implementation details (#675)
Fixes #674
Tomáš Rylek [Sun, 8 Dec 2019 21:00:04 +0000 (22:00 +0100)]
Fix race condition in live-live CoreCLR PR (#658)
I was going over the various artifacts while writing a
response to Chad Nedzlek trying to explain to him that his
assessment of the bug is incorrect and I found out he is
actually right. As the managed test components corresponding
to the "test_build_..." can be redirected to a different OS,
we need to create an explicit dependency edge as otherwise
there's no guarantee that the run job will depend on the
library build job for the corresponding platform.
Thanks
Tomas
Thomas Gassmann [Sun, 8 Dec 2019 11:22:34 +0000 (12:22 +0100)]
Fix broken links in workflow docs (#659)
* Fix broken links in workflow docs
There were several broken links in the workflow documentation.
* Update paths for CoreCLR debugging in documentation
Marco Rossignoli [Sat, 7 Dec 2019 01:41:08 +0000 (02:41 +0100)]
Implement AdvanceToEnd,UnreadSequence (#388)
* UnreadSequence, AdvanceToEnd
* default _nextPosition
* add some tests
* remove AggressiveInlining
* address PR feedback
* Update src/libraries/System.Memory/src/System/Buffers/SequenceReader.Search.cs
Co-Authored-By: Ahson Khan <ahkha@microsoft.com>
* Update src/libraries/System.Memory/src/System/Buffers/SequenceReader.cs
Co-Authored-By: Ahson Khan <ahkha@microsoft.com>
* fix docs, add some tests
Emmanuel André [Sat, 7 Dec 2019 01:09:01 +0000 (02:09 +0100)]
Test span-based CopyTo/Async method (#390)
* Add tests for span-based CopyTo/Async methods
* Add tests for cross sync streams
A custom stream implementation can override CopyTo(Stream) methods with calls to WriteAsync and vice versa
* Add new tests using CallTrackingStream
* Apply suggestions from code review
Co-Authored-By: Stephen Toub <stoub@microsoft.com>
* Apply additional suggestions from code review
Eric StJohn [Sat, 7 Dec 2019 00:19:56 +0000 (16:19 -0800)]
Merge pull request #631 from ericstj/fixNetCoreAppRestore
Fix restore of netcoreapp when not downloading any packages
mateoatr [Fri, 6 Dec 2019 23:42:52 +0000 (23:42 +0000)]
Check if a UMEntryThunk has an existing handle
before creating a new one.
Tomáš Rylek [Fri, 6 Dec 2019 23:28:14 +0000 (00:28 +0100)]
Fix two subtle bugs in CoreCLR test build (#625)
1) As Bruce reported, my recent live-live change regressed local
CoreCLR build in the sense that it always ended with a spurious
error message from the patch-corefx script. This was due to a typo
where I forgot to escape a parenthesis in an echo message that
the batch script parser ended up understanding as closing the
conditional block.
2) I added several folders to the exclusion list representing special
folders under "artifacts/tests/coreclr/os.arch.config" that should
be skipped when building the test wrappers.
Fixes: #578
Thanks
Tomas
Eirik Tsarpalis [Fri, 6 Dec 2019 22:48:36 +0000 (22:48 +0000)]
Initial SslStream stress app work (#290)
Fadi Hanna [Fri, 6 Dec 2019 22:38:02 +0000 (14:38 -0800)]
Exclude test from crossgenning (#592)
* Exclude tests from crossgenning
Tomas Weinfurt [Fri, 6 Dec 2019 22:18:28 +0000 (14:18 -0800)]
use 2x buffer when sysctl fails with ENOMEM (#591)
* use 2x buffer when sysctl fails with ENOMEM
* add comment to sysctl buffer logic
David Shulman [Fri, 6 Dec 2019 22:11:21 +0000 (14:11 -0800)]
Remove obsolete tests from System.Net.Security (#624)
These tests haven't been running nor even compiling for several years. The compilation
broke when CSPROJ changes were made regarding TargetsLinux vs. TargetsUnix.
The tests themselves required running in admin mode and made impactful changes on the
host machine (such as installing a KDC).
Now that enterprise scenario tests have been added with PR #463, these tests are no longer
needed. Much of these tests have already been incorporated into the new enterprise
tests.
Closes https://github.com/dotnet/corefx/issues/30150
Closes https://github.com/dotnet/corefx/issues/24707
Eric StJohn [Fri, 6 Dec 2019 21:19:36 +0000 (13:19 -0800)]
Fix restore of netcoreapp when not downloading any packages
Santiago Fernandez Madero [Fri, 6 Dec 2019 20:45:35 +0000 (12:45 -0800)]
Add path filters to runtime.yml triggers (#623)
* Add path filters to runtime.yml triggers
* Exclude runtime.yml from other PR and CI pipelines
Youssef Victor [Fri, 6 Dec 2019 19:13:31 +0000 (21:13 +0200)]
Remove duplicates of Interop.Kernel32.GenericOperations.GENERIC_READ and GENERIC_WRITE (#292)
Remove duplicates of Interop.Kernel32.GenericOperations.GENERIC_READ and GENERIC_WRITE
Jarret Shook [Fri, 6 Dec 2019 18:32:34 +0000 (10:32 -0800)]
Ci Health and investigation doc (#551)
* Ci Health and investigation doc
* Address feedback
* Tend->Trend
Bruce Forstall [Fri, 6 Dec 2019 18:29:44 +0000 (10:29 -0800)]
Change `mcs -dump` to show the full function name for each MC (#600)
Fadi Hanna [Fri, 6 Dec 2019 17:40:59 +0000 (09:40 -0800)]
Revert target framework version change to netcoreapp3.0 when building from VS (#584)
Fadi Hanna [Fri, 6 Dec 2019 17:39:04 +0000 (09:39 -0800)]
Revert --resilient from being the default on crossgen2 for now (#619)
Stephen Toub [Fri, 6 Dec 2019 17:02:53 +0000 (12:02 -0500)]
Add test for CancellationTokenRegistration.Unregister race condition (#313)
Tomáš Rylek [Fri, 6 Dec 2019 15:56:17 +0000 (16:56 +0100)]
Switch over CoreCLR pr.yml to use live-built libraries (#520)
* Switch over CoreCLR pr.yml to use live-built libraries
* Delete CoreFX runs against CoreCLR
* PR feedback - only use release CoreFX builds for CoreCLR testing
Stephen Toub [Fri, 6 Dec 2019 14:48:52 +0000 (09:48 -0500)]
Improve perf and scalability of Regex's cache (#542)
Regex maintains a cache used for the static methods on Regex, e.g. Regex.IsMatch. The cache is implemented as an LRU cache, which maintains a linked list and a dictionary of the cached instances. The linked list maintains the order in which the cached instances were last accessed, making it cheap to expunge older items from the cache. However, that comes at a significant cost: unless the item is the very first one in the linked list, all reads on the cache require taking a global lock, because the linked list needs to be mutated to move the found node to the beginning. That lock has both throughput and scalability implications.
This PR changes the cache from using a `Dictionary<>` and a linked list to instead using a `ConcurrentDictionary<>` and a `List<>`. Rather than making all accesses more expensive in order to make drops less expensive, it makes all reads much cheaper and more scalable, at the expense of making drops more expensive. Since dropping from the cache means we're already paying the expensive cost of creating/parsing/compiling/etc. a new Regex instance, this is a better trade-off, especially since any frequent dropping suggests the consuming app or library needs to revisit its Regex strategy, either using Regex.CacheSize to increase the cache size appropriately, or doing its own caching (e.g. creating the Regex instance it needs and storing it into a field for all future use).
The new scheme uses a `ConcurrentDictionary<Key,Node>`, a `List<Node>`, and a fast-path field storing the most recently used Regex instance (just as the existing implementation did). On lookups, if the fast-path field has the matching value, it's just returned. Otherwise, the dictionary is consulted, and if the item is found, the fast-path field is updated. No locking at all is employed, and only a few volatile read/writes are used to update a "last access stamp" that's used to indicate importance if/when items do need to be expunged. On additions, we do still take a global lock and add to the cache. If this puts us over our cache size, we pick an item from the list and remove it. If the list is small, we just examine all of the items looking for the oldest. If the list is larger, we examine a random subset of it; we may not get rid of the absolute oldest item, but it'll be old enough.
Jan Vorlicek [Fri, 6 Dec 2019 14:27:30 +0000 (15:27 +0100)]
Fix coredistools lookup for users with spaces in usernames (#607)
Locating the downloaded coredistools.dll fails for users that have
spaces in their names (their home directory has spaces in the path).
This change fixes the problem by adding quotes to the locating command.
Roman Marusyk [Fri, 6 Dec 2019 09:22:40 +0000 (10:22 +0100)]
Update links (#537)
dotnet-maestro[bot] [Fri, 6 Dec 2019 09:20:22 +0000 (10:20 +0100)]
Update dependencies from https://github.com/dotnet/arcade build
20191204.4 (#560)
- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.19604.4
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.19604.4
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.19604.4
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.19604.4
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.19604.4
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.19604.4
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.19604.4
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.19604.4
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.19604.4
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.19604.4
- Microsoft.DotNet.GenAPI - 5.0.0-beta.19604.4
- Microsoft.DotNet.GenFacades - 5.0.0-beta.19604.4
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.19604.4
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.19604.4
Bruce Forstall [Fri, 6 Dec 2019 05:56:22 +0000 (21:56 -0800)]
Force Python 3.7 for superpmi.py (#603)
Async doesn't work well on Windows before this.
Anubhav Srivastava [Fri, 6 Dec 2019 02:43:00 +0000 (18:43 -0800)]
Enable parallelism in Crossgen2 and add flags for it in SuperIlc. (#495)
* Enable parallelism in Crossgen2 and add flags for it in SuperIlc.
* Use fluent api pattern in ReadyToRunCodegenCompilationBuilder for resilient, IBC tuning, and parallelism flags.
Vladimir Sadov [Fri, 6 Dec 2019 02:34:15 +0000 (18:34 -0800)]
Merge pull request #581 from VSadov/BigProcNum
Fix Thread.GetCurrentProcessorId for > 64 CPUs on Windows.
Kristian Hellang [Fri, 6 Dec 2019 01:41:21 +0000 (02:41 +0100)]
Use MaxExpansionFactorWhileTranscoding instead of counting bytes (#478)
David Shulman [Thu, 5 Dec 2019 23:32:13 +0000 (15:32 -0800)]
Enterprise Testing System (#463)
This is the first of several PRs that add Enterprise Scenarios Testing capability to
the repo. This PR focusses on Linux which allows for docker containers to be used
in an enterprise network configuration.
I focussed on 2 workflows: 1) The 'dev' workflow, and 2) The PR/CI workflow. The dev
workflow works well since it's using containers in a docker-compose environment along
with volume mounting your current dev's repo enlistment. The PR/CI workflow gives us
an Azure DevOps pipeline to automate verification.
I still need to work with the infra team to add a real pipeline that will run. I can't
do that until this is merged. In the meantime, I have my own DevOps pipeline that verified this PR.
See: https://dev.azure.com/systemnetncl/Enterprise%20Testing/_build/results?buildId=141
I will be linking a follow-up GitHub issue describing the roadmap for building on this system
including adding Windows environments, NTLM protocol, proxies, and other libraries such as
System.Net.Mail and System.Data.SqlClient. Those libraries also use Negotiate/Kerberos/NTLM
enterprise-oriented protocols.
Contributes to:
https://github.com/dotnet/corefx/issues/41652
https://github.com/dotnet/corefx/issues/41489
https://github.com/dotnet/corefx/issues/36896
https://github.com/dotnet/corefx/issues/30150
https://github.com/dotnet/corefx/issues/24707
https://github.com/dotnet/corefx/issues/10041
https://github.com/dotnet/corefx/issues/6606
https://github.com/dotnet/corefx/issues/6161
* Address PR feedback
* Change pipeline *.yml to only run on selected filepaths for PRs
* Change kdc container Dockerfile to be based on ubuntu:18.04
* Fix typo in README.md
* Update .yml file
* Link (instead of copy) apache kerb module to the right place
Bruce Forstall [Thu, 5 Dec 2019 22:55:09 +0000 (14:55 -0800)]
Fix build-test.cmd help message for generatelayoutonly option (#595)
Erhan Atesoglu [Thu, 5 Dec 2019 22:42:00 +0000 (14:42 -0800)]
Changed corefx to libraries in System.Private.CoreLib (#532)
* Changed corefx to libraries
* Update src/libraries/System.Private.CoreLib/src/System/Runtime/Serialization/SerializationInfo.cs
Co-Authored-By: Stephen Toub <stoub@microsoft.com>
* Update src/libraries/System.Private.CoreLib/src/System/Runtime/Serialization/SerializationInfo.cs
Co-Authored-By: Stephen Toub <stoub@microsoft.com>
* Update src/libraries/System.Private.CoreLib/src/Interop/Unix/Interop.Errors.cs
Co-Authored-By: Stephen Toub <stoub@microsoft.com>
* Update src/libraries/System.Private.CoreLib/src/Internal/Win32/RegistryKey.cs
Co-Authored-By: Stephen Toub <stoub@microsoft.com>
Charles Stoner [Thu, 5 Dec 2019 22:39:40 +0000 (14:39 -0800)]
Add README.md for archived runtime packages (#570)
* Add README.md for archived runtime packages
* package -> library
Marco Rossignoli [Thu, 5 Dec 2019 21:50:08 +0000 (22:50 +0100)]
Implement SequenceReader.TryPeek(long offset, out T value) (#346)
* address PR feedback
* Update src/libraries/System.Memory/src/System/Buffers/SequenceReader.cs
Co-Authored-By: Ahson Khan <ahkha@microsoft.com>
* address PR feedback
Stephen Toub [Thu, 5 Dec 2019 21:48:11 +0000 (16:48 -0500)]
Add ConcurrentCollectionsEventSource to EventSource test exemptions list (#565)
Santiago Fernandez Madero [Thu, 5 Dec 2019 21:19:27 +0000 (13:19 -0800)]
Update libraries outerloop pipelines to use a lively built coreclr (#530)
Bruce Forstall [Thu, 5 Dec 2019 21:14:55 +0000 (13:14 -0800)]
Save PMI stdout/stderr during SuperPMI collections (#579)
Fadi Hanna [Thu, 5 Dec 2019 20:50:39 +0000 (12:50 -0800)]
Add --resilient switch to crossgen2 compilations (#588)
There are negative scenario tests that will cause the JIT to fail to compile certain methods. Adding the resilient switch to the RunCrossgen2 infrastructure prevents crossgen2 from failing on such errors (default behavior of crossgen1)
Egor Chesakov [Thu, 5 Dec 2019 20:13:34 +0000 (12:13 -0800)]
[Arm64] Fix JitDump assertions when emitting ld1 (#544)
* Add INS_ld1 case in emitter::emitInsTargetRegSize in jit/emitarm64.cpp
* Remove assertion in emitter::emitDispIns in jit/emitarm64.cpp
Carol Eidt [Thu, 5 Dec 2019 20:10:12 +0000 (12:10 -0800)]
Add bilinear interpolation benchmark (#566)
This is also checked in to the performance repo. Adding it here makes it easier to include in diffs and other code analysis. Also, it exposed a correctness bug in the Arm64 `Vector<T>` implementation; the convert from floating point to integer needs to truncate.
Egor Chesakov [Thu, 5 Dec 2019 20:10:01 +0000 (12:10 -0800)]
[Arm64] Implement BitwiseSelect hardware intrinsic (#472)
Fadi Hanna [Thu, 5 Dec 2019 20:06:21 +0000 (12:06 -0800)]
Fix inaccurate comment. (#585)
Crossgen2 is in its own package, not included with the runtime package
Fadi Hanna [Thu, 5 Dec 2019 19:59:37 +0000 (11:59 -0800)]
Unset RunCrossgen2 variable when CrossGenTest is set to false (#583)
Jan Vorlicek [Thu, 5 Dec 2019 19:26:54 +0000 (20:26 +0100)]
Fix crossgen2 gc layout handling for TypedReference (#555)
* Fix crossgen2 gc layout handling for TypedReference
The getClassGCLayout jit interface method was not handling TypedReference
explicitly like its runtime counterpart. That was causing the layout to
be incorrect for TypedReference and resulted in 100 failures (asserts)
in the coreclr pri1 tests build and one framework assembly build with
crossgen2.
The assert in JIT was in Linux specific code path, so we weren't hitting
it on Windows.
With this fix, the build and test run results match the Windows ones.
* Remove the System.Runtime.Serialization.Formatters from exclusions
The assembly is now compiled ok
Bruce Forstall [Thu, 5 Dec 2019 19:22:10 +0000 (11:22 -0800)]
Fix calling getMethodName on our function, so it ends up in the MCH (#548)
vsadov [Thu, 5 Dec 2019 18:55:16 +0000 (10:55 -0800)]
Fix GetCurrentProcessorId for > 64 CPUs on Windows.
Bruce Forstall [Thu, 5 Dec 2019 18:14:33 +0000 (10:14 -0800)]
Update superpmi.py script for new 'runtime' repo layout (#541)
Updated a few comments as well.
Fadi Hanna [Thu, 5 Dec 2019 17:27:12 +0000 (09:27 -0800)]
Two more fixes for crossgen2 (#538)
* Reference the test working directory when running crossgen2
* Fix GCRefMapBuilder to correctly detect if certain methods are "empty" and not crash
Viktor Hofer [Thu, 5 Dec 2019 17:23:07 +0000 (18:23 +0100)]
Upgrade global.json to 5.0-alpha1 SDK (#445)
* Upgrade global.json to 5.0-alpha1 SDK
* Introduce a NetCoreApp TFM
* Don't apply RuntimeFramework for depprojs
* Binplace framework refs for netcoreapp
* Update pkg testing for netcoreapp5.0 sdk
Leslie Zhai [Thu, 5 Dec 2019 16:13:54 +0000 (00:13 +0800)]
Change clang static analyzer reports store directory to BinDir/scan-build-log (#559)
https://github.com/dotnet/runtime/issues/558
Bruce Forstall [Thu, 5 Dec 2019 15:51:09 +0000 (07:51 -0800)]
Fix various SuperPMI dumping problems (#547)
Viktor Hofer [Thu, 5 Dec 2019 14:51:58 +0000 (15:51 +0100)]
Remove clean.cmd/sh scripts from coreclr root (#564)
The clean.cmd/sh scripts aren't necessary anymore as coreclr now uses
the user profile nuget cache for the packages and for the binaries there
is a common -clean action in the arcade build root script which is
invoked by the repo root scripts.
Viktor Hofer [Thu, 5 Dec 2019 14:51:17 +0000 (15:51 +0100)]
Delete broken sync.sh script from coreclr (#562)
Viktor Hofer [Thu, 5 Dec 2019 14:50:45 +0000 (15:50 +0100)]
Delete unused dotnet-download.ps1 file in coreclr (#561)
Andrew Arnott [Thu, 5 Dec 2019 13:52:29 +0000 (06:52 -0700)]
Fix xml doc comment on FlushResult.IsCompleted (#491)
Jan Vorlicek [Thu, 5 Dec 2019 10:40:22 +0000 (11:40 +0100)]
Fix crossgen2 ArgIterator for SystemV Amd64 Unix ABI (#499)
* Fix crossgen2 ArgIterator for amd64 Unix ABI
The ArgIterator was missing proper handling of the SystemV struct classification.
There were also several minor issues.
* Remove the SystemV struct classification cache
Running without the cache showed no difference in the total build time of
all the coreclr pri1 tests and all the runtime assemblies, so I am removing
the cache.
Jarret Shook [Thu, 5 Dec 2019 06:17:53 +0000 (22:17 -0800)]
Fix linux musl arm64 testing (#523)
* Fix linux musl arm64 testing
1. Increase crossgen-comparion job to 2 hour timeout
2. modify .gitignore to include .devcontainer -> vscode hidden folder
* Update comment
Sinan Kaya [Thu, 5 Dec 2019 05:25:52 +0000 (00:25 -0500)]
Document FPU options (#546)
Bruce Forstall [Thu, 5 Dec 2019 04:40:57 +0000 (20:40 -0800)]
Fix comments referencing non-existent CorInfoFlags value. (#549)
Stephen Toub [Thu, 5 Dec 2019 03:11:25 +0000 (22:11 -0500)]
Annotate System.ComponentModel.EventBasedAsync for nullable reference types (#466)
* Annotate System.ComponentModel.EventBasedAsync for nullable reference types
* Address PR feedback
Stephen Toub [Thu, 5 Dec 2019 03:11:12 +0000 (22:11 -0500)]
Annotate System.IO.FileSystem.DriveInfo for nullable reference types (#464)
* Annotate System.IO.FileSystem.DriveInfo for nullable reference types
* Address PR feedback
Stephen Toub [Thu, 5 Dec 2019 03:00:49 +0000 (22:00 -0500)]
Update nullability.md