platform/upstream/dotnet/runtime.git
4 years agoTiny cleanup - make host_startup_info return void
Vitek Karas [Mon, 13 Jan 2020 13:31:55 +0000 (05:31 -0800)]
Tiny cleanup - make host_startup_info return void

The original return value was always 0 and never actually checked.

4 years agoAnnotate System.Security.Cryptography.Encoding for nullable (#1535)
buyaa-n [Mon, 13 Jan 2020 06:32:45 +0000 (22:32 -0800)]
Annotate System.Security.Cryptography.Encoding for nullable (#1535)

* Annotate System.Security.Cryptography.Encoding for nullable

4 years agoThrow NotSupportedException for WaitHandle.WaitAny with 64 handle on STA threads...
Jan Kotas [Mon, 13 Jan 2020 02:58:25 +0000 (18:58 -0800)]
Throw NotSupportedException for WaitHandle.WaitAny with 64 handle on STA threads (#1647)

This change is technically a breaking change, but it is better to throw than silently do the wrong thing.

Fixes #1646

4 years agoMake PhysicalAddress.GetHashCode thread-safe. (#1649)
NN [Sun, 12 Jan 2020 22:56:20 +0000 (00:56 +0200)]
Make PhysicalAddress.GetHashCode thread-safe. (#1649)

4 years agoDelete RHEL6 leftovers (#1648)
Jan Kotas [Sun, 12 Jan 2020 20:49:27 +0000 (12:49 -0800)]
Delete RHEL6 leftovers (#1648)

Fixes #423
Closes dotnet/corefx#24525
Closes dotnet/corefx#37739
Closes dotnet/corefx#31792

4 years agoSimplify STA apartment condition in BlockingCollection.TryAddToAny (#1635)
Jan Kotas [Sun, 12 Jan 2020 20:43:42 +0000 (12:43 -0800)]
Simplify STA apartment condition in BlockingCollection.TryAddToAny (#1635)

Add tests for this codepath

4 years ago[armel/Linux] Fix armel build (#1609)
Alexander Soldatov [Sun, 12 Jan 2020 07:58:09 +0000 (10:58 +0300)]
[armel/Linux] Fix armel build (#1609)

Option '-DARM_SOFTFP' not passed after #27077 which required for armel build.
This commit fix it.

4 years agoAnnotate System.Security.Claims for nullable (#1431)
buyaa-n [Sun, 12 Jan 2020 06:50:48 +0000 (22:50 -0800)]
Annotate System.Security.Claims for nullable (#1431)

* Annotate System.Security.Claims for nullable

4 years agoRemove dead code in BlockingCollection (#636)
Marek Safar [Sat, 11 Jan 2020 18:24:49 +0000 (19:24 +0100)]
Remove dead code in BlockingCollection (#636)

* Remove dead code in BlockingCollection

* Revert "Remove dead code in BlockingCollection"

This reverts commit b0f1f86b909de1a4be37f55c05697bb12343e93b.

* Implement BlockingCollection::IsSTAThread

4 years agoChange NetworkStream.Socket property from protected to public (#1598)
Felipe Pessoto [Sat, 11 Jan 2020 18:19:33 +0000 (15:19 -0300)]
Change NetworkStream.Socket property from protected to public (#1598)

* Change NetworkStream.Socket property from protected to public. (https://github.com/dotnet/corefx/issues/35410)

* Removes ExposedSocketNetworkStream

* Remove superfluous property

* Update src/libraries/System.Net.Sockets/ref/System.Net.Sockets.cs

Co-Authored-By: Stephen Toub <stoub@microsoft.com>
Co-authored-by: Stephen Toub <stoub@microsoft.com>
4 years agoAnnotate Microsoft.Win32.SystemEvents for nullable reference types (#1621)
Stephen Toub [Sat, 11 Jan 2020 18:19:08 +0000 (13:19 -0500)]
Annotate Microsoft.Win32.SystemEvents for nullable reference types (#1621)

* Annotate Microsoft.Win32.SystemEvents for nullable reference types

* Address PR feedback

4 years agoUnify cmake_minimum_required to 3.14.2 (#1315)
Jan Kotas [Sat, 11 Jan 2020 17:14:54 +0000 (09:14 -0800)]
Unify cmake_minimum_required to 3.14.2 (#1315)

- Set cmake_minimum_required in top level projects to 3.14.2 (the lowest version used by the CI currently)
- Delete cmake_minimum_required from child projects
- Leave the cmake_minimum_required alone in vendored projects (zlib)
- Note CMake version in the docs

Fixed #1311

4 years agoAdd more known Regex patterns to increase test coverage. (#1617)
Eric Erhardt [Sat, 11 Jan 2020 14:05:14 +0000 (08:05 -0600)]
Add more known Regex patterns to increase test coverage. (#1617)

4 years agoAnnotate System.Diagnostics.FileVersionInfo for nullable reference types (#1614)
Stephen Toub [Sat, 11 Jan 2020 13:45:52 +0000 (08:45 -0500)]
Annotate System.Diagnostics.FileVersionInfo for nullable reference types (#1614)

* Annotate System.Diagnostics.FileVersionInfo for nullable reference types

* Address PR feedback

4 years agoNormalize Regex char class containing a single negated character (#1597)
Stephen Toub [Sat, 11 Jan 2020 11:57:27 +0000 (06:57 -0500)]
Normalize Regex char class containing a single negated character (#1597)

* Normalize Regex char class containing a single negated character

When RegexFC scans the regex node tree looking for what characters can be the first character, if it encounters a negated character, it handles it by adding everything else to the set.  That then ends up defeating optimizations based on IsSingletonInverse, which expects a negated character class containing only the character, as opposed to a non-negated character class containing everything but the character.  There is already a RegexCharClass.Canonicalize method; this PR just adds the check and fix-up for this case.

* Address PR feedback

4 years agoPartial Revert "Add asserts to catch unintentional use of generic overloads (#1200...
Jan Kotas [Sat, 11 Jan 2020 07:53:38 +0000 (23:53 -0800)]
Partial Revert "Add asserts to catch unintentional use of generic overloads (#1200)" (#1628)

This reverts commit f6d14eeaf290a78e547f4cf15476e61b53e06b5c.

4 years agoEnable TRACE_GC for the whole gc.cpp (#1604)
Andrew Au [Sat, 11 Jan 2020 06:22:12 +0000 (22:22 -0800)]
Enable TRACE_GC for the whole gc.cpp (#1604)

4 years agoChange how we detect SSL3 on Windows (#1270)
Ganbarukamo41 [Sat, 11 Jan 2020 05:35:07 +0000 (14:35 +0900)]
Change how we detect SSL3 on Windows (#1270)

4 years agoAdd Library builds for release-testing job (#1596)
Swaroop Sridhar [Sat, 11 Jan 2020 04:08:00 +0000 (20:08 -0800)]
Add Library builds for release-testing job (#1596)

* Add Library builds for release-testing job

Add library-build steps to the release-testing job
to adapt live-live builds.

Use Common template for CoreCLR and Library builds

4 years agoFollow coding conventions (#297)
Youssef Victor [Sat, 11 Jan 2020 03:35:08 +0000 (05:35 +0200)]
Follow coding conventions (#297)

* Follow coding conventions

* Update SafeProcessHandle.Unix.cs

* Update shmemory.h

* Update shmemory.cpp

* Update MetadataTests.cs

* Update SqlDependency.cs

* Update comcache.cpp

* Update stubmgr.h

* Update SqlDependencyListener.cs

* Update assemblybinder.cpp

* Update Socket.cs

* Address stephen feedback

* Address  feedback

4 years agogc, pal: Avoid linking libnuma.so on non-NUMA systems (#1468)
Leandro A. F. Pereira [Sat, 11 Jan 2020 03:34:06 +0000 (19:34 -0800)]
gc, pal: Avoid linking libnuma.so on non-NUMA systems (#1468)

* gc, pal: Avoid linking libnuma.so on non-NUMA systems

In an effort to reduce the system calls performed during the
initialization, this seemed like another low-hanging fruit: only link
in libnuma.so if the system has more than one NUMA node.

* gc, pal: Remove unneeded dlsym() call

The result of dlsym() is never attributed to any variable, and this
function is never called, so it's safe to remove these dlsym() calls.

4 years agoFixup orphaned references to GetRawSzArrayData (#1458)
Levi Broderick [Sat, 11 Jan 2020 02:45:23 +0000 (18:45 -0800)]
Fixup orphaned references to GetRawSzArrayData (#1458)

4 years agoAdd LibrariesAllConfigurationsOverridePath (#1620)
Davis Goodin [Sat, 11 Jan 2020 01:56:43 +0000 (19:56 -0600)]
Add LibrariesAllConfigurationsOverridePath (#1620)

4 years agoUpdate workflow readme (#1512)
Dan Moseley [Sat, 11 Jan 2020 00:24:38 +0000 (16:24 -0800)]
Update workflow readme (#1512)

* Update workflow readme

* CR feedback and more detail

* Add daily workflow for libraries dev

* rebuild->build

* More updates

* typo:

* more

* More edits

* testing

* space:

4 years agoFix all GC stress failures in Pri1 tests on Unix with crossgen2 (#1619)
Jan Vorlicek [Fri, 10 Jan 2020 23:58:43 +0000 (00:58 +0100)]
Fix all GC stress failures in Pri1 tests on Unix with crossgen2 (#1619)

* Fix all GC stress failures in Pri1 tests on Unix with crossgen2

Without this PR, 90% of the CoreCLR Pri1 tests are failing due to GC
holes. All of the errors are caused by various bugs in the GCRefMap
generation that this PR fixes. With this PR, all the tests that pass
without GC stress also pass with it.

1. GetCorElementType() was incorrectly casting from TypeFlags to
CoreElementType. It was checking for a range of TypeFlags, but that
range didn't map to CorElementType 1:1. That resulted e.g. in getting
ELEMENT_TYPE_R4 for TypeFlags.IntPtr.
2. Besides that, the GetCorElementType() was also incorrectly mapping
Enum to ELEMENT_TYPE_CLASS.
3. The ReportPointersFromStructInRegisters function was incorrectly
assuming that the false stack entries correspond to stack slots while
they in fact correspond to bytes.
4. Arguments that are passed on stack were incorrectly reported as being
passed in registers (if there were still any registers available)
5. Members of structs embedded in other structs were reporting wrong
offsets (we were missing updating the offset by the delta parameter in
the GcScanValueType when calling GCScanRoots)
6. The code that determines if return values are passed in registers or
on stack was completely missing handling of structs returned in
registers.

* Fix typo in comment

4 years agoMerge pull request #497 from dotnet/31GA-snap
Vivek Mishra [Fri, 10 Jan 2020 23:50:59 +0000 (15:50 -0800)]
Merge pull request #497 from dotnet/31GA-snap

Snap updates for 3.1 GA

4 years agoFixed stage-packages as per comments
Vivek Mishra [Fri, 10 Jan 2020 23:45:04 +0000 (15:45 -0800)]
Fixed stage-packages as per comments

4 years agoUpdating genCodeForBinary to be VEX aware (#1344)
Tanner Gooding [Fri, 10 Jan 2020 22:29:30 +0000 (14:29 -0800)]
Updating genCodeForBinary to be VEX aware (#1344)

* Updating genCodeForBinary to be VEX aware

* Resolving some PR feedback

4 years agoGenerate full platform manifest in official build (#1594)
Davis Goodin [Fri, 10 Jan 2020 19:11:28 +0000 (13:11 -0600)]
Generate full platform manifest in official build (#1594)

* Generate full platform manifest in official build

Generates a full platform manifest in the official build by downloading all platforms' artifacts from the CoreCLR and Libraries jobs for each Installer job.

4 years agoJIT: block local constant propagation of handles when compiling with relocs (#1593)
Andy Ayers [Fri, 10 Jan 2020 18:31:04 +0000 (10:31 -0800)]
JIT: block local constant propagation of handles when compiling with relocs (#1593)

This gives local constant prop the same behavior as value-number-based constant
prop; both now block handle propagation when the jit must generate relocs (R2R),
to ensure that handle references appear only in simple instruction forms like movs.

Prerequisite to #1309, which can enable such propagation.

4 years agoR2RDump improvements w.r.t. GC info reporting (#1539)
Tomáš Rylek [Fri, 10 Jan 2020 17:53:52 +0000 (18:53 +0100)]
R2RDump improvements w.r.t. GC info reporting (#1539)

While I haven't yet managed to nail down the cause of hangs
in the r426654 regression test, I have at least used the opportunity
to improve R2RDump GC info reporting. I have made the reporting
much more compact and more descriptive at the same time by translating
register names and showing stack-relative addresses in a much more
straightforward manner. I'm also proposing to ass a VS solution
for R2RDump - as it now depends on the ILCompiler.Reflection.ReadyToRun
project, it just sucks not to be able to work with both in the VS.

Thanks

Tomas

4 years agoFix and Test case for #27924 (#1059)
Carol Eidt [Fri, 10 Jan 2020 17:09:00 +0000 (09:09 -0800)]
Fix and Test case for #27924 (#1059)

Fix and Test case for #27924

4 years agoAlways build the OpenSSL shim in portable mode on macOS.
Jeremy Barton [Fri, 10 Jan 2020 16:13:53 +0000 (08:13 -0800)]
Always build the OpenSSL shim in portable mode on macOS.

4 years agoMerge pull request #1226 from AaronRobinsonMSFT/sdk_4099
Aaron Robinson [Fri, 10 Jan 2020 02:19:41 +0000 (18:19 -0800)]
Merge pull request #1226 from AaronRobinsonMSFT/sdk_4099

Fix parsing of metadata when looking for COM related CustomAttributes.

4 years agoAnnotate System.Linq.Expressions for nullable (#1064)
buyaa-n [Fri, 10 Jan 2020 01:33:58 +0000 (17:33 -0800)]
Annotate System.Linq.Expressions for nullable (#1064)

* Annotate System.Linq.Expressions for nullable

4 years ago[Arm64] Implement AbsoluteDifference hardware intrinsics (#1334)
Egor Chesakov [Thu, 9 Jan 2020 21:11:05 +0000 (13:11 -0800)]
[Arm64] Implement AbsoluteDifference hardware intrinsics (#1334)

And use VecBinOpTest.template in AbsoluteCompare* tests to remove duplicated logic in validation

4 years agoAdd binder tracing test for ResolveSatelliteAssembly stage (#1455)
Elinor Fung [Thu, 9 Jan 2020 19:30:46 +0000 (11:30 -0800)]
Add binder tracing test for ResolveSatelliteAssembly stage (#1455)

4 years ago[runtime] Arm64: Add TransposeEven and TransposeOdd intrinsics (#1220)
Tamar Christina [Thu, 9 Jan 2020 18:32:17 +0000 (18:32 +0000)]
[runtime] Arm64: Add TransposeEven and TransposeOdd intrinsics (#1220)

4 years agoremove unneeded code (#1521)
Tomas Weinfurt [Thu, 9 Jan 2020 18:00:27 +0000 (10:00 -0800)]
remove unneeded code (#1521)

* check

* cleanup

4 years agoAdd gcstress option to SuperILC (#1446)
Jan Vorlicek [Thu, 9 Jan 2020 17:58:34 +0000 (18:58 +0100)]
Add gcstress option to SuperILC (#1446)

* Add gcstress option to SuperILC

This option allows executing tests with gcstress mode set to a passed
in value.
I have also added additional fixes for race issues with _logWriter.

4 years agoAdd MsQuic implementation of System.Net.Quic (#427)
Justin Kotalik [Thu, 9 Jan 2020 17:55:04 +0000 (09:55 -0800)]
Add MsQuic implementation of System.Net.Quic (#427)

4 years agoFix typo (#1528)
Josh Groves [Thu, 9 Jan 2020 17:49:31 +0000 (14:19 -0330)]
Fix typo (#1528)

4 years agoNullable annotate System.Reflection.Metadata (#1056)
Eirik Tsarpalis [Thu, 9 Jan 2020 17:30:40 +0000 (19:30 +0200)]
Nullable annotate System.Reflection.Metadata (#1056)

* Nullable annotate System.Reflection.Metadata

* remove unused declaration

* update System.Diagnostics.Stacktrace for System.Reflection.Metadata annotations

* address feedback

* reinstate annotations erroneously removed by generator

* Apply feedback

* address feedback

* address feedback

4 years agofix stress script issue (#1452)
Eirik Tsarpalis [Thu, 9 Jan 2020 17:29:38 +0000 (19:29 +0200)]
fix stress script issue (#1452)

4 years agoRename file extension of RunnerTemplate.Windows.txt to .cmd (#1470)
Alexander Köplinger [Thu, 9 Jan 2020 17:17:46 +0000 (18:17 +0100)]
Rename file extension of RunnerTemplate.Windows.txt to .cmd (#1470)

So it automatically gets the EOL treatment from https://github.com/dotnet/runtime/pull/1336.

4 years agoDon't call AdvanceTo(default) in PipeReader.CopyToAsync (#1437)
Stephen Halter [Thu, 9 Jan 2020 14:18:17 +0000 (06:18 -0800)]
Don't call AdvanceTo(default) in PipeReader.CopyToAsync (#1437)

* Don't call AdvanceTo(default) in PipeReader.CopyToAsync

* Address PR feedback

4 years agoRemove BaseDomain::IsSharedDomain (always false now) (#1456)
Elinor Fung [Thu, 9 Jan 2020 14:15:39 +0000 (06:15 -0800)]
Remove BaseDomain::IsSharedDomain (always false now) (#1456)

4 years agoExpose CreateLinkedTokenSource(CancellationToken) (#1379)
Stephen Toub [Thu, 9 Jan 2020 13:56:42 +0000 (08:56 -0500)]
Expose CreateLinkedTokenSource(CancellationToken) (#1379)

4 years agoCoreFX renames (#1440)
Erhan Atesoglu [Thu, 9 Jan 2020 13:41:23 +0000 (05:41 -0800)]
CoreFX renames (#1440)

4 years agoMerge pull request #1348 from stephentoub/moreregexperf
Stephen Toub [Thu, 9 Jan 2020 10:42:17 +0000 (05:42 -0500)]
Merge pull request #1348 from stephentoub/moreregexperf

More Regex perf improvements

4 years agoimprove ALPN handling (#1347)
Tomas Weinfurt [Thu, 9 Jan 2020 07:41:46 +0000 (23:41 -0800)]
improve ALPN handling (#1347)

4 years agoFix fat call transform for x86 (#1451)
Michal Strehovský [Thu, 9 Jan 2020 07:11:21 +0000 (08:11 +0100)]
Fix fat call transform for x86 (#1451)

`AddArgumentToTail` with a null first argument will AV.

I *think* this also works with methods that take a return buffer (methods that return large structs), but might require a pair of eyes more experienced in this codebase.

4 years agoAdd a few more tests / improve code coverage
Stephen Toub [Thu, 9 Jan 2020 02:57:55 +0000 (21:57 -0500)]
Add a few more tests / improve code coverage

4 years agoAddress PR feedback
Stephen Toub [Wed, 8 Jan 2020 20:28:26 +0000 (15:28 -0500)]
Address PR feedback

4 years agoAdditional code gen improvements
Stephen Toub [Wed, 8 Jan 2020 19:36:27 +0000 (14:36 -0500)]
Additional code gen improvements

- In Go, Improve code generation from several constructs to reduce code size and increase bounds reduction in the non-backtracking compiler
- In FindFirstChars, use IndexOf instead of Boyer-Moore if the found prefix is only one character in length.
- Hide some debug-only functions from code coverage

4 years agoRename "Greedy" subexpressions to "Atomic"
Stephen Toub [Mon, 6 Jan 2020 21:43:05 +0000 (16:43 -0500)]
Rename "Greedy" subexpressions to "Atomic"

The greedy naming is really confusing (we should fix it in the framework documentation as well).

In general regular expression nomenclature, "greedy" is used as the opposite of "lazy", indicating how much a loop initially consumes, e.g. does `a*` first try consuming as many 'a's as possible or does it first try consuming as few 'a's as possible... it's "greedy" (denoted as `a*`) if it consumes as many as possible and "lazy" (denoted as `a*?` if it consumes as few.

How aggressively it consumes, however, is orthogonal to whether it backtracks.  Whereas `a*` is greedy and backtracking and `a*?` is lazy and backtracking, `(?>a*)` is greedy and non-backtracking and `(?>a*?) is lazy and non-backtracking.

Unfortunately, the nomenclature in the implementation and the documentation describes the `(?> ... )` as being a "greedy subexpression", which then conflates the meaning of "greedy".

The rest of the industry refers to these instead as "atomic", so I've changed it to that in the implementation.

4 years agoFix CharInClass handling of invariant in lookup table
Stephen Toub [Mon, 6 Jan 2020 20:46:25 +0000 (15:46 -0500)]
Fix CharInClass handling of invariant in lookup table

The Boolean logic was previously wrong, leading to characters being included in the set even if they weren't meant to be, e.g. given the set [^bc], this should include everything other than b and c, but if invariant culture was being used, it incorrectly included B and C in the set.

4 years agoAdd more tests, and fix some bugs found
Stephen Toub [Mon, 6 Jan 2020 19:31:42 +0000 (14:31 -0500)]
Add more tests, and fix some bugs found

4 years agoRestore netfx test configuration to serve as oracle
Stephen Toub [Sun, 5 Jan 2020 20:20:45 +0000 (15:20 -0500)]
Restore netfx test configuration to serve as oracle

With all the changes being made in the regex implementation in this release, we want to make sure we don't regress, and a good way to help doing that is add a bunch of tests and ensure they pass on netfx as well, where appropriate.

4 years agoAdd non-backtracing support for greedy loops and lazyloops
Stephen Toub [Sun, 5 Jan 2020 19:54:32 +0000 (14:54 -0500)]
Add non-backtracing support for greedy loops and lazyloops

4 years agoAuto-greedy for last Alternate/Loop/Lazyloop in regex
Stephen Toub [Sun, 5 Jan 2020 19:06:03 +0000 (14:06 -0500)]
Auto-greedy for last Alternate/Loop/Lazyloop in regex

4 years agoAvoid unnecessary operations when textSpanPos == 0
Stephen Toub [Sun, 5 Jan 2020 19:00:53 +0000 (14:00 -0500)]
Avoid unnecessary operations when textSpanPos == 0

4 years agoAdd non-backtracking go support for anchors, alternation, lookahead, repeaters
Stephen Toub [Sat, 4 Jan 2020 04:10:41 +0000 (23:10 -0500)]
Add non-backtracking go support for anchors, alternation, lookahead, repeaters

4 years agoUse vectorized IndeOf for some notoneloopgreedy loops
Stephen Toub [Sat, 4 Jan 2020 02:08:41 +0000 (21:08 -0500)]
Use vectorized IndeOf for some notoneloopgreedy loops

4 years agoAdd Notoneloopgreedy to go with One/Setloopgreedy
Stephen Toub [Fri, 3 Jan 2020 17:13:59 +0000 (12:13 -0500)]
Add Notoneloopgreedy to go with One/Setloopgreedy

I initially mistakenly thought this wouldn't be useful, but it is.  For example, the expression ".*\n" can be made non-backtracing.

4 years agoAuto-greedy last set/one loop in regex
Stephen Toub [Fri, 3 Jan 2020 16:38:07 +0000 (11:38 -0500)]
Auto-greedy last set/one loop in regex

4 years agoUpgrade some loops followed by boundaries to be greedy
Stephen Toub [Thu, 2 Jan 2020 19:46:42 +0000 (14:46 -0500)]
Upgrade some loops followed by boundaries to be greedy

4 years agoAdd optimized non-backtracking code gen for limited subset of regexes
Stephen Toub [Thu, 2 Jan 2020 01:51:07 +0000 (20:51 -0500)]
Add optimized non-backtracking code gen for limited subset of regexes

When we can determine from the regex that backtracking isn't needed, we
can generate much smaller / better code in RegexCompiler.GenerateGo.

4 years agoImprove StringCode lookup in RegexWriter to avoid unnecessary list
Stephen Toub [Mon, 30 Dec 2019 02:52:30 +0000 (21:52 -0500)]
Improve StringCode lookup in RegexWriter to avoid unnecessary list

4 years agoOptimize allocation for child lists of RegexNodes
Stephen Toub [Sun, 29 Dec 2019 20:26:30 +0000 (15:26 -0500)]
Optimize allocation for child lists of RegexNodes

4 years agoOptimize {Lazy}loops that directly wrap Set/One/Notone
Stephen Toub [Sun, 29 Dec 2019 19:29:51 +0000 (14:29 -0500)]
Optimize {Lazy}loops that directly wrap Set/One/Notone

When written as a* these are already transformed appropriately by the parser, but other reductions can expose them in a way that the parser won't see.  For example, (?:a)* ends up being a much more expensive regex even though it's identical functionally.  This reduces the latter to the former, for cases where a {Lazy}loop is found to directly wrap a Set/One/Notone.

4 years agoAutomatically upgrade some loops to be greedy
Stephen Toub [Thu, 26 Dec 2019 20:25:18 +0000 (15:25 -0500)]
Automatically upgrade some loops to be greedy

In an expression like A*B, backtracing may be involved even if it's impossible that it'll do any good, namely if there's no overlap between the A and B sets.  This backtracking can be avoided by using a greedy subexpression, e.g. (?>A*)B, but a) few developers think to do that, and b) even when they do the implementation has more overhead than is desirable.

This change does three things:
1. It introduces primitives for "oneloopgreedy" and "setloopgreedy" that correspond to the existing "oneloop" (a single character in a loop) and "setloop" (a char class in a loop) primitives.
2. It reduces a Greedy node that contains a one/setloop to instead just be a one/setloopgreedy node.
3. It finds cases of one/setloop concatenated with other things, and automatically updates those to be one/setloopgreedy when it can provie that the backtracking would have no benefit.

(2) removes a small amount of overhead, in particular in compiled, because it avoids needing to spit additional tracking state for backtracking.

(3) can likely be expanded in the future to catch more cases, but it handles the common cases now.  This has a very measurable impact on non-matches that would otherwise trigger backtracking into the subexpression.

4 years agoCode spit EnsureStorage replacement to avoid spilling locals
Stephen Toub [Sun, 22 Dec 2019 03:45:35 +0000 (22:45 -0500)]
Code spit EnsureStorage replacement to avoid spilling locals

Only spill if we have to grow the relevant arrays.

4 years agoClean up minor style issues in RegexCompiler
Stephen Toub [Fri, 20 Dec 2019 16:05:26 +0000 (11:05 -0500)]
Clean up minor style issues in RegexCompiler

4 years agoMinor RegexCompiler codegen tweak for text[pos++]
Stephen Toub [Fri, 20 Dec 2019 15:25:55 +0000 (10:25 -0500)]
Minor RegexCompiler codegen tweak for text[pos++]

The JIT generates slightly better code for:
```C#
text[pos] = ....;
pos++;
```
instead of:
```C#
text[pos++] = ...;
```
so use the IL equivalent of the former.

4 years agoRemove stale generated local in RegexCompiler
Stephen Toub [Wed, 18 Dec 2019 21:41:06 +0000 (16:41 -0500)]
Remove stale generated local in RegexCompiler

4 years agoFix inadvertent closure in EmitCallCharInClass
Stephen Toub [Sat, 28 Dec 2019 02:49:23 +0000 (21:49 -0500)]
Fix inadvertent closure in EmitCallCharInClass

4 years agoAdd EmitCallCharInClass optimization for singleton
Stephen Toub [Mon, 16 Dec 2019 23:29:26 +0000 (18:29 -0500)]
Add EmitCallCharInClass optimization for singleton

4 years agoUse IndexOf{Any} for case-sensitive singletons in FindFirstChar
Stephen Toub [Mon, 16 Dec 2019 23:19:49 +0000 (18:19 -0500)]
Use IndexOf{Any} for case-sensitive singletons in FindFirstChar

Takes advantage of vectorization in string.IndexOf.

4 years agoClean up more code in RegexCompiler
Stephen Toub [Mon, 16 Dec 2019 20:03:36 +0000 (15:03 -0500)]
Clean up more code in RegexCompiler

- Only use ilg from "macro" helpers
- Move some blocks of code around

4 years agoPut back Regex singleton optimization
Stephen Toub [Mon, 16 Dec 2019 19:45:08 +0000 (14:45 -0500)]
Put back Regex singleton optimization

I had removed it because it seemed illogical that anyone would write such character classes.  But it turns out the implementation/parser itself generates them for certain forms, e.g. "hello|hithere" will create a character class containing just 'h' in order to find the start of the alternation.

4 years agoDelay JIT'ing until first execution
Stephen Toub [Tue, 31 Dec 2019 19:43:19 +0000 (14:43 -0500)]
Delay JIT'ing until first execution

I'd previously removed multiple delegate allocations that were being incurred every time we needed to run a compiled regex concurrently.  In doing so, however, I inadvertently forced JIT'ing to occur when the Regex was created rather than on first use; that can in turn lead to more start-up overheads for regexes that are created on startup (e.g. to initialize a static field) but that aren't used at start-up, as well as reduce potential parallelism that could otherwise occur for distinct regexes being run for the first time concurrently.  This addresses the issue by still caching delegates, but in addition to rather than instead of the DynamicMethod objects.

4 years agoRemove dead code in build.ps1 after -help option was added (#1430)
Santiago Fernandez Madero [Thu, 9 Jan 2020 01:32:03 +0000 (17:32 -0800)]
Remove dead code in build.ps1 after -help option was added (#1430)

4 years agoForgot remove a ? from ref assembly (#1467)
buyaa-n [Thu, 9 Jan 2020 01:09:53 +0000 (17:09 -0800)]
Forgot remove a ? from ref assembly (#1467)

4 years agoUpdate superpmi.py helper script (#1474)
Bruce Forstall [Thu, 9 Jan 2020 01:03:01 +0000 (17:03 -0800)]
Update superpmi.py helper script (#1474)

Mostly, updated documentation. Added some additional logging
output for improved clarity. Tried to simplify the code so
it is easier to understand, in places.

4 years ago[master] Update dependencies from dotnet/arcade Microsoft/vstest (#1371)
dotnet-maestro[bot] [Thu, 9 Jan 2020 00:08:17 +0000 (16:08 -0800)]
[master] Update dependencies from dotnet/arcade Microsoft/vstest (#1371)

* Update dependencies from https://github.com/dotnet/arcade build 20200107.1

- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.20057.1
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.20057.1
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.20057.1
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20057.1
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.20057.1
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.20057.1
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.20057.1
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.20057.1
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.20057.1
- Microsoft.DotNet.GenAPI - 5.0.0-beta.20057.1
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.20057.1
- Microsoft.DotNet.GenFacades - 5.0.0-beta.20057.1
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20057.1
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.20057.1

* Update dependencies from https://github.com/dotnet/arcade build 20200107.5

- Microsoft.DotNet.XUnitExtensions - 5.0.0-beta.20057.5
- Microsoft.DotNet.VersionTools.Tasks - 5.0.0-beta.20057.5
- Microsoft.DotNet.ApiCompat - 5.0.0-beta.20057.5
- Microsoft.DotNet.Arcade.Sdk - 5.0.0-beta.20057.5
- Microsoft.DotNet.Build.Tasks.Configuration - 5.0.0-beta.20057.5
- Microsoft.DotNet.Build.Tasks.Feed - 5.0.0-beta.20057.5
- Microsoft.DotNet.Build.Tasks.Packaging - 5.0.0-beta.20057.5
- Microsoft.DotNet.Build.Tasks.SharedFramework.Sdk - 5.0.0-beta.20057.5
- Microsoft.DotNet.CodeAnalysis - 5.0.0-beta.20057.5
- Microsoft.DotNet.GenAPI - 5.0.0-beta.20057.5
- Microsoft.DotNet.XUnitConsoleRunner - 2.5.1-beta.20057.5
- Microsoft.DotNet.GenFacades - 5.0.0-beta.20057.5
- Microsoft.DotNet.Helix.Sdk - 5.0.0-beta.20057.5
- Microsoft.DotNet.RemoteExecutor - 5.0.0-beta.20057.5

* Update dependencies from https://github.com/microsoft/vstest build 20200103-01

- Microsoft.NET.Test.Sdk - 16.5.0-preview-20200103-01

* Fix space in resource key to react to arcade resource generation fix

Co-authored-by: Santiago Fernandez Madero <safern@microsoft.com>
4 years agoMerge pull request #1073 from DrewScoggins/dev/anscoggi/activateLiveBuilds
Drew Scoggins [Wed, 8 Jan 2020 20:01:14 +0000 (12:01 -0800)]
Merge pull request #1073 from DrewScoggins/dev/anscoggi/activateLiveBuilds

Fix passing of liveBuilds name string in perf job.

4 years agoExpose MemoryMarshal.GetArrayDataReference (#1036)
Levi Broderick [Wed, 8 Jan 2020 18:57:59 +0000 (10:57 -0800)]
Expose MemoryMarshal.GetArrayDataReference (#1036)

Given an SzArray, returns a reference to where the element at index 0 would be located, even if the array is empty. No variance checks or bounds checks are performed.

4 years agoImprove RawArrayData comments (#1450)
Next Turn [Wed, 8 Jan 2020 17:58:22 +0000 (01:58 +0800)]
Improve RawArrayData comments (#1450)

4 years agoUpdate URLs in BOTR (#1441)
Adeel Mujahid [Wed, 8 Jan 2020 16:20:19 +0000 (18:20 +0200)]
Update URLs in BOTR (#1441)

- update links to sources from dotnet/coreclr -> dotnet/runtime/src/coreclr
- fix dead urls (github and non-github) under docs/design/coreclr.
- use https://software.intel.com/sites/default/files/article/402129/mpx-linux64-abi.pdf as a new home for x64 System V ABI, as http://x86-64.org is now abandoned. Update source files as well.
- normalize “, ” and ’ to ascii " and ' and remove trailing whitespaces in modified files.

4 years agoDisable failing BasicTest_AccessInstanceProperties_NoExceptions_Bsd test (#1447)
Stephen Toub [Wed, 8 Jan 2020 15:40:03 +0000 (10:40 -0500)]
Disable failing BasicTest_AccessInstanceProperties_NoExceptions_Bsd test (#1447)

4 years agoFix ReflectionTypeLoadException nullable annotations (#545)
Stephen Toub [Wed, 8 Jan 2020 14:09:54 +0000 (09:09 -0500)]
Fix ReflectionTypeLoadException nullable annotations (#545)

We were previously using `Type[]?`, but the array can actually contain null, so it needs to be `Type?[]?`.  Further, upon reviewing a bunch of call sites, it's clear that a) code expects to be able to pass null for the arrays into the ctor but also b) expects to be able to read Types/LoaderExceptions without doing null checks.  As such, I modified the annotations accordingly, and fix the implementation to match.

4 years agoFix nullable annotations on CollectionExtensions.GetValueOrDefault (#1364)
Stephen Toub [Wed, 8 Jan 2020 14:09:28 +0000 (09:09 -0500)]
Fix nullable annotations on CollectionExtensions.GetValueOrDefault (#1364)

4 years agoSuppress ExecutionContext flow in SocketHttpHandler's ConnectHelper (#1381)
Stephen Toub [Wed, 8 Jan 2020 14:09:09 +0000 (09:09 -0500)]
Suppress ExecutionContext flow in SocketHttpHandler's ConnectHelper (#1381)

4 years agoEnsure LocalEndPoint being updated in SendToAsync(SocketAsyncEventArgs) (#808)
Anton Firszov [Wed, 8 Jan 2020 13:46:57 +0000 (14:46 +0100)]
Ensure LocalEndPoint being updated in SendToAsync(SocketAsyncEventArgs) (#808)

Fixes #915 by assigning _rightEndPoint like in other overloads, introducing a breaking change in the behavior of SendToAsync(SocketAsyncEventArgs).

4 years agoAugment several "bool IsXx" APIs with [NotNullWhen(true)] (#1372)
Stephen Toub [Wed, 8 Jan 2020 10:49:14 +0000 (05:49 -0500)]
Augment several "bool IsXx" APIs with [NotNullWhen(true)] (#1372)

If these methods return true, we know the argument passed in wasn't null.  Annotating it as such can help reduce warnings in callers.

4 years agoFix GC heap corruption on ARM. (#1389)
Anton Lapounov [Wed, 8 Jan 2020 10:12:25 +0000 (02:12 -0800)]
Fix GC heap corruption on ARM. (#1389)

The allocate_in_free code path in allocate_in_expanded_heap incorrectly calculated the large (double) alignment padding size when limiting the plug size (SHORT_PLUGS) if set_padding_on_saved_p was true:

    set_padding_in_expand (old_loc, set_padding_on_saved_p, pinned_plug_entry); // Sets the padding flag on the saved plug
    ...
    pad += switch_alignment_size (is_plug_padded (old_loc)); // Reads the padding flag from the old (different!) plug

That caused access violation during a later heap walk since the g_gc_pFreeObjectMethodTable pointer marking the gap was not placed at the right address.

4 years agoMerge pull request #1346 from vargaz/string-ctor-preserve-dep
Zoltan Varga [Wed, 8 Jan 2020 06:17:31 +0000 (07:17 +0100)]
Merge pull request #1346 from vargaz/string-ctor-preserve-dep

Add PreserveDependency attributes to String ctors to mark their dependency on the corresponding String:Ctor methods.

4 years agoFix PreserveDependencyAttribute.cs location.
Zoltan Varga [Wed, 8 Jan 2020 02:22:31 +0000 (03:22 +0100)]
Fix PreserveDependencyAttribute.cs location.

4 years agoFix reported result in binder tracing for mismatched MVID when loading assembly by...
Elinor Fung [Wed, 8 Jan 2020 02:55:09 +0000 (18:55 -0800)]
Fix reported result in binder tracing for mismatched MVID when loading assembly by path / stream (#1339)