[scudo] Correct a behavior on the shared TSD registry

Summary:
There is an error in the shared TSD registry logic when looking for a
TSD in the slow path. There is an unlikely event when a TSD's precedence
was 0 after attempting a `tryLock` which indicated that it was grabbed
by another thread in between. We dealt with that case by continuing to
the next iteration, but that meant that the `Index` was not increased
and we ended up trying to lock the same TSD.
This would manifest in heavy contention, and in the end we would still
lock a TSD, but that was a wasted iteration.
So, do not `continue`, just skip the TSD as a potential candidate.

This is in both the standalone & non-standalone versions.

Reviewers: morehouse, eugenis, vitalybuka, hctim

Reviewed By: morehouse

Subscribers: delcypher, #sanitizers, llvm-commits

Tags: #llvm, #sanitizers

Differential Revision: https://reviews.llvm.org/D63783

llvm-svn: 364345

diff --git a/compiler-rt/lib/scudo/scudo_tsd_shared.cpp b/compiler-rt/lib/scudo/scudo_tsd_shared.cpp
index 9918a08beda08f68f0ffb149e196a8815784b987..59ad2549998cccdc4dcba9b35136fff33e59d548 100644 (file)
--- a/compiler-rt/lib/scudo/scudo_tsd_shared.cpp
+++ b/compiler-rt/lib/scudo/scudo_tsd_shared.cpp
@@ -83,9 +83,7 @@ ScudoTSD *getTSDAndLockSlow(ScudoTSD *TSD) {
       }
       const uptr Precedence = TSDs[Index].getPrecedence();
       // A 0 precedence here means another thread just locked this TSD.
-      if (UNLIKELY(Precedence == 0))
-        continue;
-      if (Precedence < LowestPrecedence) {
+      if (Precedence && Precedence < LowestPrecedence) {
         CandidateTSD = &TSDs[Index];
         LowestPrecedence = Precedence;
       }

diff --git a/compiler-rt/lib/scudo/standalone/tsd_shared.h b/compiler-rt/lib/scudo/standalone/tsd_shared.h
index 126d743551bdc63334c6d6e83fac69f24ac1ec25..48747f69f98241398e2fed1262aa84ff642542c8 100644 (file)
--- a/compiler-rt/lib/scudo/standalone/tsd_shared.h
+++ b/compiler-rt/lib/scudo/standalone/tsd_shared.h
@@ -126,9 +126,7 @@ private:
         }
         const uptr Precedence = TSDs[Index].getPrecedence();
         // A 0 precedence here means another thread just locked this TSD.
-        if (UNLIKELY(Precedence == 0))
-          continue;
-        if (Precedence < LowestPrecedence) {
+        if (Precedence && Precedence < LowestPrecedence) {
           CandidateTSD = &TSDs[Index];
           LowestPrecedence = Precedence;
         }