Prevent return address hijacking of NativeCallable method

We have seen a failure in the CI where the OnHijackWorkerTripThread was
called in preemptive mode and so a contract in Thread::SetFrame down its
call chain has fired when checking that the thread is in cooperative
mode.
I have found that the issue is caused by hijacking a NativeCallable
method. Such methods switch to preemptive mode before they return and so
when they are hijacked, the OnHijackWorkerTripThread is called in
preemptive mode.

The fix is to prevent return address hijacking for NativeCallable
methods.

diff --git a/src/coreclr/src/vm/threadsuspend.cpp b/src/coreclr/src/vm/threadsuspend.cpp
index ec307e9..2722f36 100644 (file)
--- a/src/coreclr/src/vm/threadsuspend.cpp
+++ b/src/coreclr/src/vm/threadsuspend.cpp
@@ -6091,18 +6091,24 @@ BOOL Thread::HandledJITCase(BOOL ForTaskSwitchIn)
             // the method returns an object reference, so we know whether to protect
             // it or not.
             EECodeInfo codeInfo(ip);
-            VOID *pvHijackAddr = GetHijackAddr(this, &codeInfo);
+
+            GcInfoDecoder gcInfoDecoder(codeInfo.GetGCInfoToken(), DECODE_REVERSE_PINVOKE_VAR);
+
+            if (gcInfoDecoder.GetReversePInvokeFrameStackSlot() == NO_REVERSE_PINVOKE_FRAME)
+            {
+                VOID *pvHijackAddr = GetHijackAddr(this, &codeInfo);
 
 #ifdef FEATURE_ENABLE_GCPOLL
-            // On platforms that support both hijacking and GC polling
-            // decide whether to hijack based on a configuration value.
-            // COMPlus_GCPollType = 1 is the setting that enables hijacking
-            // in GCPOLL enabled builds.
-            EEConfig::GCPollType pollType = g_pConfig->GetGCPollType();
-            if (EEConfig::GCPOLL_TYPE_HIJACK == pollType || EEConfig::GCPOLL_TYPE_DEFAULT == pollType)
+                // On platforms that support both hijacking and GC polling
+                // decide whether to hijack based on a configuration value.
+                // COMPlus_GCPollType = 1 is the setting that enables hijacking
+                // in GCPOLL enabled builds.
+                EEConfig::GCPollType pollType = g_pConfig->GetGCPollType();
+                if (EEConfig::GCPOLL_TYPE_HIJACK == pollType || EEConfig::GCPOLL_TYPE_DEFAULT == pollType)
 #endif // FEATURE_ENABLE_GCPOLL
-            {
-                HijackThread(pvHijackAddr, &esb);
+                {
+                    HijackThread(pvHijackAddr, &esb);
+                }
             }
         }
     }