Fix OSX floating point state extraction (#25295)
There was a bug reported on a very recent Mac with Intel i9 processor. A
crash in the RtlRestoreContext was happening at the fxrstor instruction
due to the fact that the floating point state data were garbage.
The investigation has shown that sometimes, the x86_FLOAT_STATE64
cannot be obtained using the thread_get_state API. And it was also found
that at the same time, the x86_AVX_STATE64 can be obtained. The state
extracted by the AVX variant contains all the registers that the FLOAT
variant would extract.
However, in some cases, even the x86_AVX_STATE64 cannot be obtained and
there is a third flavor that we can get - x86_AVX512_STATE64.
Unfortunately, there are cases where none of those can be obtained.
It is not clear what causes these cases, it seems only kernel debugging
can give us an answer to that.
This change modifies the way we extract the floating point state. We
first try to get the AVX state, if we fail, we try the AVX512 and
finally we fall back to the FLOAT state. If we fail to get the floating
point state with any of these, we return context without the floating
point state flag set. Also, if only getting the FLOAT state succeeds,
we return context without the XSTATE flag set.