[Mono][jit] Emit GC transitions for "calli unmanaged" instructions (#46491)
* [metadata] Treat CallConv bit 0x09 as MONO_CALL_UNMANAGED_MD
This is the C#9 function pointers "unmanaged"+ext calling convention.
Additional calling convention details are encoded in the modopts of the return
type.
This PR doesn't handle the modopts yet
* [marshal] Add mono_marshal_get_native_func_wrapper_indirect
This will be used to add a wrapper around "calli sig" indirect calls where "sig" has
an unmanaged calling convention.
We need to do a GC transition before calling an unmanaged function from
managed. So this wrapper does it.
This is reusing much of the code implemented for
mono_marshal_get_native_func_wrapper_aot which is used for delegates.
Unfortunately that means that the function pointer is (pointlessly) boxed when
it is passed as the first argument.
* [jit] Use a wrapper for "calli unmanaged" instructions
If there's a calli with an unmanaged signature, invoke a wrapper that
does a transition to GC Safe mode and then do the call.
The wrapper is always inlined into the callee.
Because we're reusing much of the code of
mono_marshal_get_native_func_wrapper_aot, the function pointer first has to be
boxed before it's passed to the wrapper. In theory we should be able to just
pass it directly and get much simpler code. But that will require changing the
code in emit_native_wrapper_ilgen to get an unboxed function pointer arg
* fixup don't emit GC transitions in runtime invoke wrapper
* fixup don't emit two wrappers on dynamic methods
* check that calli wrapper only gets blittable args
* negate logic for when to add an indirection wrapper
assume that if the callee is a wrapper, it doesn't need the transition, and
only add it to normal managed methods.
* add disabled debug printf
* Add MonoNativeWrapperFlags arg to emit_native_wrapper
Replace the 4 boolean args by a flags arg.
Also add a new EMIT_NATIVE_WRAPPER_FUNC_PARAM_UNBOXED and use it in
mono_marshal_get_native_func_wrapper_indirect. The new flag has no effect yet.
* Address review feedback
* Use an unboxed func ptr mono_marshal_get_native_func_wrapper_indirect
Add support for the EMIT_NATIVE_WRAPPER_FUNC_PARAM_UNBOXED flag to mono_marshal_emit_native_wrapper