Correctly marshal structure return values in member functions on Win-x64 and Win...
authorJeremy Koritzinsky <jkoritzinsky@gmail.com>
Fri, 29 Mar 2019 16:49:55 +0000 (09:49 -0700)
committerGitHub <noreply@github.com>
Fri, 29 Mar 2019 16:49:55 +0000 (09:49 -0700)
commit6b889aba05b8e1ccc9cef793a2d1293e30598453
tree14ff0a7b99564cf98dddfe67c4a3302f2d5c926e
parentcda8f7bf0a826878fc6ca6b9d7bc6f6011626cc9
Correctly marshal structure return values in member functions on Win-x64 and Win-x86 (#23145)

* In Windows-x64, if we have a native member function signature with a struct return type, we need to do a by-ref return.

* Implement support for marshalling structure return values via a return buffer argument correctly in instance signatures on AMD64-Windows.

* Change field initialization ordering to satisfy warning.

* Try to narrow down the conditions that trigger these changes to just COM methods.

* Don't bash the return type on AMD64 Windows. Only treat it as a byref return buffer.

* PR feedback.

* Enable returning structs from COM methods via a return buffer on x86 for structs <= 8 bytes.

* Add test for struct returns with ThisCall. Extend the "struct return buffer" fix to functions marked as unmanaged thiscall since they all must be instance methods

* Don't include the return-type-bashing switch on AMD64 platforms.

* Don't do the signature swapping/copy on non-instance functions with struct returns.

* Cast the return type of GetStubTargetCallingConv to the right calling convention enum type.

* If we're doing a thiscall, marshal the "this" parameter before the return buffer (if the return buffer exists) on all platforms.

* Remove temporary logging code I added in for debugging.

* Clean up class naming.

* Try using a vtable instead of a pointer-to-member-function.

* Remove delete of class with non-virtual destructor
16 files changed:
src/vm/dllimport.cpp
src/vm/dllimport.h
src/vm/ilmarshalers.h
src/vm/stubgen.cpp
src/vm/stubgen.h
tests/src/Interop/CMakeLists.txt
tests/src/Interop/COM/NETClients/Primitives/NumericTests.cs
tests/src/Interop/COM/NETServer/NumericTesting.cs
tests/src/Interop/COM/NativeClients/Primitives/NumericTests.cpp
tests/src/Interop/COM/NativeServer/NumericTesting.h
tests/src/Interop/COM/ServerContracts/Server.Contracts.cs
tests/src/Interop/COM/ServerContracts/Server.Contracts.h
tests/src/Interop/PInvoke/Miscellaneous/ThisCall/CMakeLists.txt [new file with mode: 0644]
tests/src/Interop/PInvoke/Miscellaneous/ThisCall/ThisCallNative.cpp [new file with mode: 0644]
tests/src/Interop/PInvoke/Miscellaneous/ThisCall/ThisCallTest.cs [new file with mode: 0644]
tests/src/Interop/PInvoke/Miscellaneous/ThisCall/ThisCallTest.csproj [new file with mode: 0644]