Avoid failures when allocate memory in native (#8028)

Avoid failures when allocate memory in native (#8028)

* Avoid failures when allocate memory in native

using new in the native side can throw and the app will shutdown. instead we detect the failure and throw a managed exception

* Use same mechanism we return the error code from GetSortHandle

* Add assert

* Dispose the safeHandle