Don't leak ref to native handle in external memory tests