Some cleanup for ArrayPool trimming (#17518)