spi: imx: Don't skip cleanup in remove's error path