spi: Drop io_mutex in error paths