spi: img-spfi: fix potential double release