i2c: rcar: avoid race when unregistering slave client