usb: chipidea: Fix memleak for ci->hw_bank.regmap when removal