provider: Don't remove a provider on error