ip-conflict: Don't remove already removed timer