mlxsw: spectrum_router: Rollback virtual router adjacency pointer update