Fix regression by previous optimization for mDirtyFlags