Btrfs: deal with enomem in the rewind path