platform/upstream/f2fs-tools.git
4 years agofsck: Check write pointer consistency of non-open zones
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:30 +0000 (16:59 +0900)]
fsck: Check write pointer consistency of non-open zones

To catch bugs in write pointer handling code for zoned block devices,
have fsck check consistency of write pointers of non-open zones, that
current segments do not point to. Check two items comparing write pointer
positions with valid block maps in SIT.

The first item is check for zones with no valid blocks. When there is no
valid blocks in a zone, the write pointer should be at the start of the
zone. If not, next write operation to the zone will cause unaligned write
error. If write pointer is not at the zone start, reset the zone to move
the write pointer to the zone start.

The second item is check between write pointer position and the last
valid block in the zone. It is unexpected that the last valid block
position is beyond the write pointer. In such a case, report as the bug.
Fix is not required for such zone, because the zone is not selected for
next write operation until the zone get discarded.

In the same manner as the consistency check for current segments, do the
check and fix twice: at the beginning of do_fsck() to avoid unaligned
write error during fsck, and at fsck_verify() to reflect meta data
updates by fsck.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck: Check write pointer consistency of open zones
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:29 +0000 (16:59 +0900)]
fsck: Check write pointer consistency of open zones

On sudden f2fs shutdown, write pointers of zoned block devices can go
further but f2fs meta data keeps current segments at positions before the
write operations. After remounting the f2fs, this inconsistency causes
write operations not at write pointers and "Unaligned write command"
error is reported.

To avoid the error, have f2fs.fsck check consistency of write pointers
of open zones that current segments point to. Compare each current
segment's position and the write pointer position of the open zone. If
inconsistency is found and 'fix_on' flag is set, assign a new zone to the
current segment and check the newly assigned zone has write pointer at
the zone start. Leave the original zone as is to keep data recorded in
it.

To care about fsync data, refer each seg_entry's ckpt_valid_map to get
the last valid block in the zone. If the last valid block is beyond the
current segments position, fsync data exits in the zone. In case fsync
data exists, do not assign a new zone to the current segment not to lose
the fsync data. It is expected that the kernel replay the fsync data and
fix the write pointer inconsistency at mount time.

Also check consistency between write pointer of the zone the current
segment points to with valid block maps of the zone. If the last valid
block is beyond the write pointer position, report to indicate a bug. If
'fix_on' flag is set, assign a new zone to the current segment.

When inconsistencies are found, turn on 'bug_on' flag in fsck_verify() to
ask users to fix them or not. When inconsistencies get fixed, turn on
'force' flag in fsck_verify() to enforce fixes in following checks.

This check and fix is done twice. The first is done at the beginning of
do_fsck() function so that other fixes can reflect the current segment
modification. The second is done in fsck_verify() to reflect updated meta
data by other fixes.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck: Check fsync data always for zoned block devices
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:28 +0000 (16:59 +0900)]
fsck: Check fsync data always for zoned block devices

Fsck checks fsync data when UMOUNT flag is not set. When the f2fs was not
cleanly unmouted, UMOUNT flag is not recorded in meta data and fsync data
can be left in the f2fs. The first fsck run checks fsync data to reflect
it on quota status recovery. After that, fsck writes UMOUNT flag in the
f2fs meta data so that second fsck run can skip fsync data check.

However, fsck for zoned block devices need to care fsync data for all
fsck runs. The first fsck run checks fsync data, then fsck can check
write pointer consistency with fsync data. However, since second fsck run
does not check fsync data, fsck detects write pointer at fsync data end
is not consistent with f2fs meta data. This results in meta data update
by fsck and fsync data gets lost.

To have fsck check fsync data always for zoned block devices, introduce
need_fsync_data_record() helper function which returns boolean to tell
if fsck needs fsync data check or not. For zoned block devices, always
return true. Otherwise, return true if UMOUNT flag is not set in CP.
Replace UMOUNT flag check codes for fsync data with the function call.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck: Introduce move_one_curseg_info() function
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:27 +0000 (16:59 +0900)]
fsck: Introduce move_one_curseg_info() function

When fsck updates one of the current segments, update_curseg_info() is
called specifying a single current segment as its argument. However,
update_curseg_info() calls move_curseg_info() function which updates all
six current segments. Then update_curseg_info() for a single current
segment moves all current segments.

This excessive current segment move causes an issue when a new zone is
assigned to a current segment because of write pointer inconsistency.
Even when a current segment has write pointer inconsistency, all other
current segments should not be moved because they may have fsync data
at their positions.

To avoid the excessive current segment move, introduce
move_one_curseg_info() function which does same work as
move_curseg_info() only for a single current segment. Call
move_one_curseg_info() in place of move_curseg_info() from
update_curseg_info().

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck: Find free zones instead of blocks to assign to current segments
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:26 +0000 (16:59 +0900)]
fsck: Find free zones instead of blocks to assign to current segments

When fsck needs to assign a new area to a curreng segment, it calls
find_next_free_block() function to find a new block to assign. For zoned
block devices, fsck checks write pointer consistency with current
segments' positions. In case a curseg is inconsistent with the
write pointer of the zone it points to, fsck should assign not a new free
block but a new free zone/section with write pointer at the zone start,
so that next write to the current segment succeeds without error.

To extend find_next_free_block() function's capability to find not only
a block but also a zone/section, add new_sec flag to
find_next_free_block() function. When new_sec flag is true, skip check
for each block's availability so that the check is done with unit of
section. Note that it is ensured that one zone has one section for f2fs
on zoned block devices. Then the logic to find a new free section is good
to find a new free zone.

When fsck target devices have ZONED_HM model, set new_sec flag true to
call find_next_free_block() from move_curseg_info(). Set curseg's
alloc_type not SSR but LFS for the devices with ZONED_HM model, because
SSR block allocation is not allowed for zoned block devices. Also skip
relocate_curseg_offset() for the devices with ZONED_HM model for the
same reason.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agolibf2fs_zoned: Introduce f2fs_reset_zone() helper function
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:25 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_reset_zone() helper function

To prepare for write pointer consistency fix by fsck, add
f2fs_reset_zone() helper function which calls RESET ZONE command. The
function is added to lib/libf2fs_zoned which gathers zoned block device
related functions.

When f2fs-tools are built without blkzoned.h kernel header, the helper
function f2fs_reset_zone() prints an error message as other helper
functions in lib/libf2fs_zoned print. To make the message consistent
through the all helper functions, modify message strings in
f2fs_check_zones() and f2fs_reset_zones().

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agolibf2fs_zoned: Introduce f2fs_report_zone() helper function
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:24 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_report_zone() helper function

To prepare for write pointer consistency check by fsck, add
f2fs_report_zone() helper function which calls REPORT ZONE command to
get write pointer status of a single zone. The function is added to
lib/libf2fs_zoned which gathers zoned block device related functions.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agolibf2fs_zoned: Introduce f2fs_report_zones() helper function
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:23 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_report_zones() helper function

To prepare for write pointer consistency check by fsck, add
f2fs_report_zones() helper function which calls REPORT ZONE command to
get write pointer status. The function is added to lib/libf2fs_zoned
which gathers zoned block device related functions.

To check write pointer consistency with f2fs meta data, fsck needs to
refer both of reported zone information and f2fs super block structure
"f2fs_sb_info". However, libf2fs_zoned does not import f2fs_sb_info. To
keep f2fs_sb_info structure out of libf2fs_zoned, provide a callback
function in fsck to f2fs_report_zones() and call it for each zone.

Add SECTOR_SHIFT definition in include/f2fs_fs.h to avoid a magic number
to convert bytes into 512B sectors.

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: check total_segments from devices in raw_super
Qiuyang Sun [Mon, 23 Sep 2019 04:24:59 +0000 (12:24 +0800)]
fsck.f2fs: check total_segments from devices in raw_super

For multi-device F2FS, we should check if the sum of total_segments from
all devices matches segment_count.

Signed-off-by: Qiuyang Sun <sunqiuyang@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agomkfs/Makefile.am: fix build without blkid
Fabrice Fontaine [Sun, 22 Sep 2019 18:41:37 +0000 (20:41 +0200)]
mkfs/Makefile.am: fix build without blkid

Commit d56232bc1640e2a3ffc412faff42cd8e77dbb8dd hardcoded -lblkid in
libf2fs_format_la_LDFLAGS which breaks the build with --without-blkid
option so use ${libblkid_LIBS} instead

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agotools/fibmap: fix build error
Jaegeuk Kim [Mon, 7 Oct 2019 16:31:48 +0000 (09:31 -0700)]
tools/fibmap: fix build error

fibmap.c:28:10: fatal error: 'sys/sysmacros.h' file not found
         ^~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [fibmap.o] Error 1
make[3]: *** Waiting for unfinished jobs....

Fixes: ce64ea0815bf ("f2fs-tools: Add support for Casefolding")
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: add copy command
Eric Biggers [Fri, 4 Oct 2019 22:43:17 +0000 (15:43 -0700)]
f2fs_io: add copy command

Add a copy command to f2fs_io, to allow testing direct I/O writes where
the source page is from a different file (mmap) or an internal kernel
page (sendfile).  It could be useful for other tests in the future too.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: add helper functions for handling errors
Eric Biggers [Fri, 4 Oct 2019 22:43:16 +0000 (15:43 -0700)]
f2fs_io: add helper functions for handling errors

Add and use helper functions for:

- Printing an error message (optionally with errno) and exiting.
- Allocating memory, exiting on error.
- Opening a file, exiting on error.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: release 1.13.0 v1.13.0
Jaegeuk Kim [Wed, 25 Sep 2019 02:20:31 +0000 (19:20 -0700)]
f2fs-tools: release 1.13.0

This release includes:
 - enable VERITY by default on Android
 - introduce some preen mode in fsck.f2fs
 - add f2fs_io tool
 - add casefolding support

And, there are lots of bug fixes.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: fix to skip block allocation for fsynced data
Chao Yu [Wed, 14 Aug 2019 08:48:55 +0000 (16:48 +0800)]
f2fs-tools: fix to skip block allocation for fsynced data

Previously, we don't allow block allocation on unclean umounted image,
result in failing to repair quota system file.

In this patch, we port most recovery codes from kernel to userspace
tools, so that on unclean image, during fsck initialization, we will
record all data/node block address we may recover in kernel, and
then during allocation of quota file repair, we can skip those blocks
to avoid block use conflict.

Eventually, if free space is enough, we can repair the quota system
file on an unclean umounted image.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: remove unnecessary parameter]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: add missing newline symbol in log
Chao Yu [Fri, 9 Aug 2019 10:53:02 +0000 (18:53 +0800)]
f2fs-tools: add missing newline symbol in log

to show pretty log format.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: fix stat update in sload/fsck
Chao Yu [Mon, 5 Aug 2019 09:44:06 +0000 (17:44 +0800)]
f2fs-tools: fix stat update in sload/fsck

Change logic as below:
- fix to account block/node/inode stats correctly in reserve_new_block()
- check overflow in reserve_new_block()
- move stat update from f2fs_alloc_nid() to reserve_new_block()
- adjust write_checkpoint() to update stat for sload/fsck

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix the bug in reserve_new_block
Lihong Kou [Mon, 5 Aug 2019 07:26:21 +0000 (15:26 +0800)]
fsck.f2fs: fix the bug in reserve_new_block

if we new node block in fsck flow, we need to update
the valid_node_cnt at the same time.

Signed-off-by: Lihong Kou <koulihong@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix symlink correctly
Chao Yu [Mon, 12 Aug 2019 11:45:27 +0000 (19:45 +0800)]
fsck.f2fs: fix symlink correctly

inode.i_blocks includes inode, xnode and data block count, so, only
fix in below condition:
- i_blocks := 3 (inode + xnode + data_block)
- i_blocks := 2 (inode + data_block)

In addition, it recovers symlink's i_size to 4k rather than i_blocks *
4k.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix to show removed x_nid correctly
Chao Yu [Mon, 12 Aug 2019 11:45:26 +0000 (19:45 +0800)]
fsck.f2fs: fix to show removed x_nid correctly

Otherwise, we just show fixed zero x_nid value.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix to check c.fix_on before repair
Chao Yu [Mon, 12 Aug 2019 11:45:25 +0000 (19:45 +0800)]
fsck.f2fs: fix to check c.fix_on before repair

We should always set c.bug_on whenever found a bug, then fix them
if c.fix_on is on, otherwise, some bugs won't be shown unless we
enable debug log.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: add get/setflags
Jaegeuk Kim [Sat, 3 Aug 2019 00:02:26 +0000 (17:02 -0700)]
f2fs_io: add get/setflags

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
4 years agofsck.f2fs: fix to propagate error of write_dquots()
Chao Yu [Fri, 9 Aug 2019 10:53:01 +0000 (18:53 +0800)]
fsck.f2fs: fix to propagate error of write_dquots()

Propagate correct error number from write_dquots() to
quota_write_inode().

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: advise to mount unclean image to replay journal
Chao Yu [Fri, 9 Aug 2019 10:53:00 +0000 (18:53 +0800)]
f2fs-tools: advise to mount unclean image to replay journal

For defrag, resize, sload tools, let's advise to mount unclean
image to replay journal first in order to not lose any fsynced
data.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix to set large section type during allocation
Chao Yu [Fri, 9 Aug 2019 10:52:59 +0000 (18:52 +0800)]
fsck.f2fs: fix to set large section type during allocation

During block allocation in large free section, we need to change
all sub segments' type in it, otherwise, we will fail to allocate
block in non-first segment due to mismatch seg-type.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agodump.f2fs: introduce start_bidx_of_node() for cleanup
Chao Yu [Fri, 9 Aug 2019 10:52:58 +0000 (18:52 +0800)]
dump.f2fs: introduce start_bidx_of_node() for cleanup

Just cleanup, no logic change, besides, it can be reused by latter
patch.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: introduce f2fs_ra_meta_pages()
Chao Yu [Fri, 9 Aug 2019 10:52:57 +0000 (18:52 +0800)]
f2fs-tools: introduce f2fs_ra_meta_pages()

Introduce f2fs_ra_meta_pages() to readahead meta pages like we did
in kernel.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: introduce current_sit_addr() for cleanup
Chao Yu [Fri, 9 Aug 2019 10:52:56 +0000 (18:52 +0800)]
fsck.f2fs: introduce current_sit_addr() for cleanup

Just clean up, no logic change.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: allocate memory in batch in build_sit_info()
Chao Yu [Fri, 9 Aug 2019 10:52:55 +0000 (18:52 +0800)]
f2fs-tools: allocate memory in batch in build_sit_info()

Like we did in kernel, allocating memory in batch will be more
efficient.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: fix to avoid memory leak of sit_i->sentries
Chao Yu [Fri, 9 Aug 2019 10:52:54 +0000 (18:52 +0800)]
f2fs-tools: fix to avoid memory leak of sit_i->sentries

We missed to free sit_i->sentries in f2fs_do_umount(), fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: fix potential deadloop
Chao Yu [Fri, 9 Aug 2019 10:52:53 +0000 (18:52 +0800)]
f2fs-tools: fix potential deadloop

In error path of build_sit_info(), start variable is unsigned int type,
it should never be less than zero, fix it.

build_sit_info()
{
...
unsigned int start;
...
free_validity_maps:
for (--start ; start >= 0; --start)
free(sit_i->sentries[start].cur_valid_map);
...
}

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: support defrag_file
Chao Yu [Tue, 6 Aug 2019 02:50:31 +0000 (10:50 +0800)]
f2fs_io: support defrag_file

Support 'defrag_file' sub command to trigger file based defragment via
ioctl in f2fs.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: Add support for Casefolding
Daniel Rosenberg [Thu, 11 Jul 2019 20:45:41 +0000 (13:45 -0700)]
f2fs-tools: Add support for Casefolding

This adds support for f2fs casefolding. Similarly to ext4 casefolding,
this is controlled per-folder via the +F attribute. It can be toggled on
empty directories only. It is not currently compatible with encryption,
but that will likely change.

When enabling the casefold feature, use the -C flag. The format is:
-C encoding[:flag1,flag2,etc]

Signed-off-by: Daniel Rosenberg <drosen@google.com>
[Jaegeuk Kim: print "casefold" in sb->feature]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: fix to recover out-of-border inline size
Chao Yu [Wed, 14 Aug 2019 08:44:26 +0000 (16:44 +0800)]
fsck.f2fs: fix to recover out-of-border inline size

It tries to let fsck be noticed wrong inline size, and do the fix.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agomkfs.f2fs: check zeros in first 16MB for Android
Jaegeuk Kim [Thu, 8 Aug 2019 22:29:25 +0000 (15:29 -0700)]
mkfs.f2fs: check zeros in first 16MB for Android

We actually don't need to issue trim on entire disk by checking first
blocks having zeros.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agosload.f2fs: fix error path of build_directory
Chao Yu [Mon, 5 Aug 2019 01:44:59 +0000 (09:44 +0800)]
sload.f2fs: fix error path of build_directory

The error handling of build_directory is a mess, fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agosload.f2fs: fix memory leak of dentry.link
Chao Yu [Mon, 5 Aug 2019 01:44:58 +0000 (09:44 +0800)]
sload.f2fs: fix memory leak of dentry.link

We forgot to release memory allocated in dentry.link, fix to release
it after last use of it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agosload.f2fs: fix memory leak in build_directory
Xiaojun Wang [Mon, 5 Aug 2019 01:44:57 +0000 (09:44 +0800)]
sload.f2fs: fix memory leak in build_directory

This patch fix bug that variable dentries going out of scope leaks
the storage it points to.

Signed-off-by: Xiaojun Wang <wangxiaojun11@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agolibf2fs: Throw an error when zone dev info not found
Surbhi Palande [Fri, 2 Aug 2019 02:47:48 +0000 (19:47 -0700)]
libf2fs: Throw an error when zone dev info not found

When the -m option is specified to format a Zoned device,
do not fall back to the non-zoned mode in case information
about the device is not found.

Explicitly specify this error to the user.

Signed-off-by: Surbhi Palande <csurbhi@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: include f2fs_fs.h from local directory
Chao Yu [Wed, 31 Jul 2019 13:01:05 +0000 (21:01 +0800)]
fsck.f2fs: include f2fs_fs.h from local directory

f2fs_fs.h locats in local directory rather than system one, fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agosload.f2fs: fix to set temperature during sload
Chao Yu [Wed, 31 Jul 2019 13:00:45 +0000 (21:00 +0800)]
sload.f2fs: fix to set temperature during sload

This patch enable to set file's temperature while loading files
to image.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agodump.f2fs: allow to dump data on mounted device
Chao Yu [Mon, 29 Jul 2019 15:39:36 +0000 (23:39 +0800)]
dump.f2fs: allow to dump data on mounted device

In generic/38[3456], we use quotactl(2) to check if prjquota is
enabled on the given device (src/feature -P $dev in _require_prjquota),
and quotactl(2) requires the given device is a mounted device.

So it requires dump.f2fs to list/check enabled features on a mounted
device, let's relieve to allow such operation.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: write checkpoint with OPU mode
Chao Yu [Wed, 17 Jul 2019 01:28:51 +0000 (09:28 +0800)]
fsck.f2fs: write checkpoint with OPU mode

This original patch was from Weichao Guo.

We may encounter both checkpoints invalid in such a case:
1. kernel writes CP A;
2. power-cut when kernel writes CP B, then CP B is corrupted;
3. fsck: load CP A, fix meta/data;
4. power-cut when fsck writes CP A in-place, then CP A is corrupted too;

To avoid both checkpoints being invalid, this patch changes to duplicate
valid checkpoint to mirror position first, and then, write fixed checkpoint
to CP #0 position.

This can make sure that, while fsck repairing, even there is sudden
power-cut, last valid checkpoint can be kept in CP #1 position.

Signed-off-by: Weichao Guo <guoweichao@huawei.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: add fallocate
Jaegeuk Kim [Fri, 12 Jul 2019 22:25:17 +0000 (15:25 -0700)]
f2fs_io: add fallocate

This simply supports keep_size.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs_io: fix wrong buffered/dio option
Jaegeuk Kim [Fri, 12 Jul 2019 21:20:18 +0000 (14:20 -0700)]
f2fs_io: fix wrong buffered/dio option

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: add more sanity check for superblock
Chao Yu [Thu, 13 Jun 2019 08:10:51 +0000 (16:10 +0800)]
fsck.f2fs: add more sanity check for superblock

Add more sanity check logic for superblock like we did in kernel side.

This fixes bug reported by Seulbae Kim from bugzilla.

https://bugzilla.kernel.org/show_bug.cgi?id=203861

Reproted-by: Seulbae Kim <seulbae@gatech.edu>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: introduce get_checkpoint_version() for cleanup
Chao Yu [Thu, 13 Jun 2019 08:09:03 +0000 (16:09 +0800)]
f2fs-tools: introduce get_checkpoint_version() for cleanup

Just cleanup, no logic change.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: detect corrupted sit journal
Chao Yu [Fri, 24 May 2019 08:52:41 +0000 (16:52 +0800)]
f2fs-tools: detect corrupted sit journal

Enables to detect corrupted sit journal
a) total count of entries.
b) segment no. in entry.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: update checkpoint data atomically
Chao Yu [Fri, 24 May 2019 08:26:40 +0000 (16:26 +0800)]
f2fs-tools: update checkpoint data atomically

This patch keeps line with f2fs kernel module for checkpoint update:
1. writeback all blocks inside checkpoint except last cp_park block;
2. fsync;
3. write last cp_park block;
4. fsync;

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: calculate inode checksum and writeback
Xue Liu [Fri, 24 May 2019 07:28:11 +0000 (15:28 +0800)]
f2fs-tools: calculate inode checksum and writeback

Calculate inode checksum and writeback when processing
sload or resize.

Signed-off-by: Xue Liu <liuxueliu.liu@huawei.com>
[Chao Yu: change commit message and code]
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: Fix up nat journal corruption with -a (auto-fix) option
Sahitya Tummala [Fri, 24 May 2019 06:09:47 +0000 (11:39 +0530)]
fsck.f2fs: Fix up nat journal corruption with -a (auto-fix) option

Build segment manager and node manager always to catch and auto-fix
invalid NAT entries found in the NAT journal.

Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: show elapsed time of full scan
Jaegeuk Kim [Tue, 21 May 2019 20:17:24 +0000 (13:17 -0700)]
fsck.f2fs: show elapsed time of full scan

This patch introduces the elapsed time of fsck.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: Fix device zoned model detection
Shin'ichiro Kawasaki [Mon, 22 Apr 2019 02:04:24 +0000 (11:04 +0900)]
f2fs-tools: Fix device zoned model detection

A partition device does not have the "zoned" nor "chunk_sectors" sysfs
attribute files. Only the owner block device of the partition has these
files. This causes the detection of the zoned model and zone size of a
partition device to fail when executing mkfs.f2fs.

Fix this problem by using the owner device sysfs directory as the base
directory for accessing the zoned and chunk_sectors files. This is done
by using the device major:minor symbolic link under the /sys/dev/block
directory, reading this link and removing the partition device name from
the link path for a partition device (which is indicated by the presence
of the "partition" file under the directory).

Also add a check for the ENOENT error when opening the device "zoned"
sysfs attribute file. The absence of this file indicates that the
kernel does not support zoned block devices. Since the device file is
already open, it exists, and so the device can safely be assumed as not
being zoned.

Changes from v2:
* Addressed Chao Yu's comment on snprintf buffer length

Changes from v1:
* Addressed Chao Yu's comment on ENOENT and return value checks
* Rewrite of sysfs file handling (simplified)
* Rebased on dev-test tree

Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: fix to check total valid block count before block allocation
Chao Yu [Mon, 15 Apr 2019 09:14:38 +0000 (17:14 +0800)]
f2fs-tools: fix to check total valid block count before block allocation

Now, kernel can catch incorrect total valid block count which is exceed
max user block count of image.

Then, generic/051,476 of fstest reports below message:

Apr 15 11:08:03 szvp000201624 kernel: [ 2533.515813] F2FS-fs (zram1): Wrong valid_user_blocks: 469505, user_block_count: 469504
Apr 15 11:08:03 szvp000201624 kernel: [ 2533.519166] F2FS-fs (zram1): Failed to get valid F2FS checkpoint

The reason is that when fsck repairs corrupted quota sysfile, it didn't
check max user block count when allocating new block for quota sysfile,
so ckpt.valid_block_count can exceed max user block count, result in
mount failure later.

Adding upper boundary check of block count in reserve_new_block() to
fix this issue.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agomkfs.f2fs: make the default extensions list much more sensical
Park Ju Hyung [Tue, 16 Apr 2019 06:43:54 +0000 (15:43 +0900)]
mkfs.f2fs: make the default extensions list much more sensical

Following extensions are removed:
 - divx: deprecated video format and it's usually wrapped with avi
 - asf: deprecated streaming format
 - asx: redirecting file to asf(small)
 - wmx: redirecting file to wma/wmv(small)
 - rm: deprecated media container
 - video: unused
 - wv: unpopular audio format from 1998

The extensions list is limited to 64 and those don't deserve to be
on this space-precious list.

Common prefixes are introduced and are checked with
https://en.wikipedia.org/wiki/List_of_filename_extensions
to avoid treating possible hot files as cold:
 - mp: covers mp3, mp4, mpeg, mpg
 - wm: covers wma, wmb, wmv
 - og: covers oga, ogg, ogm, ogv
 - jp: covers jpg, jpeg, jp2

Following extensions are added:
 - webm: extremely popular free media container format from Google
         VP8/VP9/AV1 and Vorbis/Opus is often wrapped with this container
 - wav: uncompressed audio format, commonly used with voice recorders
 - svg: vector image format commonly used in web
 - webp: free lossy image format commonly used in web
 - jar: Java archive file
 - deb: Debian software package
 - iso: disk image file
 - gz: gzip compressed file, unable to randomly update
 - xz: xz compressed file, unable to randomly update
 - zst: zstd compressed file, unable to randomly update
 - pdf: PDF document
 - pyc: Python bytecode automatically generated when
        executing python to run .py files
 - ttc, ttf: font files
 - cnt: image alias files commonly used in Android apps
 - exo: EXO player's cache files, commonly used in Android's YouTube app
 - odex, vdex: Android RunTime files found in /data/app/*/oat

Total entries on the list changed from 34 to 36.

Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: Allow using host-aware devices as regular devices
Damien Le Moal [Tue, 2 Apr 2019 02:44:55 +0000 (11:44 +0900)]
f2fs-tools: Allow using host-aware devices as regular devices

Host-aware zoned block devices can accept random writes anywhere and so
do not require to be handled under F2FS_ZONED_HM mode. Allow host aware
disks to be treated as regular devices if c.zoned_mode is false, that
is, if the -m option is not specified in mkfs.f2fs.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fixed wrong spaces]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: Fix multi-device format with zoned devices
Damien Le Moal [Tue, 2 Apr 2019 02:44:54 +0000 (11:44 +0900)]
f2fs-tools: Fix multi-device format with zoned devices

There is no need to require conventional zones for a zoned block device
that is not the first device of a multi-device volume. As a result,
there is no need to check the number of conventional zones of the
device if the device index is not 0.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agof2fs-tools: Improve zoned model check
Damien Le Moal [Tue, 2 Apr 2019 02:44:53 +0000 (11:44 +0900)]
f2fs-tools: Improve zoned model check

Return an error if an unknown zoned model is reported for a device or
if parsing of the device zoned model fails. Also add comments to
briefly explain the zone models and what to do in the absence of a
kernel reported zoned model for a device.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
[Jaegeuk Kim: Fix one missing function def change]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
4 years agofsck.f2fs: check only max extra_isize
Jaegeuk Kim [Sat, 17 Aug 2019 01:02:28 +0000 (18:02 -0700)]
fsck.f2fs: check only max extra_isize

If we use later kernel having larger extra_isize, old fsck will delete
entire old files.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: update curseg .next_blkoff/.alloc_type preferential
Chao Yu [Thu, 16 May 2019 12:40:43 +0000 (20:40 +0800)]
fsck.f2fs: update curseg .next_blkoff/.alloc_type preferential

If .next_blkoff is inconsistent, we can update curseg .next_blkoff
to first unused block address, and change .alloc_type to SSR
preferential, instead of move curseg to other position.

This can help to repair fuzzed image which has no more free segment.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: introduce check_curseg_offsets()
Chao Yu [Thu, 16 May 2019 12:40:42 +0000 (20:40 +0800)]
fsck.f2fs: introduce check_curseg_offsets()

Let check_curseg_offset() just check one curseg, and introduce
check_curseg_offsets() to check all cursegs, no logic changes.

In addition, update message printed a bit.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: relocate chksum_offset for large_nat_bitmap feature
Chao Yu [Tue, 14 May 2019 09:33:40 +0000 (17:33 +0800)]
f2fs-tools: relocate chksum_offset for large_nat_bitmap feature

For large_nat_bitmap feature, there is a design flaw:

Previous:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----|-------+
| ......                   |      |       |
| checksum_value           |<-----+       |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Obviously, if nat_bitmap size + sit_bitmap size is larger than
MAX_BITMAP_SIZE_IN_CKPT, nat_bitmap or sit_bitmap may overlap
checkpoint checksum's position, once checkpoint() is triggered
from kernel, nat or sit bitmap will be damaged by checksum field.

In order to fix this, let's relocate checksum_value's position
to the head of sit_nat_version_bitmap as below, then nat/sit
bitmap and chksum value update will become safe.

After:

struct f2fs_checkpoint layout:
+--------------------------+  0x0000
| checkpoint_ver           |
| ......                   |
| checksum_offset          |------+
| ......                   |      |
| sit_nat_version_bitmap[] |<-----+
| ......                   |<-------------+
|                          |              |
+--------------------------+  0x1000      |
|                          |      nat_bitmap + sit_bitmap
| payload blocks           |              |
|                          |              |
+--------------------------|<-------------+

Related report and discussion:

https://sourceforge.net/p/linux-f2fs/mailman/message/36642346/

In addition, during writing checkpoint, if large_nat_bitmap feature is
enabled, we need to set CP_LARGE_NAT_BITMAP_FLAG flag in checkpoint.

Reported-by: Park Ju Hyung <qkrwngud825@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: allow unfixed f2fs_checkpoint.checksum_offset
Chao Yu [Tue, 14 May 2019 09:33:39 +0000 (17:33 +0800)]
f2fs-tools: allow unfixed f2fs_checkpoint.checksum_offset

Previously, f2fs_checkpoint.checksum_offset points fixed position of
f2fs_checkpoint structure:

"#define CP_CHKSUM_OFFSET 4092"

It is unnecessary, and it breaks the consecutiveness of nat and sit
bitmap stored across checkpoint park block and payload blocks.

This patch allows f2fs-tools to handle unfixed .checksum_offset.

In addition, for the case checksum value is stored in the middle of
checkpoint park, calculating checksum value with superposition method
like we did for inode_checksum.

In addition, add below change:
- using MAX_BITMAP_SIZE_IN_CKPT to clean up codes.
- introduce verify_checksum_chksum() to verify chksum_{offset,value}

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: fix to repair cp_loads blocks at correct position
Chao Yu [Sun, 19 May 2019 05:05:52 +0000 (13:05 +0800)]
fsck.f2fs: fix to repair cp_loads blocks at correct position

Let's fix the wrong start offset of cp_payload blocks in
fix_checkpoint().

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agosload.f2fs: introduce f2fs_sparse_initialize_meta()
Chao Yu [Mon, 6 May 2019 08:58:06 +0000 (16:58 +0800)]
sload.f2fs: introduce f2fs_sparse_initialize_meta()

This patch fixes to initialize NAT/SIT/CP.payload region in sparse
file mode for sload.

Reported-and-tested-by: zhaowuyun <zhaowuyun@wingtech.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agomkfs.f2fs: write fill chunk in sparse file for zeroed block
Chao Yu [Sun, 5 May 2019 07:45:05 +0000 (15:45 +0800)]
mkfs.f2fs: write fill chunk in sparse file for zeroed block

As zhaowuyun reported:

we met one problem of f2fs, and found one issue of make_f2fs, so I write
this email to search for your help to confirm this issue.

The issue was found on one of Android projects. We use f2fs as the filesystem
of userdata, and make sparse userdata.img using following command, which
invoked in script mkf2fsuserimg.sh
make_f2fs -S $SIZE -f -O encrypt -O quota -O verity $MKFS_OPTS $OUTPUT_FILE

use fastboot to flash this userdata.img to device, and it encountered f2fs
problem and leading to the mount fail of data partition.

we can make this issue 100% persent reproduced by making the data partition
dirty before flashing userdata.img.

suspect that issue is caused by the dirty data in the data partition.
so we checked that source code of make_f2fs in f2fs-tool, found that when
making f2fs, it use dev_fill to do some process:

...

we change code to the following, and the issue is gone.

if (c.sparse_mode)
       return dev_write(buf, offset, len);

Chao Yu:
>
> After checking the codes, IIUC, I guess the problem here is, unlike
> img2simg, mkfs.f2fs won't record zeroed block in sparse image, so
> during transforming to normal image, some critical region like
> NAT/SIT/CP.payload area weren't be zeroed correctly, later kernel may
> load obsoleting data from those region.
>
> Also, The way you provide will obviously increase the size of sparse
> file, since with it we need to write all zeroed blocks of
> NAT/SIT/CP.payload to sparse file, it's not needed.
>
> Not sure, maybe we should use sparse_file_add_fill() to record zeroed
> blocks, so that this will make formatted image more like img2simged one.

Jaegeuk:
> We have to call sparse_file_add_fill() for dev_fill().

This patch fixes to support writing fill chunk sparse file for those
zeroed blocks in mkfs.f2fs.

Reported-and-tested-by: zhaowuyun <zhaowuyun@wingtech.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: get rid of unneeded fields in on-disk inode
Chao Yu [Sun, 28 Apr 2019 09:17:37 +0000 (17:17 +0800)]
f2fs-tools: get rid of unneeded fields in on-disk inode

As Jaegeuk reminded:

Once user updates f2fs-tools which support new fields in inode layout,
but do keep the kernel which can not support those fields, it will cause
old f2fs fail to mount new image due to root_inode's i_extra_isize value
sanity check.

So if f2fs-tools doesn't enable feature which will use new fields of
inode, we don't need to expand i_extra_isize to include them, let's just
let i_extra_isize point to the end of last valid extra field's position.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: improve xattr value printing
Eric Biggers [Wed, 24 Apr 2019 17:57:30 +0000 (10:57 -0700)]
f2fs-tools: improve xattr value printing

- Print the values of xattrs that have an unknown ->e_name_index, rather
  than ignoring them.

- Replace char with u8.  Otherwise xattr values containing bytes >= 0x80
  are printed incorrectly on platforms where char is signed.

- Only parse the encryption xattr if it has a known format number and
  size.  Otherwise print it as hex.

- Remove incorrect le16_to_cpu() on ->e_name_len which has type u8.

- Consolidate the code that prints the xattr value as hex.

- Constify pointers to the xattr entry.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: improve filename printing
Eric Biggers [Wed, 24 Apr 2019 17:59:09 +0000 (10:59 -0700)]
f2fs-tools: improve filename printing

- Make buffers for pretty-printed filenames 341 bytes long, long enough
  for 255 (NAME_MAX) base64-encoded bytes.  Then print encrypted
  filenames in full, base64-encoded.  This will be useful for tests I'm
  writing which verify the correct ciphertext is stored on-disk.

- Rename convert_encrypted_name() to pretty_print_filename(), to make it
  clear that it handles unencrypted names too.  Also make the output
  'char' rather than 'unsigned char', as it's for printing; and remove
  the unnecessary return value.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: open ro disk if we want to check fs only
Jaegeuk Kim [Tue, 16 Apr 2019 18:46:31 +0000 (11:46 -0700)]
fsck.f2fs: open ro disk if we want to check fs only

This patch fixes the "open failure" issue on ro disk, reported by Hagbard.

"
 If I boot with kernel option "ro rootfstype=f2fs
 I get the following halfway trough boot:

  * Checking local filesystems  ...
 Info: Use default preen mode
 Info: Mounted device!
 Info: Check FS only due to RO
         Error: Failed to open the device!
  * Filesystems couldn't be fixed
"

Reported-by: Hagbard Celine <hagbardcelin@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: fix to repair ro mounted device w/ -f
Chao Yu [Tue, 23 Apr 2019 02:42:20 +0000 (10:42 +0800)]
fsck.f2fs: fix to repair ro mounted device w/ -f

As Hagbard Celine reported:

"
Referring to the output from the fsck running against a "ro"
filesystem, especially this line:
Info: Check FS only due to RO

As far as i can tell this says that opposed to other filesystems
running fsck against a "ro" mounted f2fs partition will never fix any
errors.
So I tried running fsck against the same partition mounted "rw":
- mount -o remount,rw /mnt/f2fstest/
- fsck.f2fs  -f /dev/nvme0n1p7
Info: Force to fix corruption
Info: Mounted device!
        Error: Not available on mounted device!

I might be misunderstanding something, but all this tells me that
unless one make a custom initramfs that runs fsck before root is
mounted (something no distributions has, as far as I know), fsck will
never fix an f2fs formatted root partition during boot.
If this is by design and not a bug/unintended behavior, it should be
documented somewhere least more people will experience system crashes
like mine.

All tests above done with kernel 5.0.5 and f2fs-tools 1.12.0 with
"fsck.f2fs: allow to fsck readonly image w/ -f option"-patch by Chao
Yu.
"

We try to make our fsck behavior keeping line with e2fsprogs, but w/
-f option, we can just check a RO mounted device rather repair it, so
let's fix this.

Reported-and-Tested-by: Hagbard Celine <hagbardcelin@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools:fix memory leak in write dquot
Xiaojun Wang [Wed, 24 Apr 2019 06:47:42 +0000 (14:47 +0800)]
f2fs-tools:fix memory leak in write dquot

this patch free ddquot in qtree_write_dquot to avoid memory leak

Signed-off-by: Xiaojun Wang <wangxiaojun11@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: fix to do sanity check with inode.i_inline_xattr_size
Chao Yu [Mon, 4 Mar 2019 09:21:37 +0000 (17:21 +0800)]
fsck.f2fs: fix to do sanity check with inode.i_inline_xattr_size

This patch adds to do sanity check with inode.i_inline_xattr_size,
and once it is corrupted, recover it to default value.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: add missing FIX_MSG during extra_attr field recovery
Chao Yu [Fri, 1 Mar 2019 08:02:27 +0000 (16:02 +0800)]
fsck.f2fs: add missing FIX_MSG during extra_attr field recovery

Add missing printed message during extra_attr field recovery.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck/mount: Gracefully terminate the mount procedure in case the program runs out...
Sotirios-Efstathios Maneas [Thu, 11 Oct 2018 22:21:35 +0000 (18:21 -0400)]
fsck/mount: Gracefully terminate the mount procedure in case the program runs out of memory.

Added the necessary statements so that the mount procedure gracefully terminates in case the program runs out of memory.

Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agolib/libf2fs_io.c: Added a missing free statement.
Sotirios-Efstathios Maneas [Thu, 11 Oct 2018 22:21:14 +0000 (18:21 -0400)]
lib/libf2fs_io.c: Added a missing free statement.

Added a missing free statement related to each device’s path.

Signed-off-by: Sotirios-Efstathios Maneas <smaneas@cs.toronto.edu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: preserve CP_TRIMMED_FLAG
Jaegeuk Kim [Fri, 25 Jan 2019 04:38:10 +0000 (20:38 -0800)]
fsck.f2fs: preserve CP_TRIMMED_FLAG

Let's avoid unnecessary trim.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: fix incorrect boundary of IS_VALID_NID()
Chao Yu [Wed, 9 Jan 2019 02:48:25 +0000 (10:48 +0800)]
fsck.f2fs: fix incorrect boundary of IS_VALID_NID()

nid should never equal to max_nid, fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: check validity of nat journal
Chao Yu [Mon, 14 Jan 2019 01:33:08 +0000 (09:33 +0800)]
fsck.f2fs: check validity of nat journal

As reported by Aravind:

I built f2fs tools from source (at tag v1.12.0) and was able to get this backtrace in gdb:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f8eb54 in f2fs_set_bit (nr=1041170432,
    addr=0x7fffff621010 <error: Cannot access memory at address 0x7fffff621010>) at libf2fs.c:312
312        mask = 1 << (7 - (nr & 0x07));
(gdb) where
    addr=0x7fffff621010 <error: Cannot access memory at address 0x7fffff621010>) at libf2fs.c:312

> [ 5338.040024] nats:8781, sits:6
> [ 5338.040027] F2FS-fs (sda2): Failed to initialize F2FS segment manager
> [ 5338.128893] nats:8781, sits:6
> [ 5338.128895] F2FS-fs (sda2): Failed to initialize F2FS segment manager

nat_count/nid/blkaddr recorded in journal may be corrupted, let's do
sanity check on them, skip loading invalid ones during build_node_manager().

Reported-by: Aravind R S <aravindet@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs_io: exit if system(3) is failed
Jaegeuk Kim [Sat, 23 Mar 2019 10:23:40 +0000 (19:23 +0900)]
f2fs_io: exit if system(3) is failed

Exit the request, if system(3) is failed to be executed.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: Add f2fs_io to .gitignore
Damien Le Moal [Mon, 18 Mar 2019 06:39:56 +0000 (15:39 +0900)]
f2fs-tools: Add f2fs_io to .gitignore

GIT ignore compiled executable tools/f2fs_io/f2fs_io.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: Fix various compilation warnings
Damien Le Moal [Mon, 18 Mar 2019 06:39:55 +0000 (15:39 +0900)]
f2fs-tools: Fix various compilation warnings

Avoid various compilation warnings due to strncpy:

libf2fs.c:590:33: warning: ‘snprintf’ output may be truncated before
the last format character [-Wformat-truncation=]
  snprintf(rootdev, ret, "/dev/%s", buf);

../include/f2fs_fs.h:1384:2: warning: ‘strncpy’ specified bound
depends on the length of the source argument [-Wstringop-overflow=]
  strncpy(buf, features, strlen(features) + 1);

f2fstat.c:243:3: warning: ‘strncpy’ output truncated before
terminating nul copying as many bytes from a string as its length
[-Wstringop-truncation]
   strncpy(ptr_buf, name[i], strlen(name[i]));

This patch does not change any functionality.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: add c++ for UNUSED
Jaegeuk Kim [Fri, 1 Mar 2019 17:47:13 +0000 (09:47 -0800)]
f2fs-tools: add c++ for UNUSED

This can avoid build failure for f2fscrypt.

Change-Id: I33eddc62eea486dc219c25d089112e011864dd4f
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
5 years agof2fscrypt: avoid to fail android build
Jaegeuk Kim [Wed, 16 Jan 2019 02:52:38 +0000 (18:52 -0800)]
f2fscrypt: avoid to fail android build

This fixes some static build errors.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: fix build breakage
Jaegeuk Kim [Wed, 27 Feb 2019 22:29:09 +0000 (14:29 -0800)]
f2fs-tools: fix build breakage

Fix this first, and will do upstream right away.

Change-Id: I6debbb0988595772452f7a878c2cc177fa2e26f3
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
5 years agof2fs_io: add gc_urgent
Jaegeuk Kim [Wed, 20 Feb 2019 03:07:21 +0000 (19:07 -0800)]
f2fs_io: add gc_urgent

e.g.,
f2fs_io gc_urgent dm-4 [start/end/run] [time in sec]

This controls sysfs/gc_urgent to run f2fs_gc urgently.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs_io: add fiemap
Jaegeuk Kim [Wed, 16 Jan 2019 23:17:25 +0000 (15:17 -0800)]
f2fs_io: add fiemap

f2fs_io fiemap [offset in 4kb] [count] [file_path]

Change-Id: Iee9e8b742ada6a12a9f761b3a6de8d82a60e1b59
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
5 years agof2fs_io: add read
Jaegeuk Kim [Wed, 16 Jan 2019 19:15:59 +0000 (11:15 -0800)]
f2fs_io: add read

f2fs_io read [chunk_size in 4kb] [offset in chunk_size] [count] [IO] [print_nbytes] [file_path]

Read data in file_path and print nbytes
IO can be
  buffered : buffered IO
  dio      : direct IO

Change-Id: I912adc4f443c3656ad067d29a1e2f581b79d28e6
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
5 years agof2fs_io: add write
Jaegeuk Kim [Wed, 16 Jan 2019 17:29:42 +0000 (09:29 -0800)]
f2fs_io: add write

f2fs_io write [chunk_size in 4kb] [offset in chunk_size] [count] [pattern] [IO] [file_path]

Write given patten data in file_path
pattern can be
  zero     : zeros
  inc_num  : incrementing numbers
  rand     : random numbers
IO can be
  buffered : buffered IO
  dio      : direct IO

Change-Id: Icc866d5b9933423639d1d0e4d5e556ddf4f15feb
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
5 years agofsck.f2fs: check nat_bits separately from bug_on
Jaegeuk Kim [Wed, 28 Nov 2018 23:11:28 +0000 (15:11 -0800)]
fsck.f2fs: check nat_bits separately from bug_on

It checks nat_bits separately, and enable it seamlessly.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: don't ask anything on dry-run
Jaegeuk Kim [Fri, 4 Jan 2019 21:02:22 +0000 (13:02 -0800)]
fsck.f2fs: don't ask anything on dry-run

It doesn't need to ask recovering corruption on dry-run mode.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs_io: add pin_file control command
Jaegeuk Kim [Fri, 4 Jan 2019 00:54:04 +0000 (16:54 -0800)]
f2fs_io: add pin_file control command

This requires a kernel patch "f2fs: export pin_file flag to user".

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agodump.f2fs: fix incorrect endian conversion
Chao Yu [Mon, 10 Dec 2018 03:19:01 +0000 (11:19 +0800)]
dump.f2fs: fix incorrect endian conversion

f2fs_inode.i_mode is variable of __le16 type, during endian converison,
it should to use le16_to_cpu instead of le32_to_cpu, fix it.

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck: Fix big endian platforms
Rosen Penev [Sat, 8 Dec 2018 01:45:31 +0000 (17:45 -0800)]
fsck: Fix big endian platforms

Since "fsck.f2fs: integrate sanity_check_inode to __check_inode_mode",
big endian platforms have been broken due to an endianness bug.

More info: https://github.com/openwrt/openwrt/pull/1575

Signed-off-by: Rosen Penev <rosenp@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agotools: add f2fs_io
Jaegeuk Kim [Wed, 28 Nov 2018 18:52:05 +0000 (10:52 -0800)]
tools: add f2fs_io

This is initial commit to support shutdown ioctl.

Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agoparse.f2fs: fix parsing of f2fs_trace_ios
Sahitya Tummala [Fri, 28 Dec 2018 07:36:43 +0000 (13:06 +0530)]
parse.f2fs: fix parsing of f2fs_trace_ios

It is incorrectly taking blkaddr as len because it missed to parse
op_flags in the trace print of f2fs_trace_ios.

Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: introduce preen mode 2
Chao Yu [Tue, 27 Nov 2018 12:36:35 +0000 (20:36 +0800)]
fsck.f2fs: introduce preen mode 2

As Michael reported:

after updating to f2fs-tools 1.12.0, a routine fsck of my file systems
took quite a while and output ten-thousands instances of the following
line:

> [FIX] (fsck_chk_inode_blk: 954)  --> Regular: 0xXYZ reset i_gc_failures from 0x1 to 0x00

The reason is that:

We start to support reseting .i_gc_failures's value to zero in fsck since
91bb7b21f740 ("f2fs-tools: fix to reset i_gc_failures offline"), this is
because if .i_gc_failures continues increasing and exceed threshold, it
can make f2fs break atomic_write semantics during GC, so I added that
patch to avoid such condition.

But the problem here is even .i_gc_failures's value is one which was
initialized duing inode creation by old kernel, and it never be increased
by GC flow, we will still trigger such fix in fsck. I think it's not
necessary.

As Jaegeuk suggested, we introduce a new preen mode 2 to handle such
condition:
  - 2: same as 0, but skip some checks for old kernel

Suggested-by: Jaegeuk Kim <jaegeuk@kernel.org>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agofsck.f2fs: allow to fsck readonly image w/ -f option
Chao Yu [Mon, 26 Nov 2018 10:53:37 +0000 (18:53 +0800)]
fsck.f2fs: allow to fsck readonly image w/ -f option

To keep line with e2fsprogs, let's allow to fsck mounted image as
readonly w/ -f option.

Reported-by: Perfect Gentleman <perfect007gentleman@gmail.com>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agof2fs-tools: fix to check return value of {c,m}alloc()
Chao Yu [Mon, 26 Nov 2018 13:36:00 +0000 (21:36 +0800)]
f2fs-tools: fix to check return value of {c,m}alloc()

It needs to fix to handle error case of {c,m}alloc().

Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
5 years agolibf2fs: avoid to use static local string for global f2fs_conf
Jaegeuk Kim [Thu, 22 Nov 2018 11:57:06 +0000 (03:57 -0800)]
libf2fs: avoid to use static local string for global f2fs_conf

We don't need to use c.rootdev_name, but do it locally.

Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>