Jaegeuk Kim [Fri, 2 Apr 2021 03:13:55 +0000 (20:13 -0700)]
resize.f2fs: fix wrong ovp calculation
beroal reported a mount failure due to broken valid_user_blocks.
[ 6890.647749] F2FS-fs (loop0): Wrong valid_user_blocks:
16040048,
user_block_count:
10016768
From fsck,
segment_count_main [0x 9a95 : 39573]
-> 39573 * 2MB = 78GB as user space
overprov_segment_count [0x 4e29 : 20009]
-> 20009 * 2MB = 40GB as overprovisioned space which user can't see.
But,
[FSCK] valid_block_count matching with CP [Ok..] [0xf4c070]
-> 0xf4c070 =
16040048
valid_block_count [0x f4c070 :
16040048]
-> So, this is correct.
It turns out resize.f2fs gave very large and wrong overprovisioning space
result in shortage of user blocks. The root cause was f2fs_get_usable_segments()
didn't consider resize case which needs segment_count_main from new superblock.
Fixes:
f8410857b7a8 ("f2fs-tools: zns zone-capacity support")
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daniel Abrecht [Mon, 15 Mar 2021 22:08:53 +0000 (22:08 +0000)]
Add -P option to preserve file owner
I use user namespaces & debootstrap to bootstrap a system, and then
copy that into an image using sload.f2fs. However, without an option to
preserve the owner user & group of files, some files end up being owned
by the wrong user / group (0:0), which causes all kinds of other issues
This patch adds an option -P to preserve the user and group of files.
Signed-off-by: Daniel Abrecht <public@danielabrecht.ch>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Junyong Sun [Tue, 2 Mar 2021 03:21:55 +0000 (11:21 +0800)]
libf2fs: fix memory leak caused by get_rootdev()
memory which malloc for uevent don't free at get_rootdev(),
it would cause memory leak, so free it before return.
Signed-off-by: Junyong Sun <sunjy516@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Park Ju Hyung [Sun, 14 Feb 2021 09:43:11 +0000 (18:43 +0900)]
mkfs.f2fs: add VM disk files to hot data types
Similar to .db files, these are randomly updated extremely frequently.
Signed-off-by: Park Ju Hyung <qkrwngud825@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 28 Dec 2020 18:41:09 +0000 (10:41 -0800)]
libzoned: use blk_zone_v2 and blk_zone_report_v2 by default
Old blk_zone and blk_zone_report should be compatible with v2.
And, old blk_zone_report uses reserved[4] which breaks the build.
struct blk_zone_report {
__u64 sector;
__u32 nr_zones;
__u8 reserved[4];
struct blk_zone zones[0];
};
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 23 Dec 2020 18:10:27 +0000 (10:10 -0800)]
f2fs-tools: fix wrong blk_zone_rep_v2 definition
If HAVE_BLK_ZONE_REP_V2 id defined, we should set v2.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 17 Dec 2020 18:12:59 +0000 (10:12 -0800)]
mkfs.f2fs: allocate zones together to avoid random access
This patch allocates zones to initial logs together, if it's on zoned device.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 17 Dec 2020 17:52:01 +0000 (09:52 -0800)]
mkfs.f2fs: adjust zone alignment when using multi-partitions
When formatting conventional partition with zoned one, we should align
the starting block address of next device to the zone size.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 17 Dec 2020 16:39:01 +0000 (08:39 -0800)]
fsck.f2fs: fix alignment on multi-partition support
- mkfs.f2fs -s 4 -c second_dev first_dev
- fsck.f2fs first
Info: Segments per section = 1
Info: Sections per zone = 1
Info: sector size = 512
Info: total sectors =
69668488 (34017 MB)
Segment count (19128) mismatch with total segments from devices (19130) Can't find a valid F2FS superblock at 0x0
Segment count (19128) mismatch with total segments from devices (19130) Can't find a valid F2FS superblock at 0x1
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
John A. Leuenhagen [Thu, 17 Dec 2020 09:14:57 +0000 (04:14 -0500)]
f2fs-tools: Miscellaneous cleanup to README.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
John A. Leuenhagen via Linux-f2fs-devel [Thu, 17 Dec 2020 07:57:44 +0000 (02:57 -0500)]
mkfs.f2fs.8: Better document the -g argument.
To use Android defaults, you must use `-g android`. However, the man
page previously implied that you need only use `-g`.
Since you must specify `android`, I figured it would be appropriate
to name that field `default-options` in the man page. If there was ever
a reason to have a different set of default options, the `-g` option
could be used with a new `default-options` value, i.e. `-g example`.
Signed-off-by: John A. Leuenhagen <john@zlima12.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
John A. Leuenhagen via Linux-f2fs-devel [Sat, 12 Dec 2020 01:39:23 +0000 (20:39 -0500)]
mkfs.f2fs.8: fix formatting for -l parameter in man page
Signed-off-by: John A. Leuenhagen <john@zlima12.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jordan Webb [Thu, 10 Dec 2020 15:28:11 +0000 (09:28 -0600)]
f2fs-tools: Make sload.f2fs reproduce hard links
If sload.f2fs encounters a file with nr_links > 1, it will mark it
as a possible hard link by remembering the original device and
inode. When sload.f2fs creates the file, it will check if it has
already created a file for the same original device and inode. If
so, it will add the original inode to the directory and increment
the number of links to it, instead of writing a new inode.
This allows sload.f2fs to accurately reproduce a directory tree that
contains hard links, such as those created by ostree. Without this
patch, directory trees containing hard links result in the content of
the files being duplicated.
This is version 2 of the patch; it has been rebased against the dev
branch and includes a fix from Jaegeuk Kim to avoid building data
contents twice on hard linked files.
Co-authored-by: Jaegeuk Kim <jaegeuk@kernel.org>
Signed-off-by: Jordan Webb <jordan@getseam.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Tue, 8 Dec 2020 08:15:54 +0000 (16:15 +0800)]
f2fs-tools:sload.f2fs compression support
Add F2FS compression support for sload
* Support file extension filter, either default-accept or default-deny
policy
* Support choice of compression algorithm, LZO (version 2) or LZ4
(default)
* Support custom log of cluster size
* Support minimum number of compressed blocks per cluster (default 1).
A cluster will not be compressed if the number can not be met.
* suuport -r (read-only) option
This releases compressed blocks to secure free space in advance. Note that,
all compressed files will have the immutable bit.
* Added manpage update
* Remove unecessary qbuf allocation (Jaegeuk, suggested by Satya)
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: fix some bugs and refactor names]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Tue, 8 Dec 2020 01:19:10 +0000 (10:19 +0900)]
f2fs_io: add compress/decompress commands
Added new commands, compress and decompress, to support
F2FS_IOC_COMPRESS_FILE and F2FS_IOC_DECOMPRESS_FILE.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Wed, 2 Dec 2020 00:57:22 +0000 (08:57 +0800)]
f2fs-tools: Added #ifdef WITH_func
Add proprocessor defines (options) 'WITH_func',
where func = DUMP, DEFRAG, RESIZE, or SLOAD
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Victor Westerhuis [Tue, 17 Nov 2020 16:44:55 +0000 (08:44 -0800)]
f2fs-tools: fix a few spelling errors in f2fs-tools
typo fix.
Signed-off-by: Victor Westerhuis <victor@westerhu.is>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Wed, 11 Nov 2020 09:40:57 +0000 (17:40 +0800)]
f2fs-tools: skipped to end on error syntax error
When error were found, we won't need to do any initialization but
just quit.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 9 Nov 2020 18:03:06 +0000 (10:03 -0800)]
mkfs.f2fs: show a message when compression is enabled
This patch adds a message when formatting the disk with compression.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Tue, 3 Nov 2020 23:36:14 +0000 (08:36 +0900)]
f2fs_io: add get/set compression option
Added new commands, get_coption and set_coption, to support
F2FS_IOC_GET_COMPRESS_OPTION and F2FS_IOC_SET_COMPRESS_OPTION.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Wed, 4 Nov 2020 03:13:08 +0000 (11:13 +0800)]
Fix ASSERT() macro with '%' in the expression
Fix a compiling error triggered by ASSERT(exp), when exp contains '%'
(e.g. integer modular operator)
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Mon, 26 Oct 2020 09:40:00 +0000 (17:40 +0800)]
f2fs-toos: fsck.f2fs Fix bad return value
'ret' should not have been used here: otherwise, it would be wrongly used
as the error code and then be returned from main().
Signed-off-by: Robin Hsu <robinhsu@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Sun, 1 Nov 2020 12:53:31 +0000 (20:53 +0800)]
fsck.f2fs: do xnid sanity check only during fsck
As Eric reported:
Commit
7a22451bc2b4 ("fsck.f2fs: fix to check validation of i_xattr_nid")
This commit caused a regression where 'dump.f2fs -i <inode> <device>'
now segfaults if the inode has any extended attributes.
It's because read_all_xattrs() now calls fsck_sanity_check_nid(), which
eventually dereferences f2fs_fsck::main_area_bitmap, which is NULL.
I'm not sure what was intended here.
Here's the output from gdb:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7f750fa in f2fs_test_bit (nr=1024, p=0x0) at libf2fs.c:304
304 return (mask & *addr) != 0;
(gdb) bt
ntype=TYPE_XATTR, ni=0x7fffffffdd20) at fsck.c:449
ntype=TYPE_XATTR, ni=0x7fffffffdd20) at fsck.c:495
fsck_sanity_check_nid() should only called from fsck.f2fs context, rather
than dump.f2fs, otherwise it may cause dereferencing structure fields of
fsck incorrectly.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Ocean Chen [Tue, 13 Oct 2020 17:22:40 +0000 (01:22 +0800)]
f2fs_io: add erase option
f2fs_io erase [block_device_path]
Signed-off-by: Ocean Chen <oceanchen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Eric Biggers [Fri, 9 Oct 2020 22:20:56 +0000 (15:20 -0700)]
mkfs.f2fs.8: document the verity feature
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Eric Biggers [Fri, 9 Oct 2020 04:07:45 +0000 (21:07 -0700)]
fsck: clear unexpected casefold flags
Inodes aren't allowed to have the casefold flag set when they aren't
directories, or if the filesystem superblock doesn't have the casefold
feature enabled. Clear any such unexpected casefold flags.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
John A. Leuenhagen [Sun, 4 Oct 2020 18:08:35 +0000 (14:08 -0400)]
mkfs.f2fs: add -h and --help
These two arguments would simply print the usage message.
Signed-off-by: John A. Leuenhagen <john@zlima12.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Thu, 10 Sep 2020 05:29:48 +0000 (14:29 +0900)]
f2fs_io: change fibmap to fiemap
Currently we support fiemap command using fibmap. It's simple and
easy to use, but we cannot use this for compressed file. To support
more different types of files, we need to change this to use fiemap.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: add Android build]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 24 Aug 2020 21:55:54 +0000 (14:55 -0700)]
f2fs-tools: release 1.14.0
This release includes mainly, but not limited to:
- add IO cache to speed up fsck.f2fs run
- support casefolding
- support data compression
- support zns zone-capacity
- enhance fsck.f2fs for zoned device
- enhance f2fs_io tool
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Theotime Combes [Tue, 18 Aug 2020 11:18:07 +0000 (11:18 +0000)]
mkfs.f2fs: add -r (fake_seed) flag
r flag sets the checkpointing seed to 0 (initially used to
remove randomness for apex generation)
Signed-off-by: Theotime Combes <tcombes@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Theotime Combes [Tue, 18 Aug 2020 11:18:50 +0000 (11:18 +0000)]
mkfs.f2fs: add -T flag
T flag sets timestamps to a given value
Signed-off-by: Theotime Combes <tcombes@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Mon, 17 Aug 2020 05:01:30 +0000 (14:01 +0900)]
sload.f2fs: handle root mount point properly when setting file attribute
Need to remove "/" of mount point name from the file path name
when mount point is "/". Otherwise, we will transfer file path
name whose first two characters are like "//" to fs_config function.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Aravind Ramesh [Tue, 21 Jul 2020 16:38:11 +0000 (22:08 +0530)]
f2fs-tools: zns zone-capacity support
NVM Express Zoned Namespace (ZNS) devices can have zone-capacity(zc) less
than the zone-size. ZNS defines a per zone capacity which can be equal
or less than the zone-size. Zone-capacity is the number of usable blocks
in the zone. If zone-capacity is less than zone-size, then the segments
which start at/after zone-capacity are considered unusable. Only those
segments which start before the zone-capacity are considered as usable
and added to the free_segment_count and free_segment_bitmap of the kernel.
In such cases, the filesystem should not write/read beyond the
zone-capacity.
Update the super block with the usable number of blocks and free segment
count in the ZNS device zones, if zone-capacity is less than zone-size.
Set reserved segment count and overprovision ratio based on the usable
segments in the zone.
Allow fsck to find the free_segment_count based on the zone-capacity and
compare with checkpoint values.
Signed-off-by: Aravind Ramesh <aravind.ramesh@wdc.com>
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
[Jaegeuk Kim: add UNUSED to is_usable_seg()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 13 Aug 2020 19:46:29 +0000 (12:46 -0700)]
f2fs_io: measure performance of write()
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 14 Aug 2020 01:11:22 +0000 (18:11 -0700)]
f2fs_io: add OSYNC option
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 7 Aug 2020 02:02:31 +0000 (10:02 +0800)]
fsck.f2fs: correct return value
As Norbert Lange reported:
"
$ fsck.f2fs -a /dev/mmcblk0p5; echo $?
Info: Fix the reported corruption.
Info: Mounted device!
Info: Check FS only on RO mounted device
Error: Failed to open the device!
255
"
Michael Laß reminds:
"
I think the return value is exactly the problem here. See fsck(8) (
https://linux.die.net/man/8/fsck) which specifies the return values.
Systemd looks at these and decides how to proceed:
https://github.com/systemd/systemd/blob/
a859abf062cef1511e4879c4ee39c6036ebeaec8/src/fsck/fsck.c#L407
That means, if fsck.f2fs returns 255, then
the FSCK_SYSTEM_SHOULD_REBOOT bit is set and systemd will reboot.
"
So the problem here is fsck.f2fs didn't return correct value to userspace
apps, result in later unexpected behavior of rebooting, let's fix this.
Reported-by: Norbert Lange <nolange79@gmail.com>
Reported-by: Michael Laß <bevan@bi-co.net>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
[Jaegeuk Kim: fix FSCK_ERROR_CORRECTED]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Thu, 30 Jul 2020 06:29:27 +0000 (15:29 +0900)]
f2fs_io: open file in read only mode for pinning file
To use "pinfile" command even with read only file, changed open
mode for it.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 18 Jul 2020 03:10:21 +0000 (20:10 -0700)]
mkfs.f2fs: add casefolding and project quota config
This can be used for Android build support.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Sat, 18 Jul 2020 02:58:22 +0000 (19:58 -0700)]
mkfs.f2fs: should initialize sparse file in Windows
Otherwise it fails to format in Windows.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 17 Jul 2020 01:04:43 +0000 (18:04 -0700)]
f2fs_io: add mmap read operation
This patch adds an option, mmap, when reading data.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Theotime Combes [Wed, 15 Jul 2020 19:08:05 +0000 (19:08 +0000)]
fsck.f2fs: avoid randomness in sum_blk
This patch removes random bytes in sum_blk when loading a new directory.
Signed-off-by: Theotime Combes <tcombes@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 14 Jul 2020 05:59:41 +0000 (22:59 -0700)]
dump.f2fs: check block address first before reading the block
This should avoid to read out wrong block address.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 30 Jun 2020 21:41:38 +0000 (14:41 -0700)]
sload.f2fs: fix missing fsck_init
This patch allocates some data structures that will be used for loading files
on existing image.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Zoltán Böszörményi [Thu, 9 Jul 2020 11:21:11 +0000 (13:21 +0200)]
mkfs.f2fs: allow setting volume UUID manually
This patch adds an option to assign UUID manually.
Usage:
# mkfs.f2fs -U uuid_string
Signed-off-by: Zoltán Böszörményi <zboszor@pr.hu>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Wed, 1 Jul 2020 08:23:29 +0000 (16:23 +0800)]
fsck.f2fs: Split build_segment_manager() to speed up auto-fix
Speed up fsck in auto-fix mode by splitting
build_segment_manager() into two disjoint parts:
early_build_segment_manager(), and
late_build_segment_manager(),
where in some cases (when !need_fsync_data_record(), or cannot
find_fsync_inode()), late_build_segment_manager() won't be needed in
auto-fix mode. This speeds up a little bit in those cases.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 30 Jun 2020 17:37:32 +0000 (10:37 -0700)]
dump.f2fs: dump symlink contents in lost_found
Let's allow to show symlink path.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Fri, 19 Jun 2020 09:52:12 +0000 (17:52 +0800)]
fsck.f2fs: Fix slow fsck in auto-fix mode
Split f2fs_init_nid_bitmap() into two disjoint parts:
f2fs_early_init_nid_bitmap(), and
f2fs_late_init_nid_bitmap(),
where f2fs_late_init_nid_bitmap() won't be called in auto-fix mode, when
no errors were found.
f2fs_late_init_nid_bitmap() contains the loop to create NID bitmap from
NAT. which is the main reason for slow fsck.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Wed, 5 Feb 2020 07:41:57 +0000 (15:41 +0800)]
fsck: prevent buffer overrun in quota code
A maliciously corrupted file systems can trigger buffer overruns in
the quota code used by fsck.
To fix it, quota file sizes are checked against real allocated
block index tables (inode, direct nodes, indirect nodes, double
indirect nodes). If the size mismatches, the quota file is considered
corrupted and will be regenerated.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Wuyun Zhao [Thu, 18 Jun 2020 12:48:07 +0000 (20:48 +0800)]
f2fs-tools: set cold flag for non-dir node
Signed-off-by: Wuyun Zhao <zhaowuyun@wingtech.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Rolf Eike Beer [Fri, 12 Jun 2020 08:31:25 +0000 (10:31 +0200)]
lib: pass all needed include paths to libf2fs
Otherwise the paths to uuid and blkid headers extracted from pkg-config are not
respected.
Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Rolf Eike Beer [Fri, 12 Jun 2020 08:14:10 +0000 (10:14 +0200)]
lib: use ${libuuid_LIBS} instead of -luuid everywhere
Otherwise linking will fail if the library is in an uncommon location or has a
non-standard name.
Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Rolf Eike Beer [Fri, 12 Jun 2020 08:12:24 +0000 (10:12 +0200)]
lib: fix include path for blkid.h
$ pkg-config --cflags blkid
-I/usr/include/blkid
The "blkid/" directory is actually part of the include path. This usually still
works because most people have the path one level up in their default include
path.
Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Rolf Eike Beer [Fri, 12 Jun 2020 08:09:58 +0000 (10:09 +0200)]
lib: fix include path for uuid.h
$ pkg-config --cflags uuid
-I/usr/include/uuid
The "uuid/" directory is actually part of the include path. This usually still
works because most people have the path one level up in their default include
path.
Signed-off-by: Rolf Eike Beer <eb@emlix.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 8 Jun 2020 18:37:28 +0000 (11:37 -0700)]
libf2fs: fix build error on Windows
Windows doesn't support S_ISREG, so let's avoid depedency.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Eric Biggers [Fri, 29 May 2020 20:47:38 +0000 (13:47 -0700)]
fsck.f2fs: fix dirent position check for encrypted+casefolded dentries
fsck.f2fs reports corruption if the filesystem contains any encrypted +
casefolded directories with any substantial number of dentries:
[ASSERT] (f2fs_check_dirent_position:1374) --> Wrong position of dirent pino:8, name:۟�[I�^*�(�5~�}�D��#]7�8�ˎ�, level:1, dir_level:0, pgofs:4, correct range:[2, 3]
The problem is that f2fs_check_dirent_position() computes the wrong hash
for encrypted+casefolded dentries. It's not actually possible for it to
compute the correct hash, because it would need the encryption key.
However, the on-disk dentry already contains the hash code, and its
correctness was already verified by f2fs_check_hash_code() if possible.
So, make f2fs_check_dirent_position() use the hash code from disk rather
than recompute it.
Also fix it to print the filename in human-readable form.
This bug was causing 'kvm-xfstests -c f2fs/encrypt -g casefold'
to fail with the test_dummy_encryption_v2 and encryption+casefolding
kernel patches applied.
Fixes:
7f3767ee8dc5 ("f2fs-tools: Casefolded Encryption support")
Cc: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daeho Jeong [Wed, 27 May 2020 03:33:13 +0000 (12:33 +0900)]
f2fs_io: add randread
I've added a new command to evaluate random read.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Wed, 15 Apr 2020 03:26:48 +0000 (11:26 +0800)]
fsck.f2fs: fix to avoid overflow during print_inode_info()
While dumping files during fsck, print_inode_info() didn't check
sanity of inode, so insane i_extra_isize could cause overflow
when printing i_addr, to avoid that, let's add a check condition.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 7 Apr 2020 10:01:07 +0000 (18:01 +0800)]
fsck.f2fs: fix to check validation of i_xattr_nid
Otherwise, fsck.f2fs will access invalid memory address as below:
- fsck_verify
- dump_node
- dump_file
- dump_inode_blk
- dump_xattr
- read_all_xattrs
- get_node_info
access &(F2FS_FSCK(sbi)->entries[nid])
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Tue, 7 Apr 2020 10:01:06 +0000 (18:01 +0800)]
fsck.f2fs: fix to check validation of block address
Otherwise, if block address is invalid, we may access invalid
memory address in is_sit_bitmap_set().
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Érico Nogueira [Sun, 12 Apr 2020 21:34:35 +0000 (18:34 -0300)]
mkfs.f2fs: update manual and help message
Add options that were missing from the mkfs.f2fs man page, and explain
them more thoroughly when possible. Special care was given to document
all the features that can be enabled during the creation of the
filesystem.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Thu, 19 Mar 2020 10:28:49 +0000 (18:28 +0800)]
fsck.f2fs: fix wrong addrs_per_{inode,block}
generic/339 reports below assertion on image w/ compression feature
enabled.
[ASSERT] (f2fs_check_dirent_position:1366) -->
Wrong position of dirent pino:4521, name:"....", level:9, dir_level:0,
pgofs:1880, correct range:[1882, 1883]
The root cause is we calculate blkaddr number in direct node
incorrectly for directory inode, since during calculation, we only
need align blkaddr number to cluster size for regular inode rather
than directory inode, let's fix it.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 16 Mar 2020 03:05:04 +0000 (11:05 +0800)]
dump.f2fs: fix to print all blkaddr in .i_addr
There are totally ADDRS_PER_INODE() blkaddrs in .i_addr, fix to
print all of them.
In addition, use get_extra_isize() rather than __get_extra_isize()
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Theodore Ts'o [Fri, 13 Mar 2020 15:28:54 +0000 (11:28 -0400)]
man: add missing man page for f2fs_io
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
[Jaegeuk Kim: add missing entries and fix Makefile.am]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 1 May 2020 23:02:28 +0000 (16:02 -0700)]
f2fs_io: show more flags
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 1 May 2020 22:59:02 +0000 (15:59 -0700)]
f2fs_io: don't give garbage data in upper 32bits
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Fri, 1 May 2020 21:50:24 +0000 (14:50 -0700)]
f2fs_io: add fsync
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Wed, 1 Apr 2020 20:24:38 +0000 (13:24 -0700)]
fsck.f2fs: quick fix of CLOCK_BOOTTIME in mac
This fixes build error on mac.
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Sahitya Tummala [Tue, 24 Mar 2020 09:12:50 +0000 (14:42 +0530)]
fsck.f2fs: allow fsck to fix issues with online resize due to SPO
Add support for new CP flag CP_RESIZEFS_FLAG set during online
resize FS. If SPO happens after SB is updated but CP isn't, then
allow fsck to fix it.
The fsck errors without this fix -
Info: CKPT version =
6ed7bccb
Wrong user_block_count(2233856)
[f2fs_do_mount:3365] Checkpoint is polluted
The subsequent mount failure without this fix -
[ 11.294650] F2FS-fs (sda8): Wrong user_block_count: 2233856
[ 11.300272] F2FS-fs (sda8): Failed to get valid F2FS checkpoint
Signed-off-by: Sahitya Tummala <stummala@codeaurora.org>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Thu, 5 Mar 2020 08:12:49 +0000 (16:12 +0800)]
fsck.f2fs: account COMPRESS_ADDR as reserved blocks correctly
We should not account COMPRESS_ADDR as reserved block once we
released compress block on compress inode.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Fri, 6 Mar 2020 06:38:25 +0000 (14:38 +0800)]
f2fs_io: support compress blocks ioctls
Adds to support below three commands to operate reserved blocks
in compressed inode.
- get_cblocks
- release_cblocks
- reserve_cblocks
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Wei Wang [Mon, 9 Mar 2020 19:54:27 +0000 (12:54 -0700)]
fsck.f2fs: report real wall time
clock_t time is per-process time, which is not wall time. For example,
if the CPU is shared by other processes, clock_t time may advance slower
than wall clock. On the other hand, if the current process is
multithreaded and more than one execution core is available, clock_t
time may advance faster than wall clock.
this CL changes it to use CLOCK_BOOTTIME (Linux-specific)
Signed-off-by: Wei Wang <wvw@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Tom Cherry [Wed, 4 Mar 2020 21:41:55 +0000 (13:41 -0800)]
Include private/fs_config.h directly when needed
This is no longer a transitive include of android_filesystem_config.h
Signed-off-by: Tom Cherry <tomcherry@google.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Daniel Rosenberg [Wed, 20 Nov 2019 04:21:33 +0000 (20:21 -0800)]
f2fs-tools: Casefolded Encryption support
This adds support for casefolded and encrypted directories.
Fsck cannot check the hashes of such directories because it would
require access to the encryption key to generate the siphash
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 24 Feb 2020 11:31:15 +0000 (19:31 +0800)]
fsck.f2fs: allow --dry-run to check readonly mounted fs
As Ondřej Jirman <megi@xff.cz> reported:
I was trying to run: fsck.f2fs --dry-run /dev/mmcblk0p2 on a RO mounted device,
and fsck refuses to run. Strace shows that it tries to open the block device
with O_EXCL even in RO mode, which will always fail if the block device
is mounted.
fsck.f2fs --dry-run /dev/mmcblk0p2
Info: Dry run
Info: Mounted device!
Info: Check FS only on RO mounted device
Error: Failed to open the device!
I suggest not using O_EXCL for --dry-run check.
Let's change to allow --dry-run to check readonly mounted fs.
Reported-by: Ondřej Jirman <megi@xff.cz>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
xiongping1 [Mon, 13 Jan 2020 06:52:30 +0000 (14:52 +0800)]
resize.f2fs: add option for large_nat_bitmap feature
resize.f2fs has already supported large_nat_bitmap feature, but has no
option to turn on it.
This change add a new '-i' option to control turning on it.
Signed-off-by: xiongping1 <xiongping1@xiaomi.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 25 Mar 2019 13:20:33 +0000 (21:20 +0800)]
dump.f2fs: print more info of inode layout
This patch adds to print all valid blkaddrs in inode's i_addr field,
besides, it also adds to print meaning of specific blkaddr(flag).
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Chao Yu [Mon, 25 Mar 2019 13:19:35 +0000 (21:19 +0800)]
f2fs-tools: support data compression
This patch adds to support compression, introducing '-O compression'
option to enable this feature in image.
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Mon, 18 Nov 2019 10:13:21 +0000 (18:13 +0800)]
fsck.f2fs: Enable user-space cache
Added command line options -c <num_cache_entry> and -m <max_hash_collision>
to activate cache for fsck. It may significantly speed up fsck.
Signed-off-by: Robin Hsu <robinhsu@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Mon, 18 Nov 2019 10:13:07 +0000 (18:13 +0800)]
libf2fs_io: Add user-space cache
Implemented cache options in F2FS configuration 'c' (i.e. user
options):
* use c.cache_config.num_cache_entry to set the number of
cache entries (in block), minimum 1024, or 0 to disable cache.
* use c.cache_config.max_hash_collision to set maximum hash
collision (max 16).
* use c.cavhe_config.dbg_en to enable printing of debug messages.
Cache properties:
* Per block save based (block size = f2fs block size = 4K Bytes)
* Device block indices are used to hash (simple modular hash
function) to cache indices.
* A hash collision (when a block is hashed to an already occupied
cache block) will trigger offset-based relocation to max 16
other places, at first empty slot found policy.
* A hash collision with all 17 possible relocating places occupied
will evict the oldest used one (replaced by the new read block).
A global 'use' tick is kept ticking-up per block read for this
purpose.
* On write cache hit, it will update the corresponding cached
block.
* On write cache miss, the data won't be cached.
* Write-through policy only: never delayed write.
* On read cache hit, it returns the cached block.
* On read cache miss, it issues an I/O read to the kernel, and
then caches and returns the block.
* Currently read ahead is not implemented. Yet, the read ahead
logic is kept as before, i.e. using kernel's read ahead
mechanism.
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: Fix one bug in dcache_update_rw()]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 2 Jan 2020 18:44:42 +0000 (10:44 -0800)]
f2fs-tools: handle /sys is not mounted
Format a partition, even if /sys is not mounted.
Reported-by: David Heidelberg <david@ixit.cz>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 14 Oct 2019 17:10:31 +0000 (10:10 -0700)]
fsck.f2fs: add --{no-}kernel-check to bypass kernel version diff or not
Given this option, fsck.f2fs does not run fsck forcefully, even if kernel
is updated. Android devices will do --kernel-check by default, while others
will not.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 12 Feb 2019 02:21:30 +0000 (18:21 -0800)]
f2fs_io: add set_fsverity
f2fs_io set_verity [file]
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
Ingo Prochaska [Tue, 24 Dec 2019 12:49:54 +0000 (13:49 +0100)]
fsck.f2fs: fix typo
matcing->matching
Signed-off-by: Nvertigo <nvertigo67@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Santiago Hormazabal [Wed, 18 Dec 2019 18:46:05 +0000 (15:46 -0300)]
sg_write_buffer: fix sg_write_buffer build outside the source dir
Current Makefile.am included the 'include' directory
directly as -I./include, but this doesn't guarantee that
said directory exists when building outside of the source
directory.
By adding $(srcdir) instead of '.' this gets fixed automatically.
Signed-off-by: Santiago Hormazabal <santiagohssl@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Robin Hsu [Thu, 31 Oct 2019 13:57:59 +0000 (21:57 +0800)]
f2fs-tools: reuse same pointer, exit on error without clean-up
This patch tries to fix memory leak problem reported in Android.
Fixed the following problems in fsck.f2fs, make_f2fs and sload_f2fs:
* reuse of same pointer without clean-up
* exit on error without clean-up
Signed-off-by: Robin Hsu <robinhsu@google.com>
[Jaegeuk Kim: add missing definition to avoid build error]
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Thu, 19 Dec 2019 01:26:03 +0000 (17:26 -0800)]
f2fs-tools: avoid unused parameter build errors
This patch specifies unused parameters explicitly to avoid build errors.
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 8 Oct 2019 22:06:09 +0000 (15:06 -0700)]
f2fs_io: accept fallocate to have bigger size
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Tue, 10 Dec 2019 16:42:20 +0000 (08:42 -0800)]
fsck.f2fs: do not access nat etnries in ckpt before initialization
ckpt->entries is initialized by fsck_init(), but we tried to access it during
f2fs_do_mount().
The call sequence is:
- f2fs_do_mount
- record_fsync_data
- traverse_dnodes
- do_record_fsync_data
- ADDRS_PER_PAGE
- get_node_info
- node_info_from_raw_nat(fsck->entries[nid])
- do_fsck
- fsck_init
- build_nat_area_bitmap
- fsck->entries = calloc(fsck->nr_nat_entries);
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:30 +0000 (16:59 +0900)]
fsck: Check write pointer consistency of non-open zones
To catch bugs in write pointer handling code for zoned block devices,
have fsck check consistency of write pointers of non-open zones, that
current segments do not point to. Check two items comparing write pointer
positions with valid block maps in SIT.
The first item is check for zones with no valid blocks. When there is no
valid blocks in a zone, the write pointer should be at the start of the
zone. If not, next write operation to the zone will cause unaligned write
error. If write pointer is not at the zone start, reset the zone to move
the write pointer to the zone start.
The second item is check between write pointer position and the last
valid block in the zone. It is unexpected that the last valid block
position is beyond the write pointer. In such a case, report as the bug.
Fix is not required for such zone, because the zone is not selected for
next write operation until the zone get discarded.
In the same manner as the consistency check for current segments, do the
check and fix twice: at the beginning of do_fsck() to avoid unaligned
write error during fsck, and at fsck_verify() to reflect meta data
updates by fsck.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:29 +0000 (16:59 +0900)]
fsck: Check write pointer consistency of open zones
On sudden f2fs shutdown, write pointers of zoned block devices can go
further but f2fs meta data keeps current segments at positions before the
write operations. After remounting the f2fs, this inconsistency causes
write operations not at write pointers and "Unaligned write command"
error is reported.
To avoid the error, have f2fs.fsck check consistency of write pointers
of open zones that current segments point to. Compare each current
segment's position and the write pointer position of the open zone. If
inconsistency is found and 'fix_on' flag is set, assign a new zone to the
current segment and check the newly assigned zone has write pointer at
the zone start. Leave the original zone as is to keep data recorded in
it.
To care about fsync data, refer each seg_entry's ckpt_valid_map to get
the last valid block in the zone. If the last valid block is beyond the
current segments position, fsync data exits in the zone. In case fsync
data exists, do not assign a new zone to the current segment not to lose
the fsync data. It is expected that the kernel replay the fsync data and
fix the write pointer inconsistency at mount time.
Also check consistency between write pointer of the zone the current
segment points to with valid block maps of the zone. If the last valid
block is beyond the write pointer position, report to indicate a bug. If
'fix_on' flag is set, assign a new zone to the current segment.
When inconsistencies are found, turn on 'bug_on' flag in fsck_verify() to
ask users to fix them or not. When inconsistencies get fixed, turn on
'force' flag in fsck_verify() to enforce fixes in following checks.
This check and fix is done twice. The first is done at the beginning of
do_fsck() function so that other fixes can reflect the current segment
modification. The second is done in fsck_verify() to reflect updated meta
data by other fixes.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:28 +0000 (16:59 +0900)]
fsck: Check fsync data always for zoned block devices
Fsck checks fsync data when UMOUNT flag is not set. When the f2fs was not
cleanly unmouted, UMOUNT flag is not recorded in meta data and fsync data
can be left in the f2fs. The first fsck run checks fsync data to reflect
it on quota status recovery. After that, fsck writes UMOUNT flag in the
f2fs meta data so that second fsck run can skip fsync data check.
However, fsck for zoned block devices need to care fsync data for all
fsck runs. The first fsck run checks fsync data, then fsck can check
write pointer consistency with fsync data. However, since second fsck run
does not check fsync data, fsck detects write pointer at fsync data end
is not consistent with f2fs meta data. This results in meta data update
by fsck and fsync data gets lost.
To have fsck check fsync data always for zoned block devices, introduce
need_fsync_data_record() helper function which returns boolean to tell
if fsck needs fsync data check or not. For zoned block devices, always
return true. Otherwise, return true if UMOUNT flag is not set in CP.
Replace UMOUNT flag check codes for fsync data with the function call.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:27 +0000 (16:59 +0900)]
fsck: Introduce move_one_curseg_info() function
When fsck updates one of the current segments, update_curseg_info() is
called specifying a single current segment as its argument. However,
update_curseg_info() calls move_curseg_info() function which updates all
six current segments. Then update_curseg_info() for a single current
segment moves all current segments.
This excessive current segment move causes an issue when a new zone is
assigned to a current segment because of write pointer inconsistency.
Even when a current segment has write pointer inconsistency, all other
current segments should not be moved because they may have fsync data
at their positions.
To avoid the excessive current segment move, introduce
move_one_curseg_info() function which does same work as
move_curseg_info() only for a single current segment. Call
move_one_curseg_info() in place of move_curseg_info() from
update_curseg_info().
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:26 +0000 (16:59 +0900)]
fsck: Find free zones instead of blocks to assign to current segments
When fsck needs to assign a new area to a curreng segment, it calls
find_next_free_block() function to find a new block to assign. For zoned
block devices, fsck checks write pointer consistency with current
segments' positions. In case a curseg is inconsistent with the
write pointer of the zone it points to, fsck should assign not a new free
block but a new free zone/section with write pointer at the zone start,
so that next write to the current segment succeeds without error.
To extend find_next_free_block() function's capability to find not only
a block but also a zone/section, add new_sec flag to
find_next_free_block() function. When new_sec flag is true, skip check
for each block's availability so that the check is done with unit of
section. Note that it is ensured that one zone has one section for f2fs
on zoned block devices. Then the logic to find a new free section is good
to find a new free zone.
When fsck target devices have ZONED_HM model, set new_sec flag true to
call find_next_free_block() from move_curseg_info(). Set curseg's
alloc_type not SSR but LFS for the devices with ZONED_HM model, because
SSR block allocation is not allowed for zoned block devices. Also skip
relocate_curseg_offset() for the devices with ZONED_HM model for the
same reason.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:25 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_reset_zone() helper function
To prepare for write pointer consistency fix by fsck, add
f2fs_reset_zone() helper function which calls RESET ZONE command. The
function is added to lib/libf2fs_zoned which gathers zoned block device
related functions.
When f2fs-tools are built without blkzoned.h kernel header, the helper
function f2fs_reset_zone() prints an error message as other helper
functions in lib/libf2fs_zoned print. To make the message consistent
through the all helper functions, modify message strings in
f2fs_check_zones() and f2fs_reset_zones().
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:24 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_report_zone() helper function
To prepare for write pointer consistency check by fsck, add
f2fs_report_zone() helper function which calls REPORT ZONE command to
get write pointer status of a single zone. The function is added to
lib/libf2fs_zoned which gathers zoned block device related functions.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Shin'ichiro Kawasaki [Thu, 28 Nov 2019 07:59:23 +0000 (16:59 +0900)]
libf2fs_zoned: Introduce f2fs_report_zones() helper function
To prepare for write pointer consistency check by fsck, add
f2fs_report_zones() helper function which calls REPORT ZONE command to
get write pointer status. The function is added to lib/libf2fs_zoned
which gathers zoned block device related functions.
To check write pointer consistency with f2fs meta data, fsck needs to
refer both of reported zone information and f2fs super block structure
"f2fs_sb_info". However, libf2fs_zoned does not import f2fs_sb_info. To
keep f2fs_sb_info structure out of libf2fs_zoned, provide a callback
function in fsck to f2fs_report_zones() and call it for each zone.
Add SECTOR_SHIFT definition in include/f2fs_fs.h to avoid a magic number
to convert bytes into 512B sectors.
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Qiuyang Sun [Mon, 23 Sep 2019 04:24:59 +0000 (12:24 +0800)]
fsck.f2fs: check total_segments from devices in raw_super
For multi-device F2FS, we should check if the sum of total_segments from
all devices matches segment_count.
Signed-off-by: Qiuyang Sun <sunqiuyang@huawei.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Fabrice Fontaine [Sun, 22 Sep 2019 18:41:37 +0000 (20:41 +0200)]
mkfs/Makefile.am: fix build without blkid
Commit
d56232bc1640e2a3ffc412faff42cd8e77dbb8dd hardcoded -lblkid in
libf2fs_format_la_LDFLAGS which breaks the build with --without-blkid
option so use ${libblkid_LIBS} instead
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Jaegeuk Kim [Mon, 7 Oct 2019 16:31:48 +0000 (09:31 -0700)]
tools/fibmap: fix build error
fibmap.c:28:10: fatal error: 'sys/sysmacros.h' file not found
^~~~~~~~~~~~~~~~~
1 error generated.
make[3]: *** [fibmap.o] Error 1
make[3]: *** Waiting for unfinished jobs....
Fixes:
ce64ea0815bf ("f2fs-tools: Add support for Casefolding")
Reviewed-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Eric Biggers [Fri, 4 Oct 2019 22:43:17 +0000 (15:43 -0700)]
f2fs_io: add copy command
Add a copy command to f2fs_io, to allow testing direct I/O writes where
the source page is from a different file (mmap) or an internal kernel
page (sendfile). It could be useful for other tests in the future too.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>