Qu Wenruo [Thu, 19 Oct 2017 06:16:17 +0000 (14:16 +0800)]
btrfs-progs: mkfs: Only zero out the first 1M for rootdir
It's a waste of IO to fill the whole image before creating btrfs on it,
just wiping the first 1M, and then write 1 byte to the last position to
create a sparse file.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 12 Oct 2017 06:24:34 +0000 (14:24 +0800)]
btrfs-progs: mkfs: Enhance minimal device size calculation to fix mkfs failure on small file
Since commit
c11e36a29e84 ("Btrfs-progs: Do not force mixed block group
creation unless '-M' option is specified"), mkfs no longer use mixed
block group unless specified manually.
This breaks the minimal device size calculation, which only considered
mixed block group use case.
This patch enhances minimal device size calculation for mkfs, by using
different minimal stripe length (calculated from code) for different
profiles, and use them to calculate minimal device size.
Reported-by: Wesley Aptekar-Cassels <W.Aptekar@gmail.com>
Fixes:
c11e36a29e84 ("Btrfs-progs: Do not force mixed block group creation unless '-M' option is specified")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
[ updated comments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 1 Nov 2017 01:30:42 +0000 (09:30 +0800)]
btrfs-progs: test/common: Enhance prepare_test_dev to reset device size
So prepare_test_dev() can be called several times in one test case, to
test different device sizes.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ switch to [ ] ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 1 Nov 2017 01:30:41 +0000 (09:30 +0800)]
btrfs-progs: test/common: Introduce run_mustfail_stdout
For later test case which needs info from stderr.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 27 Nov 2017 22:58:23 +0000 (23:58 +0100)]
btrfs-progs: tests: fix typos in test names
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Fri, 24 Nov 2017 05:21:15 +0000 (14:21 +0900)]
btrfs-progs: mkfs: check the status of file at mkfs
Currently, only the status of block devices is checked at mkfs,
but we should also check for regular files whether they are already
formatted or mounted to prevent overwrite accidentally.
Device status is checked by test_dev_for_mkfs().
The part which is not related to block device is split from this
and used for both block device and regular file.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 27 Nov 2017 22:50:45 +0000 (23:50 +0100)]
btrfs-progs: tests: mkfs/008 mkfs with force
With extended tests in the following patch a file based filesystem image
also needs -f, otherwise it will fail.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 10 Nov 2017 01:34:19 +0000 (09:34 +0800)]
btrfs-progs: test/fsck/021: Cleanup custom check by overriding check_image
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update comment ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 10 Nov 2017 01:34:18 +0000 (09:34 +0800)]
btrfs-progs: test/fsck/020: Cleanup custom check function by overriding check_image function
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 10 Nov 2017 01:34:17 +0000 (09:34 +0800)]
btrfs-progs: test/fsck: Introduce test images containing tree reloc tree
Reloc tree is a special tree with very short life span. It acts as a
special snapshot for any tree, with related nodes/leaves or EXTENT_DATA
modified to point to new position.
Considering the short life span and its special purpose, it should be
quite reasonable to keep them as both corner case for fsck and
educational dump for anyone interested in relocation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Fri, 10 Nov 2017 09:47:10 +0000 (17:47 +0800)]
btrfs-progs: lowmem check: Reword an unclear error message about file extent gap
This error occurs when no_holes is not set, but there is a gap
before the file extent.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Howard [Sun, 26 Nov 2017 05:51:43 +0000 (19:51 -1000)]
btrfs-progs: docs: update btrfs-subvolume manual page
To simplify, I suggest moving the 'writable/readonly' issue only to the
-r line, instead of having it introduced in two places.
Pull-request: #80
Author: Howard <hwj@BridgeportContractor.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Fri, 24 Nov 2017 20:26:08 +0000 (21:26 +0100)]
btrfs-progs: dump_tree: remove superfluous _TREE
-# btrfs inspect-internal dump-tree -t fs /dev/block/device
ERROR: unrecognized tree id: fs
Without this fix I can't dump-tree fs, but I can dump-tree fs_tree and
also fs_tree_tree, which is a bit silly.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Hans van Kranenburg <hans@knorrie.org>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 24 Nov 2017 14:32:13 +0000 (15:32 +0100)]
btrfs-progs: tests: enable check lowmem in travis CI
We missed some regressions because the lowmem mode was not run in the CI
tests. This is partially due to the incomplete implementation but we
have exceptions for the --repair mode in the tests.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:25 +0000 (17:03 +0800)]
btrfs-progs: fsck-tests: Introduce test case with keyed data backref with shared tree blocks
For snapshot shared tree blocks with source subvolume, the keyed backref
counter only counts the exclusive owned references.
In the following case, 258 is a snapshot of 257, which inherits all the
reference to this data extent.
------
item 4 key (
12582912 EXTENT_ITEM 524288) itemoff 3741 itemsize 140
refs 179 gen 9 flags DATA
extent data backref root 257 objectid 258 offset 0 count 49
extent data backref root 257 objectid 257 offset 0 count 1
extent data backref root 256 objectid 258 offset 0 count 128
extent data backref root 256 objectid 257 offset 0 count 1
------
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:24 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix false alerts of referencer count mismatch for snapshot
Btrfs lowmem check reports such false alerts:
------
ERROR: extent[
366498091008,
134217728] referencer count mismatch (root: 827, owner: 73782, offset:
134217728) wanted: 4, have: 26
ERROR: extent[
366498091008,
134217728] referencer count mismatch (root: 818, owner: 73782, offset:
134217728) wanted: 4, have: 26
ERROR: extent[
366498091008,
134217728] referencer count mismatch (root: 870, owner: 73782, offset:
134217728) wanted: 4, have: 26
------
While in extent tree, the extent has:
------
item 81 key (
366498091008 EXTENT_ITEM
134217728) itemoff 9008 itemsize 169
refs 39 gen 224 flags DATA
extent data backref root 827 objectid 73782 offset
134217728 count 4
extent data backref root 818 objectid 73782 offset
134217728 count 4
extent data backref root 259 objectid 73482 offset
134217728 count 1
extent data backref root 644 objectid 73782 offset
134217728 count 26
extent data backref root 870 objectid 73782 offset
134217728 count 4
------
And in root 827, there is one leaf with 4 references to that extent
which is owned by 827:
------
leaf
714964992 items 68 free space 10019 generation 641 owner 827
leaf
714964992 flags 0x1(WRITTEN) backref revision 1
......
item 64 key (73782 EXTENT_DATA
134217728) itemoff 11878 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset 0 nr 6410240 ram
134217728
extent compression 0 (none)
item 65 key (73782 EXTENT_DATA
140627968) itemoff 11825 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset 6410240 nr 5120000 ram
134217728
extent compression 0 (none)
item 66 key (73782 EXTENT_DATA
145747968) itemoff 11772 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset
11530240 nr 7675904 ram
134217728
extent compression 0 (none)
item 67 key (73782 EXTENT_DATA
153423872) itemoff 11719 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset
19206144 nr 6397952 ram
134217728
extent compression 0 (none)
------
And starts from next leaf, there are 22 references to the data extent:
------
leaf
894861312 items 208 free space 59 generation 261 owner 644
leaf
894861312 flags 0x1(WRITTEN) backref revision 1
item 0 key (73782 EXTENT_DATA
159821824) itemoff 16230 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset
25604096 nr 8192 ram
134217728
extent compression 0 (none)
item 1 key (73782 EXTENT_DATA
159830016) itemoff 16177 itemsize 53
generation 224 type 1 (regular)
extent data disk byte
366498091008 nr
134217728
extent data offset
25612288 nr 7675904 ram
134217728
extent compression 0 (none)
......
------
However the next leaf is owned by other subvolume, normally owned by
(part of) the snapshot source.
Fix it by also checking the leaf's owner before increasing the reference
counter.
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:23 +0000 (17:03 +0800)]
btrfs-progs: fsck-test: Add new image with shared block ref only metadata backref
The image is dumped by modifying kernel to sleep long enough before
merging relocation trees, so we can just copy the whole image to other
place before kernel begins to merge reloc trees.
And the base image is created by the following script to bump metadata
size:
------
dev=~/test.img
mnt=/mnt/btrfs
umount $mnt &> /dev/null
fallocate -l 128M $dev
mkfs.btrfs -f -n 4k -m single -d single $dev
mount $dev $mnt -o nospace_cache,max_inline=2048
btrfs subvolume create $mnt/src
for i in $(seq -w 0 128); do
xfs_io -f -c "pwrite 0 2k" $mnt/src/file_$i > /dev/null
done
for i in $(seq -w 0 64); do
btrfs subvolume snapshot $mnt/src/ $mnt/snapshot_$i
touch $mnt/snapshot_$i/new
done
sync
------
The image triggers several corner cases that the old lowmem mode didn't
consider.
Like metadata backref with FULL_BACKREF flag and only SHARED_BLOCK_REF
backrefs for metadata. And several tree reloc trees with shared
leaves/nodes to confuse old lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:22 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix false alerts for image with shared block ref only backref
[BUG]
For image with shared block ref only metadata item like:
------
item 66 key (
21573632 METADATA_ITEM 0) itemoff 3971 itemsize 24
refs 66 gen 9 flags TREE_BLOCK|FULL_BACKREF
tree block skinny level 0
item 0 key (
21573632 SHARED_BLOCK_REF
21676032) itemoff 3995 itemsize 0
shared block backref
item 1 key (
21573632 SHARED_BLOCK_REF
21921792) itemoff 3995 itemsize 0
shared block backref
item 2 key (
21573632 SHARED_BLOCK_REF
21995520) itemoff 3995 itemsize 0
shared block backref
item 3 key (
21573632 SHARED_BLOCK_REF
22077440) itemoff 3995 itemsize 0
shared block backref
...
------
Lowmem mode check will report false alerts like:
------
ERROR: extent[
21573632 4096] backref lost (owner: 256, level: 0)
------
[CAUSE]
In fact, the false alerts are not even from extent tree verfication, but
a fs tree helper which is designed to make sure there is some tree block
referring to the fs tree block.
The idea is to find inlined tree backref then keyed TREE_BLOCK_REF_KEY.
However it missed SHARED_BLOCK_REF_KEY, and caused such false alert.
[FIX]
Add SHARED_BLOCK_REF_KEY to make the warning shut up.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:21 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix function call stack overflow caused by wrong tree reloc tree detection
For reloc tree root, its backref points to itself. So for such case,
we should finish the lookup.
Previous end condition is to ensure it's reloc tree *and* needs its root
bytenr to match the bytenr passed in.
However the @root passed can be another tree, e.g. other tree reloc root
which shares the node/leaf. This makes any check based on @root passed
in invalid.
The patch removes the unreliable root objectid detection, and only uses
root->bytenr check.
For the possibility of invalid self-pointing backref, extent tree
checker should have already handled it, so we don't need to bother in
fs tree checker.
Fixes:
54c8f9152fd9 ("btrfs-progs: check: Fix lowmem mode stack overflow caused by fsck/023")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:20 +0000 (17:03 +0800)]
btrfs-progs: backref: Allow backref walk to handle direct parent ref
[BUG]
Btrfs lowmem mode fails with the following ASSERT() on certain valid
image.
------
backref.c:466: __add_missing_keys: Assertion `ref->root_id` failed, value 0
------
[REASON]
Lowmem mode uses btrfs_find_all_roots() when walking down fs trees.
However if a tree block with only shared parent backref like below,
backref code from btrfs-progs doesn't handle it correct.
------
item 72 key (
604653731840 METADATA_ITEM 0) itemoff 13379 itemsize 60
refs 4 gen 7198 flags TREE_BLOCK|FULL_BACKREF
tree block skinny level 0
shared block backref parent
604498477056
shared block backref parent
604498460672
shared block backref parent
604498444288
shared block backref parent
604498411520
------
Such shared block ref is *direct* ref, which means we don't need to
solve its key, nor its rootid.
As the objective of backref walk is to find all direct parents until it
reaches tree root.
So for such direct ref, it should be pended to pref_stat->pending, other
than pending it to pref_stat->pending_missing_key.
[FIX]
For direct ref, pending it to pref_state->pending directly to solve the
problem.
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:19 +0000 (17:03 +0800)]
btrfs-progs: fsck-test: Introduce test case for false data extent backref lost
Introduce a new test image, which has an extent item with no inlined
extent data ref, but all keyed extent data ref.
Only in this case we can trigger fase data extent backref lost bug in
lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:18 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix false backref lost warning for keyed extent data ref
For keyed extent ref, its offset is calculated offset (file offset -
file extent offset), just like inlined extent data ref.
However the code is using file offset to hash extent data ref offset,
causing false backref lost warning like:
------
ERROR: data extent[
16913485824 7577600] backref lost
------
Fixes:
b0d360b541f0 ("btrfs-progs: check: introduce function to check data backref in extent tree")
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:17 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix inlined data extent ref lookup
When lowmem fsck tries to find backref of a specified file extent, it
searches inlined data ref first.
However, extent data ref contains both owner root objectid, inode number
and calculated offset (file offset - extent offset).
The code only checks owner root objectid, not checking inode number nor
calculated offset.
This makes lowmem mode fail to detect any backref mismatch if there is
a inlined data ref with the same owner objectid.
Fix it by also checking extent data ref's objectid and offset.
Fixes:
b0d360b541f0 ("btrfs-progs: check: introduce function to check data backref in extent tree")
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:16 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix NULL pointer access caused by large tree reloc tree
[BUG]
v4.14 btrfs-progs can't pass new self test image with large tree reloc
trees. It will fail with later "shared_block_ref_only.raw.xz" test
image with NULL pointer access.
[CAUSE]
For image with higher (level >= 2) tree reloc tree, for function
need_check() its ulist will be empty as tree reloc tree won't be
accounted in btrfs_find_all_roots(). Then accessing ulist->roots with
rb_first() will return NULL pointer.
[FIX]
For need_check() function, if @roots is empty, meaning it's a tree reloc
tree, always check them. Although this can be slow, but at least it's
safe that we won't skip any possible wrong tree block.
Fixes:
5e2dc770471b ("btrfs-progs: check: skip shared node or leaf check for low_memory mode")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Nov 2017 09:03:15 +0000 (17:03 +0800)]
btrfs-progs: lowmem check: Fix regression which screws up extent allocator
[BUG]
Commit
723427d7e6b7 ("btrfs-progs: check: change the way lowmem mode
traverses metadata") introduces a regression which could make some fsck
self test case to fail.
For fsck test case 004-no-dir-item, btrfs check --mode=lowmem --repair
can cause BUG_ON() with ret = -17 (-EEXIST) when committing transaction.
The problem happens with the following backtrace:
./btrfs(+0x22045)[0x555d0dade045]
./btrfs(+0x2216f)[0x555d0dade16f]
./btrfs(+0x29df1)[0x555d0dae5df1]
./btrfs(+0x2a142)[0x555d0dae6142]
./btrfs(btrfs_alloc_free_block+0x78)[0x555d0dae6202]
./btrfs(__btrfs_cow_block+0x177)[0x555d0dad00a2]
./btrfs(btrfs_cow_block+0x116)[0x555d0dad05a8]
./btrfs(commit_tree_roots+0x91)[0x555d0db1fd4f]
./btrfs(btrfs_commit_transaction+0x18c)[0x555d0db20100]
./btrfs(btrfs_fix_super_size+0x190)[0x555d0db005a4]
./btrfs(btrfs_fix_device_and_super_size+0x177)[0x555d0db00771]
./btrfs(cmd_check+0x1757)[0x555d0db4f6ab]
./btrfs(main+0x138)[0x555d0dace5dd]
/usr/lib/libc.so.6(__libc_start_main+0xea)[0x7fa5e4613f6a]
./btrfs(_start+0x2a)[0x555d0dacddda]
The bug is triggered by that, extent allocator considers range
[
29360128,
29376512) as free and allocates it. However when inserting
EXTENT_ITEM, btrfs finds there is already one tree block (fs tree root),
returning -EEXIST and causing the later BUG_ON().
[CAUSE]
The cause is in repair mode, lowmem check always pins all metadata
blocks. However pinned metadata blocks will be unpined when transaction
commits, and will be marked as *FREE* space.
So later extent allocator will consider such range free and allocates
them incorrectly.
[FIX]
Don't pin metadata blocks without valid reason or preparation (like
discard all free space cache to re-calculate free space on next write).
Fixes:
723427d7e6b7 ("btrfs-progs: check: change the way lowmem mode traverses metadata")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 23 Nov 2017 19:24:39 +0000 (20:24 +0100)]
btrfs-progs: fix build of btrfs-show-super
The standalone utility btrfs-show-super has been obsoleted by 'btrfs
inspect-internal dump-super' but it's still in the repository and should
build in case somebody still uses it.
Reported-by: "John L. Center" <jlcenter15@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Tue, 31 Oct 2017 05:40:24 +0000 (14:40 +0900)]
btrfs-progs: dump-tree: print c/o/s/r time of ROOT_ITEM
Currently ctime/otime/stime/rtime of ROOT_ITEM are not printed in
print_root_item(). Fix this and print them if the values are not zero.
The function print_timespec() is moved forward to reuse.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 24 Nov 2017 14:41:33 +0000 (15:41 +0100)]
btrfs-progs: tests: fix path for travis helper script
The helper script ./travis-should-run-test has been moved to a directory
in 4.13.3 but the path in the config was not updated. This was not
caught in the CI environment and the tests did not report a failure.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 20 Nov 2017 16:39:52 +0000 (17:39 +0100)]
Btrfs progs v4.14
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Nov 2017 14:52:26 +0000 (15:52 +0100)]
btrfs-progs: update CHANGES for v4.14
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Nov 2017 15:16:54 +0000 (16:16 +0100)]
btrfs-progs: docs: move the rescue fix-device-size command and update
The subcommands are supposed to be in alphabetical order, move it to the
right spot and reword.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Nov 2017 14:31:45 +0000 (15:31 +0100)]
btrfs-progs: build: mention library dependency for reiserfs
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Nov 2017 14:24:39 +0000 (15:24 +0100)]
btrfs-progs: build: require libzstd support by default
The kernel 4.14 supports zstd, for version parity the btrfs-progs now
require libzstd by default. This can still be disabled by
./configure --disable-zstd.
Signed-off-by: David Sterba <dsterba@suse.com>
Baruch Siach [Tue, 31 Oct 2017 12:57:54 +0000 (14:57 +0200)]
btrfs-progs: convert: add missing types header
Build with musl libc needs the sys/types.h header for the dev_t type,
since this header is not included indirectly. This fixes the following
build failure:
In file included from convert/source-fs.c:23:0:
./convert/source-fs.h:112:1: error: unknown type name ‘dev_t’
dev_t decode_dev(u32 dev);
^~~~~
convert/source-fs.c:31:1: error: unknown type name ‘dev_t’
dev_t decode_dev(u32 dev)
^~~~~
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 31 Oct 2017 06:07:58 +0000 (14:07 +0800)]
btrfs-progs: print-tree: Print offset as tree objectid for ROOT_ITEM
For cases like reloc trees and subvolume trees, their key offset is the
tree id. The key will be printed as:
(TREE_RELOC ROOT_ITEM
18446744073709551607)
The negative number is long and even guys with real engineer brains
can't easily get the meaning.
This patch will change the output format to:
(TREE_RELOC ROOT_ITEM DATA_RELOC_TREE)
While for special offset value like 0 or (u64)-1, it's still shown as
is.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ reword comment ]
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Tue, 31 Oct 2017 09:13:44 +0000 (17:13 +0800)]
btrfs-progs: qgroup: split update_qgroup to reduce arguments
The function update_qgroup has too many arguments that are too difficult
to use. Therefore, split it to update_qgroup_info, update_qgroup_limit,
update_qgroup_relation.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Tue, 31 Oct 2017 09:13:43 +0000 (17:13 +0800)]
btrfs-progs: qgroup: cleanup the redundant function add_qgroup
There are reusable parts between update_qgroup and add_qgroup. So
introduce the function get_or_add_qgroup and use update_qgroup instead
of add_qgroup.
No functional changes.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Fri, 3 Nov 2017 08:28:06 +0000 (16:28 +0800)]
btrfs-progs: test: Add test image for lowmem mode referencer count mismatch false alert
Add a image which can reproduce the extent item referencer count
mismatch false alert for lowmem mode.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Fri, 3 Nov 2017 08:28:05 +0000 (16:28 +0800)]
btrfs-progs: lowmem check: Fix false alert about referencer count mismatch
The normal back reference counting doesn't care about the extent referred
by the extent data in the shared leaf. The check_extent_data_backref
function need to skip the leaf that owner mismatch with the root_id.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Fri, 3 Nov 2017 08:28:04 +0000 (16:28 +0800)]
btrfs-progs: lowmem check: Output more detailed information about file extent interrupt
Make lowmem mode output more detailed information about file extent
interrupt.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Fri, 3 Nov 2017 08:28:03 +0000 (16:28 +0800)]
btrfs-progs: test: Add test image for lowmem mode file extent interrupt
Add a image that the inlined extent coexist with the regular extent.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 7 Nov 2017 18:17:41 +0000 (19:17 +0100)]
btrfs-progs: docs: add note about mount option applicability
Copied from https://btrfs.wiki.kernel.org/index.php/Mount_options .
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 7 Nov 2017 18:14:48 +0000 (19:14 +0100)]
btrfs-progs: docs: add impact of atime/noatime
Copy from wiki
https://btrfs.wiki.kernel.org/index.php?title=Mount_options
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 7 Nov 2017 18:03:44 +0000 (19:03 +0100)]
btrfs-progs: docs: update mount options
Enhance the text, update for 4.14, sync with existing wiki page.
Signed-off-by: David Sterba <dsterba@suse.com>
Benjamin Peterson [Mon, 6 Nov 2017 06:49:57 +0000 (22:49 -0800)]
btrfs-progs: docs: correct grammar
Signed-off-by: Benjamin Peterson <bp@benjamin.pe>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 27 Oct 2017 16:03:42 +0000 (18:03 +0200)]
btrfs-progs: tests: extend fsck/028 to test fix-device-size and mount
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 10 Oct 2017 07:32:28 +0000 (15:32 +0800)]
btrfs-progs: tests/fsck: Add test case image for 'rescue fix-dev-size'
The image has 2 problems mixed:
1) Too small super total_bytes
This super total_bytes is manually modified to create such problem.
2) Unaligned dev item total_bytes
This is created by v4.12 kernel, with 128M + 2K device added, and
original device removed.
Then we can create such image with unaligned dev item total_bytes.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 10 Oct 2017 05:53:24 +0000 (13:53 +0800)]
btrfs-progs: check: Also check and repair unaligned/mismatch device and super sizes
Along with the rescue introduced, also introduce check and repair for them.
Unlike normal check functions, some of the check is optional, and even if
the image failed to pass optional check, kernel can still runs fine.
(But may cause noisy kernel warning)
So some check, mainly for alignment, will not cause btrfs check to fail,
but only to output warning and instructs how to fix it.
For repair, it just calls the same repair function in rescue, and is
included in 'btrfs check --repair'.
But 'btrfs rescue' is still the preferred method, since it can be used
independent of all the 'check' passes, if we know what's the exact
problem to fix.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 17 Oct 2017 07:45:50 +0000 (15:45 +0800)]
btrfs-progs: rescue: Introduce fix-device-size
Introduce new subcommand 'fix-device-size' to the rescue group, to fix
device size alignment-related problems.
Especially for people unable to mount their fs with super::total_bytes
mismatch, this tool will fix the problems and let the mount continue.
Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Qu Wenruo [Tue, 17 Oct 2017 07:12:39 +0000 (15:12 +0800)]
btrfs-progs: Introduce function to fix super block total bytes
Recent kernel (starting from v4.6) will refuse to mount if super block
total bytes is smaller than all devices' size.
This makes end user unable to do anything to their otherwise quite
healthy fs.
To fix such problem, introduce repair function to fix it on an unmounted
filesystem.
Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 17 Oct 2017 05:00:41 +0000 (13:00 +0800)]
btrfs-progs: Introduce function to fix unaligned device size
Recent kernel introduced alignment check for dev item, however older
kernel doesn't align device size when adding new device or shrinking
existing device.
This makes noisy kernel warning every time when any DEV_ITEM gets updated.
Introduce function to fix device size on an unmounted filesystem.
Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Fri, 27 Oct 2017 06:00:24 +0000 (11:30 +0530)]
btrfs-progs: tests/common: Display warning only after searching for btrfs kernel module
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 27 Oct 2017 13:50:34 +0000 (15:50 +0200)]
btrfs-progs: tests: don't pass size to prepare_test_dev if not necessary
Most tests don't need a specific size of the test device, the default
2GiB should be fine.
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Fri, 20 Oct 2017 01:43:05 +0000 (10:43 +0900)]
btrfs-progs: doc: add description of missing and example, of device remove
This patch updates help/document of "btrfs device remove" in two points:
1. Add explanation of 'missing' for 'device remove'. This is only
written in wikipage currently.
(https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices)
2. Add example of device removal in the man document. This is because
that explanation of "remove" says "See the example section below", but
there is no example of removal currently.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
[ move "" from the macro to help strings ]
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Fri, 20 Oct 2017 01:42:13 +0000 (10:42 +0900)]
btrfs-progs: device: add description of alias to help message
State that the 'delete' is the alias of 'remove' as the man page says.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 26 Oct 2017 18:44:55 +0000 (20:44 +0200)]
btrfs-progs: don't use __u8 for fsid buffers
The underscore types are for ioctl structures and should not be used for
regular code that does not need them.
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Mon, 23 Oct 2017 04:45:48 +0000 (13:45 +0900)]
btrfs-progs: fi: enable fi usage for filesystem on top of seed device
Currently "fi usage" (and "dev usage") cannot run for the filesystem
using the seed device.
This is because FS_INFO ioctl returns the number of devices excluding
seeds, but load_device_info() tries to access valid device from devid 0
to max_id, and results in accessing seeds too (thus causing mismatching
number of devices).
Since only the size of non-seed devices matters, fix this by just
skipping seed device by checking device's fsid and comparing it to the
fsid obtained by FS_INFO ioctl.
Anand Jain:
%fi_args.num_devices provides number of devices excluding the seed device.
So when looping through the device list for a given fsid, determine if the
given device is a seed device by reading its superblock and then skip it
if its a seed device. Reading of the superblock is done by the function
dev_to_fsid() which can fail if the user is not root OR if the device has
media errors as well. So skip the seed check altogether if we fail to know
the device superblock and thus the fsid.
With this now we are able to view the btrfs fi usage when the device is
bad.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Mon, 23 Oct 2017 04:44:01 +0000 (13:44 +0900)]
btrfs-progs: fi: move dev_to_fsid to cmds-fi-usage for later use
Move dev_to_fsid() from cmds-filesystem.c to cmds-fi-usage.c in order to
call it from both "fi show" and "fi usage".
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 17 Oct 2017 09:13:12 +0000 (17:13 +0800)]
btrfs-progs: rescue: Fix zero-log mounted branch
Seems to be a typo that, in (ret > 0) branch of check_mounted(),
zero-log set the return value but doesn't return.
Fix it by adding back the missing return.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Thu, 26 Oct 2017 08:32:51 +0000 (16:32 +0800)]
btrfs-progs: qgroup: fix qgroup show sort by multi items
We should not free the string until we don't call strtok any longer.
If the string is freed in advance, in fact, the second and subsequent
sort items will be ignored.
Fixes:
9fcdf8f8945b ("btrfs-progs: don't write to optarg in btrfs_qgroup_parse_sort_string")
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Mon, 16 Oct 2017 08:22:56 +0000 (16:22 +0800)]
btrfs-progs: mkfs: refactor test_minimum_size to use the calculated minimal size
test_minimum_size() function is only called to check if provided device
is large enough.
However the minimal device size only needs to be calculated once, and
can be reused everywhere.
Refactor that function to make later modification easier.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 26 Oct 2017 07:28:37 +0000 (15:28 +0800)]
btrfs-progs: convert: Open the fs readonly for rollback
For rollback, we only needs to open the fs to check if it meets the
condition to rollback. And this RW read makes us failed to rollback
btrfs with v2 space cache.
In fact, we don't even start a transaction during rollback.
So open the fs RO for rollback, to avoid v2 space cache problem.
Reported-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Gu JinXiang <gujx@cn.fujitsu.com>
Tested-by: Gu JinXiang <gujx@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:36 +0000 (13:41 +0800)]
btrfs-progs: mkfs: error out gracefully for --rootdir
--rootdir option will start a transaction to fill the fs, however if
something goes wrong, from ENOSPC to lack of permission, we won't commit
the transaction and cause BUG_ON triggered by uncommitted transaction:
------
extent buffer leak: start
29392896 len 16384
extent_io.c:579: free_extent_buffer: BUG_ON `eb->flags & EXTENT_DIRTY` triggered, value 1
------
The root fix is to introduce btrfs_abort_transaction() in btrfs-progs,
however in this particular case, we can workaround it by force
committing the transaction.
Since during mkfs, the magic of btrfs is set to an invalid one, without
setting fs_info->finalize_on_close() the fs is never able to be mounted.
So even we force to commit wrong transaction we won't screw up things
worse.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:35 +0000 (13:41 +0800)]
btrfs-progs: mkfs: fix overwritten return value for mkfs
For mkfs failure, especially --rootdir errors like EPERM/ENOSPC, the out
branch will overwrite the return value, causing wrong status code.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:34 +0000 (13:41 +0800)]
btrfs-progs: mkfs: avoid positive return value from cleanup_temp_chunks
Since we're calling btrfs_search_slot() the return value can be
positive. However we just pass that return value out, causing undefined
return value.
This can cause mkfs to return 1, which indicates something wrong.
Fix it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:33 +0000 (13:41 +0800)]
btrfs-progs: mkfs: avoid BUG_ON for chunk allocation when ENOSPC happens
When passing directory larger than block device using --rootdir
parameter, we get the following backtrace:
------
extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28
./mkfs.btrfs(+0x1a05d)[0x557939e6b05d]
./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8]
./mkfs.btrfs(+0xb0b6)[0x557939e5c0b6]
./mkfs.btrfs(main+0x15d5)[0x557939e5de04]
/usr/lib/libc.so.6(__libc_start_main+0xea)[0x7f83b101af6a]
./mkfs.btrfs(_start+0x2a)[0x557939e5af5a]
------
Nothing special, just BUG_ON() abusing from ancient code.
Fix them by using correct return.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Pavel Kretov [Tue, 24 Oct 2017 22:47:45 +0000 (01:47 +0300)]
btrfs-progs: defrag: add a brief warning about ref-link breakage
There is a warning in btrfs-filesystem(8) saying that running 'defrag'
in Linux will almost certainly break ref-links, with much data potentially
being physically duplicated.
However, many users tend to read man pages *after* trying to run things
on their own risk and may miss this important information. This commit
adds a brief copy of this warning into the command built-in help message
where it has good chances to be spotted before user is stuck with
a crowded filesystem.
Pull-request: #73
Signed-off-by: Pavel Kretov <firegurafiku@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:23:13 +0000 (01:23 +0200)]
btrfs-progs: image: move sanitization to new file
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:11:11 +0000 (01:11 +0200)]
btrfs-progs: image: pass sanitize mode and name tree separately to sanitize_name
Now sanitize_name and all callees do not depend on metadump_struct.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:11:11 +0000 (01:11 +0200)]
btrfs-progs: image: pass sanitize mode and name tree separately to sanitize_dir_item
We don't need the full mdrestore structure.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:11:11 +0000 (01:11 +0200)]
btrfs-progs: image: pass sanitize mode and name tree separately to sanitize_inode_ref
We don't need the full mdrestore structure.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:06:53 +0000 (01:06 +0200)]
btrfs-progs: image: drop unused parameter from sanitize_xattr
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:04:55 +0000 (01:04 +0200)]
btrfs-progs: image: pass rb_root to find_collisions
We don't need the full metadump structure, pass only the name tree.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 23:01:43 +0000 (01:01 +0200)]
btrfs-progs: image: introduce symbolic names for the sanitization modes
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 22:48:30 +0000 (00:48 +0200)]
btrfs-progs: image: start a new header for sanitization functions
Will be used by following cleanups, so far only the name structure could
be moved here.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 22:31:44 +0000 (00:31 +0200)]
btrfs-progs: build: use variables for btrfs-image images
More separate sources will be created for btrfs-image.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 19 Oct 2017 22:24:13 +0000 (00:24 +0200)]
btrfs-progs: image: move metadump definitions to own header
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Wed, 18 Oct 2017 02:00:43 +0000 (11:00 +0900)]
btrfs-progs: test: add new cli-test for subvol get/set-default
Add new test to check functionality of subvol get/set-default.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
[ fix style issues, add missing SUDO_HELPER ]
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 17 Oct 2017 18:27:12 +0000 (20:27 +0200)]
btrfs-progs: docs: update btrfs-properties
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Sat, 14 Oct 2017 12:54:54 +0000 (21:54 +0900)]
btrfs-progs: prop: also allow "none" to disable compression
Some people were asking why disabling compression via properties is not
set by "none" instead. As this is purely userspace conversion to "" that
kernel accepts, let's add "none" as well for convenience.
Signed-off-by: David Sterba <dsterba@suse.com>
Satoru Takeuchi [Sat, 14 Oct 2017 12:54:54 +0000 (21:54 +0900)]
btrfs-progs: allow "no" to disable compression for convenience
It's messy to use "" to disable compression. Introduce the new value "no"
which can also be used for this purpose.
Signed-off-by: Satoru Takeuchi <satoru.takeuchi@gmail.com>
[ coding style fixes ]
Signed-off-by: David Sterba <dsterba@suse.com>
Misono, Tomohiro [Fri, 6 Oct 2017 01:05:08 +0000 (10:05 +0900)]
btrfs-progs: subvol: change set-default to also accept path
This patch changes "subvol set-default" to also accept the subvolume path
for convenience.
If there are two args, they are assumed as subvol id and path to the fs
(the same as current behavior), and if there is only one arg, it is assumed
as the path to the subvolume.
subvol id is resolved by test_issubvolume() + lookup_path_rootid().
The empty subvol (ino == 2) will get error on test_issubvolume() which
checks whether inode num is 256 or not.
Issue: #35
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
[ update documentation, use the new multi-line command scheme ]
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 17 Oct 2017 17:39:47 +0000 (19:39 +0200)]
btrfs-progs: help: print multiple syntax schemas on separate lines
The help string for some commands could be split to more lines for
clarity, eg. as is now in the receive command. The 'btrfs help' listing
should indent all the lines properly, similar the command specific
help with "usage:'.
The syntax of the first help string line is to separate all command
usage schemas by "\n".
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Wed, 11 Oct 2017 17:57:16 +0000 (11:57 -0600)]
btrfs-progs: do not add stale device into fs_devices
If one of btrfs' devices was pulled out and we've replaced it with a
new one, then they have the same uuid.
If that device gets reconnected, 'btrfs filesystem show' will show the
stale one instead of the new one, but on the kernel side btrfs has a fix
not to include the stale one, this could confuse users as people may
monitor btrfs by running that command.
This does the similar thing to what kernel side has done.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
[ format string adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 27 Sep 2017 06:34:40 +0000 (14:34 +0800)]
btrfs-progs: fsck-tests: 027/bad_extent_inline_ref_type
This case is for avoiding crash in lowmem check mode.
Field type of extent_inline_ref in an extent is corrupted.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 27 Sep 2017 06:34:39 +0000 (14:34 +0800)]
btrfs-progs: check: check extent_inline_ref in lowmem
Lowmem check does not skip invalid type in extent_inline_ref and then
calls btrfs_extent_inline_ref_size(type) which causes a crash.
Error:
$ btrfs check --mode=lowmem /tmp/data_small
Checking filesystem on /tmp/data_small
UUID:
ee205d69-8724-4aa2-a4f5-
bc8558a62169
checking extents
ERROR: extent[
20971520 16384] backref type mismatch, missing bit: 2
ERROR: extent[
20971520 16384] backref generation mismatch,
wanted: 7, have: 0
ERROR: extent[
20971520 16384] is referred by other roots than 3
ctree.h:1754: btrfs_extent_inline_ref_size: BUG_ON `1` triggered,
value 1
btrfs(+0x543db)[0x55fabc2ab3db]
btrfs(+0x587f7)[0x55fabc2af7f7]
btrfs(+0x5fa44)[0x55fabc2b6a44]
btrfs(cmd_check+0x194a)[0x55fabc2bd717]
btrfs(main+0x88)[0x55fabc2682e0]
/usr/lib/libc.so.6(__libc_start_main+0xea)[0x7f021c3824ca]
btrfs(_start+0x2a)[0x55fabc267e7a]
[1] 5188 abort (core dumped) btrfs check --mode=lowmem /tmp/data_small
Fix it by introducing check_extent_inline_ref() to check the type.
If the checker returns a non-zero value, we should not try to check the
corrupted extent item anymore.
Suggested-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 27 Sep 2017 06:34:38 +0000 (14:34 +0800)]
btrfs-progs: check: error or return value of repair_root_items()
Return value of repair_root_items():
<0 on error
=0 does nothing
>0 if repair is enabled, N roots are repaired;
else N roots are corrupted.
In the repair mode, there should be no error if the return value is
bigger than 0. This fixes the test fsck/006 again.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 27 Sep 2017 08:28:39 +0000 (16:28 +0800)]
btrfs-progs: check: call repair_root_items before any repairs
The annotation of repair_root_items says:
"This must be run before any other repair code - not doing it so,
makes other repair code delete or modify backrefs in the extent tree
for example, which will result in an inconsistent fs after repairing
the root items."
However, the rule was broken by commit
1f728b1a514f ("Btrfs-progs,
fsck: move root items repair after root rebuilding").
The commit intends to fix failure of test-fsck/013 so it moves
repair_root_items() after check_extents_and_chunks().
The correct way is to skip calling repair_root_item() when
init_extent_tree is non-zero.
Now put repair_root_items() before do_check_chunks_and_extents() and
do not call repair_root_items() if init_extent_tree is set.
Then test-fsck/013 works well.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 27 Sep 2017 06:34:36 +0000 (14:34 +0800)]
btrfs-progs: check: return value of check_extent_refs
In original check mode (without option --repair), check_extent_refs()
always returns 0.
Add a variable @err to record status while checking extents. At the end
of check_extent_refs(), let it return -EIO if @err is non-zero.
The test fsck/006-bad-root-items will fail after this patch and fixed by
the following patches.
Example:
$ btrfs check bad-extent-inline-ref-type.raw
Checking filesystem on bad-extent-inline-ref-type.raw
UUID:
1942d6fe-617b-4499-9982-
cc8ffae5447f
checking extents
corrupt extent record: key
29360128 169 16384
ref mismatch on [
29360128 16384] extent item 0, found 1
Backref
29360128 parent 5 root 5 not found in extent tree
backpointer mismatch on [
29360128 16384]
bad extent [
29360128,
29376512), type mismatch with chunk
checking free space cache
checking fs roots
checking csums
checking root refs
found 114688 bytes used, no error found
total csum bytes: 0
total tree bytes: 114688
total fs tree bytes: 32768
total extent tree bytes: 16384
btree space waste bytes: 109471
file data blocks allocated: 0
referenced 0
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[ add note about the failing test, rename variable to err ]
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Tue, 12 Sep 2017 03:56:30 +0000 (11:56 +0800)]
btrfs-progs: check: repair block group accounting in lowmem mode
Add a macro named BG_ACCOUNT_ERROR meaning that block group used size
does not equal the total.
After extent-tree repair, BG_ACCOUNT_ERROR should be fixed up.
Clean bits at end of check_chunks_and_extents_v2().
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 06:30:02 +0000 (14:30 +0800)]
btrfs-progs: check: introduce repair_extent_data_item()
The only thing repair_extent_data_item() does is that it adds backref of the
tree_block. Just like what original mode does:
It first searches the corresponding extent item.
1. If the extent item exists but backref is missing, add one backref to the
extent.
2. Found nothing, just add an extent item and add one backref.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 06:26:12 +0000 (14:26 +0800)]
btrfs-progs: check: introduce repair_tree_block_ref()
The only thing repair_tree_block_ref() does is that it adds backref of the
tree_block. Just like what original repair do:
It first searches the corresponding extent item then
1. If the extent item exists but backref is missing, add one backref to the
extent.
2. if found nothing, just add an extent item and add one backref.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 06:22:37 +0000 (14:22 +0800)]
btrfs-progs: check: introduce repair_chunk_item()
Because this patchset concentrates on repair of extent tree,
repair_chunk_item() now only inserts missed chunk group item into
extent tree.
There are some things left TODO, for example dev_item fix.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 06:20:06 +0000 (14:20 +0800)]
btrfs-progs: check: delete wrong items in lowmem repair
Introduce delete_extent_tree_item() and repair_extent_item() to do
only deletion.
While checking the extent tree, just delete the wrong item. For extent
item, free wrong backref. Otherwise, delete. So the remaining items in
extent tree should be correct.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 06:12:37 +0000 (14:12 +0800)]
btrfs-progs: check: change the way lowmem mode traverses metadata
This patch is a preparation for extent-tree repair in lowmem mode.
In the lowmem mode, checking tree blocks of various trees is recursive.
But during repair, adding or deleting item(s) may modify upper nodes
which will cause the repair to be complicated and dangerous.
Before this patch:
One problem of lowmem check is that it only checks the lowest node's
backref in check_tree_block_ref.
This way ensures checked tree blocks are valid and avoids to traverse
all trees for performance reasons.
However, there is one shortcoming that it can not detect backref mistake
if one extent whose owner == offset but lacks the other backref(s).
In check, correctness is more important than speed.
If errors can not be detected, repair is impossible.
Changes in the patch:
check_chunks_and_extents now has to check *ALL* trees so lowmem check
will behave like original mode.
Changing the way of traversal to be same as fs tree which calls
walk_down_tree_v2() and walk_up_tree_v2() is easy for further
repair.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[ heavy coding style fixes ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 1 Mar 2017 01:21:51 +0000 (09:21 +0800)]
btrfs-progs: tests: Allow check test to repair in lowmem mode for certain errors
Since lowmem mode can repair certain corruptions (mostly in fs tree),
insert a beacon into each fsck test cases to allow some of them be
tested in lowmem mode.
With this patch, fsck option override will check the beacon file
".lowmem_repairable" in the same directory of the test image, and if the
beacon exists, then it will also run lowmem mode repair to repair the
image.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Fri, 1 Sep 2017 02:56:16 +0000 (10:56 +0800)]
btrfs-progs: check: special case for last item
Since repair functions will search path again, if the last item
was checked, the location where the path points is invalid.
Fix it by saving the last valid key if err contains LAST_ITEM,
and call btrfs_next_item() before return of check_inode_item().
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Tue, 29 Aug 2017 03:33:23 +0000 (11:33 +0800)]
btrfs-progs: check: punch_extent_hole in lowmem
While checking file extents, there are two errors that may occur:
1) There is one hole between the last extent end and beginning of the
current extent but no-holes is disabled.
2) No-holes is disabled, one file's nbytes equals 0 but isize is not 0.
Those both mean the file may have lost some extents.
To avoid btrfsck's error message, fix it by introducing function
'punch_extent_hole' to punch holes.
For case 1, punch a hole extent whose length is
(current extent begin - last extent end)
while checking one extent.
For case 2, punch a hole extent whose length is
(file isize - actual file size)
after traversing one entire file.
Then repair_inode_nbytes will set the nbytes to isize.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Tue, 29 Aug 2017 03:23:46 +0000 (11:23 +0800)]
btrfs-progs: check: repair inode nlink in lowmem
New function repair_inode_nlinks_lowmem() sets nlink of the inode to refs.
If refs equals 0, move the inode to lost+found and set refs to 1
initially.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Tue, 29 Aug 2017 06:11:26 +0000 (14:11 +0800)]
btrfs-progs: check: count dir inode isize again
repair_ternary_lowmem() may delete dir_item(s), later traversal can cause
wrong isize of the dirctory inode.
Introduce count_dir_iszie() to count directory isize if any
dir_item(s) in the directory has been repaired.
check_dir_item() now returns DIR_COUNT_AGAIN means the inode should be
counted isize again.
It is unnessary to do recount after check_inode_ref(), since
inode_ref is irrelevant to isize.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>