Liu Bo [Wed, 28 Jun 2017 21:29:49 +0000 (15:29 -0600)]
Btrfs-progs: convert: do not clear nodatasum flag in inode item
With the current btrfs-convert, if we convert a ext4 without data checksum,
it'd not set nodatasum flag in inode item, nor create csum item, reading
file ends up with checksum errors.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Tue, 27 Jun 2017 21:25:45 +0000 (15:25 -0600)]
Btrfs-progs: man: nodesize must be power of 2 now
This updates mkfs.btrfs's man page with the new limitation that nodesize must
be a power of 2 as well.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Thu, 22 Jun 2017 08:12:56 +0000 (16:12 +0800)]
btrfs-progs: lowmem check: Fix false alert about file extent interrupt
As Qu mentioned in this thread
(https://www.spinics.net/lists/linux-btrfs/msg64469.html), compression
can cause regular extent to co-exist with an inlined extent. This
coexistence makes things confusing. Since it is currently allowed and
can appear in a filesystem, fix btrfsck to prevent a bunch of error
reports to appear that will make user feel uneasy.
When checking a file extent, record the extent_end of the regular extent
to check if there is a gap between the regular extents. Normally there
is only one inlined extent, so the extent_end of inlined extent is
useless. However, if a regular extent can co-exist with an inlined
extent, the extent_end of the inlined extent also needs to be recorded.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 16 Jun 2017 01:43:23 +0000 (09:43 +0800)]
btrfs-progs: tests: Add test case to check file hole extents with NO_HOLES flag
Add test case which we have NO_HOLES incompat flag while still have
hole file extent.
This can be created by enabling NO_HOLES feature on an existing
filesystem, which lowmem mode would cause false alert for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 16 Jun 2017 01:43:22 +0000 (09:43 +0800)]
btrfs-progs: check: Fix false alert about EXTENT_DATA that shouldn't be a hole
Since the incompat feature NO_HOLES still allows us to have an explicit
hole file extent, current check is too strict and will cause false
alerts like:
root 5 EXTENT_DATA[257, 0] shouldn't be hole
Fix it by removing the strict file hole extent check.
Link: https://www.spinics.net/lists/linux-btrfs/msg66374.html
Reported-by: Henk Slager <eye1tm@gmail.com>
Tested-by: Henk Slager <eye1tm@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Sat, 3 Jun 2017 09:57:45 +0000 (15:27 +0530)]
btrfs-progs: convert: Add larger device support
With larger file system (in this case its 22TB), ext2fs_open() returns
EXT2_ET_CANT_USE_LEGACY_BITMAPS error message with
ext2fs_read_block_bitmap().
To overcome this issue,
(a) we need pass EXT2_FLAG_64BITS flag with ext2fs_open.
(b) use 64-bit functions like ext2fs_get_block_bitmap_range2,
ext2fs_inode_data_blocks2,ext2fs_read_ext_attr2
(c) use 64bit types with btrfs_convert_context fields
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=194795
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Mon, 26 Jun 2017 12:43:31 +0000 (14:43 +0200)]
btrfs-progs: convert: widen int types in convert context
The u32 types in the convert context might not be enough for some very
large filesytems (20TB). Use 64bit types to be safe.
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Thu, 22 Jun 2017 07:57:53 +0000 (13:27 +0530)]
btrfs-progs: Fix 'btrfs device stats --check' cli option
"Bug 194961 - btrfs device stats --check <folder> does not work"
The long option --check is not recognized as it's missing from the
option table.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=194961
Reported-by: Tomas Thiemel<thiemel@centrum.cz>
Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Thu, 22 Jun 2017 17:49:55 +0000 (12:49 -0500)]
btrfs-progs: docs: replace <dev> with <device>
As discussed in #50, use 'device' everywhere a path do a device is
expected.
Pull request: #52
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Thu, 22 Jun 2017 01:52:01 +0000 (20:52 -0500)]
btrfs-progs: docs: Wrong section in ref to manpage
Pull request: #51
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 19 Jun 2017 16:21:44 +0000 (18:21 +0200)]
btrfs-progs: build: sync recent makefile changes to android.mk
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Wed, 14 Jun 2017 05:26:08 +0000 (00:26 -0500)]
btrfs-progs: docs: Expand confusing abbreviation in documentation
While the command interpreter may be able to disambiguate the meaning,
the reader is not helped by being forced to do so.
Pull request: #48
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 19 Jun 2017 13:08:50 +0000 (15:08 +0200)]
btrfs-progs: docs: update note about device deletion
User Kasijjuf points out the VFS initialism is not explained anywhere.
While this could be fixed, the whole note about inability to delete the
device by which the filesystem has been mounted, is wrong.
Issue: #49
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Tue, 6 Jun 2017 13:00:58 +0000 (15:00 +0200)]
btrfs-progs: send operates on ro snapshots only
While talking to another btrfs user on IRC today, it became clear that a
major point of confusion in the btrfs send manual is that it's not
telling the user soon enough that send/receive solely operates on
subvolume snapshots instead of the original (read/write) subvolumes.
So, change the first few lines to explicitly mention snapshots instead.
Technically, snapshots are also just subvolumes, but requiring this
level of technical detailed knowledge doesn't help the user who is just
trying out things.
Signed-off-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 5 Jun 2017 14:39:33 +0000 (16:39 +0200)]
btrfs-progs: docs: fix sentence for no-dump file attribute
User Kasijjuf found a nonsensical sentence in the maunal page.
Issue: #45
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 31 May 2017 11:49:08 +0000 (13:49 +0200)]
btrfs-progs: docs: update formatting of btrfs-property
Reported by a wiki user, that there are formatting artifacts in the
'get' section:
in html rendered as "The -t <em><type></em> option can be..."
This is probably due to the nesting '' and <>. We don't need the <> in
the explanation, as this is only to describe the command line syntax.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 31 May 2017 11:49:08 +0000 (13:49 +0200)]
btrfs-progs: docs: update formatting of btrfs-rescue
Reported by a wiki user, that the 'zero-log' section indentaion is
wrong.
Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 17 May 2017 01:41:31 +0000 (02:41 +0100)]
btrfs-progs: test for restoring multiple devices fs into a single device
Test that we are able to create an image from a multiple devices fs, that
we are able to restore that image into a single device and finally that we
are able to mount it.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
[ added shell quotation and chmod a+w so testsuite on NFS works ]
Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 17 May 2017 01:41:21 +0000 (02:41 +0100)]
btrfs-progs: Fix restoring image from multi devices fs into single device
We correctly build an image from a multiple devices filesystem but when
restoring the image into a single device we were missing updating the
number of devices in the superblock to the value 1 (we already took care
of setting the number of stripes to 1 for each chunk item and setting
the device id for each chunk item to match the device id from the super
block).
This missing update of the number of devices makes it impossible to mount
the restored filesystem on recent kernels, more specifically since the
linux kernel commit
99e3ecfcb9f4ca35192d20a5bea158b81f600062
("Btrfs: add more validation checks for superblock"), that produce the
following message in the dmesg/syslog:
[21097.542047] BTRFS error (device sdi): super_num_devices 2 mismatch with num_devices 1 found here
[21097.543972] BTRFS error (device sdi): failed to read chunk tree: -22
[21097.720360] BTRFS error (device sdi): open_ctree failed
So fix this by updating the number of devices to 1 in the superblock.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Tsutomu Itoh [Tue, 16 May 2017 03:01:53 +0000 (12:01 +0900)]
btrfs-progs: tests: remove variable quotation from convert-tests
In btrfs-progs-v4.11-rc1, the following convert-tests failed.
[TEST/conv] 008-readonly-image
[TEST/conv] readonly image test, btrfs defaults
failed: mke2fs -t ext4 -b 4096 -F /Build/btrfs-progs-v4.11-rc1/tests/test.img
test failed for case 008-readonly-image
Makefile:271: recipe for target 'test-convert' failed
make: *** [test-convert] Error 1
[TEST/conv] 009-common-inode-flags
[TEST/conv] common inode flags test, btrfs defaults
failed: mke2fs -t ext4 -b 4096 -F /Build/btrfs-progs-v4.11-rc1/tests/test.img
test failed for case 009-common-inode-flags
Makefile:271: recipe for target 'test-convert' failed
make: *** [test-convert] Error 1
So, remove quotes from $default_mke2fs.
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 18 May 2017 14:16:03 +0000 (16:16 +0200)]
Btrfs progs v4.11
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 31 Mar 2017 12:20:21 +0000 (14:20 +0200)]
btrfs-progs: update CHANGES for v4.11
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 15 May 2017 14:34:31 +0000 (16:34 +0200)]
btrfs-progs: tests: fix misc/019-receive-clones-on-munted-subvol
Patches "btrfs-progs: tests: correctly receive clones to mounted subvol"
(
8eaf63bc9a7b957c566be23df7c6701a9a5b22a2) and followup are missing last
unmount which leads to failure of misc/020.
Signed-off-by: David Sterba <dsterba@suse.com>
Christian Brauner [Sat, 29 Apr 2017 21:54:05 +0000 (23:54 +0200)]
btrfs-progs: receiv: fail on first -ENODATA only
Returning -ENODATA is only considered invalid on the first run of the
loop where we would detect entirely empty stream.
The enhanced test misc-tests/018-recv-end-of-stream now passes.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=195597
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Mon, 8 May 2017 07:38:10 +0000 (15:38 +0800)]
btrfs-progs: print-tree: Add leaf flags and backref revision output
Btrfs header has a u64 member flags, whose lowest 56 bits are for header
flags like WRITTEN and RELOC.
And its highest 8 bits are for backref revision.
Manually checking btrfs_header_flags() will be a pain, so add such leaf
flags and backref revision output for print-tree.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 May 2017 08:42:40 +0000 (16:42 +0800)]
btrfs-progs: check: Avoid reading beyond item boundary for dir_item and dir_index
When reading out name from inode_ref, it's possible that corrupted
name_len can lead to read beyond boundary of item or even extent buffer.
This happens when checking fuzzed image /tmp/bko-161811.raw, for both
lowmem mode and original mode.
Below is the example from lowmem mode.
ERROR: root 5 INODE REF[256 256] doesn't have related DIR_INDEX[256
216172782113783808] namelen 255 filename bar filetype 0
ERROR: root 5 INODE REF[256 256] doesn't have related DIR_ITEM[256
1306590535] namelen 255 filename bar filetype 0
WARNING: root 5 INODE[256] mode 0 shouldn't have DIR_INDEX[256
1167283096]
WARNING: root 5 DIR_ITEM[256
1167283096] name too long
==13013== Invalid read of size 1
==13013== at 0x4C31A38: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13013== by 0x431518: read_extent_buffer (extent_io.c:863)
==13013== by 0x4752AB: check_dir_item (cmds-check.c:4627)
==13013== by 0x475E5C: check_inode_item (cmds-check.c:4911)
==13013== by 0x476200: check_fs_first_inode (cmds-check.c:5011)
==13013== by 0x476276: check_fs_root_v2 (cmds-check.c:5044)
==13013== by 0x4769FB: check_fs_roots_v2 (cmds-check.c:5242)
==13013== by 0x488B5B: cmd_check (cmds-check.c:13033)
==13013== by 0x40A8C5: main (btrfs.c:246)
==13013== Address 0x5c95b80 is 0 bytes after a block of size 4,224 alloc'd
==13013== at 0x4C2CF35: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13013== by 0x4307E0: __alloc_extent_buffer (extent_io.c:538)
==13013== by 0x430C37: alloc_extent_buffer (extent_io.c:642)
==13013== by 0x413DFE: btrfs_find_create_tree_block (disk-io.c:193)
==13013== by 0x414370: read_tree_block_fs_info (disk-io.c:340)
==13013== by 0x40B5D5: read_tree_block (disk-io.h:125)
==13013== by 0x40CFD2: read_node_slot (ctree.c:652)
==13013== by 0x40E5EB: btrfs_search_slot (ctree.c:1172)
==13013== by 0x4761A8: check_fs_first_inode (cmds-check.c:5001)
==13013== by 0x476276: check_fs_root_v2 (cmds-check.c:5044)
==13013== by 0x4769FB: check_fs_roots_v2 (cmds-check.c:5242)
==13013== by 0x488B5B: cmd_check (cmds-check.c:13033)
Fix it by double checking dir_item, name_len against item boundary
before trying to read out name from extent buffer, for both original
mode and lowmem mode.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 May 2017 08:42:39 +0000 (16:42 +0800)]
btrfs-progs: check: Avoid reading beyond item boundary for inode_ref
When reading out name from inode_ref, it's possible that corrupted
name_len can lead to read beyond boundary of item or even extent buffer.
This happens when checking fuzzed image /tmp/bko-161811.raw, for both
lowmem mode and original mode.
ERROR: root 5 INODE REF[256 256] doesn't have related DIR_INDEX[256
504403158265495680] namelen 0 filename filetype 0
ERROR: root 5 INODE REF[256 256] doesn't have related DIR_ITEM[256
4294967294] namelen 0 filename filetype 0
WARNING: root 5 INODE_REF[256 256] name too long
==13022== Invalid read of size 8
==13022== at 0x4C319BE: memmove (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13022== by 0x431518: read_extent_buffer (extent_io.c:863)
==13022== by 0x474730: check_inode_ref (cmds-check.c:4307)
==13022== by 0x475D65: check_inode_item (cmds-check.c:4890)
==13022== by 0x476200: check_fs_first_inode (cmds-check.c:5011)
==13022== by 0x476276: check_fs_root_v2 (cmds-check.c:5044)
==13022== by 0x4769FB: check_fs_roots_v2 (cmds-check.c:5242)
==13022== by 0x488B5B: cmd_check (cmds-check.c:13033)
==13022== by 0x40A8C5: main (btrfs.c:246)
==13022== Address 0x5c96780 is 0 bytes after a block of size 4,224 alloc'd
==13022== at 0x4C2CF35: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==13022== by 0x4307E0: __alloc_extent_buffer (extent_io.c:538)
==13022== by 0x430C37: alloc_extent_buffer (extent_io.c:642)
==13022== by 0x413DFE: btrfs_find_create_tree_block (disk-io.c:193)
==13022== by 0x414370: read_tree_block_fs_info (disk-io.c:340)
==13022== by 0x40B5D5: read_tree_block (disk-io.h:125)
==13022== by 0x40CFD2: read_node_slot (ctree.c:652)
==13022== by 0x40E5EB: btrfs_search_slot (ctree.c:1172)
==13022== by 0x4761A8: check_fs_first_inode (cmds-check.c:5001)
==13022== by 0x476276: check_fs_root_v2 (cmds-check.c:5044)
==13022== by 0x4769FB: check_fs_roots_v2 (cmds-check.c:5242)
==13022== by 0x488B5B: cmd_check (cmds-check.c:13033)
=
Fix it by double checking inode_ref, name_len against item boundary
before trying to read out name from extent buffer, for both original
mode and lowmem mode.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 3 May 2017 01:50:14 +0000 (09:50 +0800)]
btrfs-progs: check: Fix heap use after free
fsck/004-no-dir-index makes valgrinds complaining about Invalid read.
==31890== Invalid read of size 1
==31890== at 0x453D09: repair_inode_backrefs (cmds-check.c:2690)
==31890== by 0x453D09: check_inode_recs (cmds-check.c:3330)
==31890== by 0x453D09: check_fs_root (cmds-check.c:4012)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890== Address 0x5cb7b90 is 16 bytes inside a block of size 50 free'd
==31890== at 0x4C2C14B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31890== by 0x453D08: repair_inode_backrefs (cmds-check.c:2684)
==31890== by 0x453D08: check_inode_recs (cmds-check.c:3330)
==31890== by 0x453D08: check_fs_root (cmds-check.c:4012)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890== Block was alloc'd at
==31890== at 0x4C2AF1F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31890== by 0x45055C: get_inode_backref (cmds-check.c:1075)
==31890== by 0x45055C: add_inode_backref (cmds-check.c:1097)
==31890== by 0x45180C: process_dir_item (cmds-check.c:1525)
==31890== by 0x45180C: process_one_leaf (cmds-check.c:1838)
==31890== by 0x45180C: walk_down_tree (cmds-check.c:2134)
==31890== by 0x45180C: check_fs_root (cmds-check.c:3957)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890==
==31890== Invalid read of size 8
==31890== at 0x452D66: repair_inode_backrefs (cmds-check.c:2731)
==31890== by 0x452D66: check_inode_recs (cmds-check.c:3330)
==31890== by 0x452D66: check_fs_root (cmds-check.c:4012)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890== Address 0x5cb7b90 is 16 bytes inside a block of size 50 free'd
==31890== at 0x4C2C14B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31890== by 0x453D08: repair_inode_backrefs (cmds-check.c:2684)
==31890== by 0x453D08: check_inode_recs (cmds-check.c:3330)
==31890== by 0x453D08: check_fs_root (cmds-check.c:4012)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890== Block was alloc'd at
==31890== at 0x4C2AF1F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==31890== by 0x45055C: get_inode_backref (cmds-check.c:1075)
==31890== by 0x45055C: add_inode_backref (cmds-check.c:1097)
==31890== by 0x45180C: process_dir_item (cmds-check.c:1525)
==31890== by 0x45180C: process_one_leaf (cmds-check.c:1838)
==31890== by 0x45180C: walk_down_tree (cmds-check.c:2134)
==31890== by 0x45180C: check_fs_root (cmds-check.c:3957)
==31890== by 0x45E788: check_fs_roots (cmds-check.c:4098)
==31890== by 0x45E788: cmd_check (cmds-check.c:13031)
==31890== by 0x40A88A: main (btrfs.c:246)
==31890==
While iterating over backrefs in repair_inode_backrefs, there are
several situations to repair one backref according
backref->found_dir_item and backref->found_dir_index. Two of these
branches may free the backref, but next checks will still access the
freed memory.
Because these branches are independent, let repair_inode_backrefs skip
to handle next backref after free can fix it.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Sun, 23 Apr 2017 19:00:00 +0000 (00:30 +0530)]
btrfs-progs: tests: add variable quotation to convert-tests
Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Christophe de Dinechin [Fri, 28 Apr 2017 09:51:21 +0000 (11:51 +0200)]
btrfs-progs: check: make max_size consistent with nr
Since we memset tmpl, max_size==0. This does not seem consistent with nr = 1.
In check_extent_refs, we will call:
set_extent_dirty(root->fs_info->excluded_extents,
rec->start,
rec->start + rec->max_size - 1);
This ends up with BUG_ON(end < start) in insert_state.
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Christophe de Dinechin [Fri, 28 Apr 2017 09:50:23 +0000 (11:50 +0200)]
btrfs-progs: check: prevent attempt to insert extent record with max_size==0
When this happens, we will trip a BUG_ON(end < start) in insert_state
because in check_extent_refs, we use this max_size expecting it's not zero:
set_extent_dirty(root->fs_info->excluded_extents,
rec->start,
rec->start + rec->max_size - 1);
See https://bugzilla.redhat.com/show_bug.cgi?id=1435567
for an example where this scenario occurs.
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Christophe de Dinechin [Fri, 28 Apr 2017 09:12:35 +0000 (11:12 +0200)]
btrfs-progs: check: disambiguate between cases where add_tree_backref fails
See https://bugzilla.redhat.com/show_bug.cgi?id=1435567 for an example
where the message occurs.
Signed-off-by: Christophe de Dinechin <dinechin@redhat.com>
[ un-indent strings overfowing 80 cols ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 25 Apr 2017 08:40:16 +0000 (16:40 +0800)]
btrfs-progs: Use more strict check to read out tree root
Fuzzed image bko-156811-bad-parent-ref-qgroup-verify.raw causes qgroup
to report -ENOMEM.
But the fact is, such image is heavily damaged so there is no valid root
item for the extent tree.
Normal extent tree key in root tree should be (EXTENT_TREE ROOT_ITEM 0),
while in that fuzzed image, we got (EXTENT_TREE EXXTENT_DATA SOME_NUMBER).
It's btrfs_find_last_root() that only checks the objectid, not caring
about the key type leading to such problem.
Fix it by doing extra check on key type.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ edit changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Tue, 2 May 2017 07:36:09 +0000 (15:36 +0800)]
btrfs-progs: tests: Fix fuzz-test for bko-161821.raw.txt
Fuzzed image bko-161821.raw causes btrfs check to get segmentation fault.
The function check_owner_ref attempts to access a non-exist quota tree
when dealing with extent_item [4198400 4096] in the corrupted filesystem.
The function btrfs_new_fs_info always allocates memory for
fs_info->quota_root regardless of whether quota_tree exists or not.
Additionally, the function btrfs_read_fs_root will directly return
fs_info->quota_root if location->objectid == BTRFS_QUOTA_TREE_OBJECTID.
This patch does the following things:
1. Do extra check and return ENOENT if quota tree does not exist in the
function btrfs_read_fs_root.
2. Free useless fs_info->quota_root in the function btrfs_setup_all_roots
to reduce confusion.
3. free_extent_buffer even if check_child_node failed in the function
walk_down_tree.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 2 May 2017 03:17:25 +0000 (11:17 +0800)]
btrfs-progs: check: Fix memory leak in check_chunks_and_extents
fsck/003-shift-offsets makes valgrinds complaining about memory leaks.
==5910==
==5910== HEAP SUMMARY:
==5910== in use at exit: 1,112 bytes in 11 blocks
==5910== total heap usage: 161 allocs, 150 frees, 164,800 bytes allocated
==5910==
==5910== 216 (72 direct, 144 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 5
==5910== at 0x4C2AF1F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==5910== by 0x4815A3: add_root_item_to_list (cmds-check.c:9683)
==5910== by 0x481CE2: check_chunks_and_extents (cmds-check.c:9886)
==5910== by 0x48888B: cmd_check (cmds-check.c:12977)
==5910== by 0x40A8C5: main (btrfs.c:246)
==5910==
The check_chunks_and_extents() memory leaks are caused by not freeing
added root items of normal_trees and dropping_trees.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 2 May 2017 14:23:54 +0000 (16:23 +0200)]
btrfs-progs: fssum: fix warning, include correct header for major()
tests/fssum.c:599:13: warning: In the GNU C Library, "major" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "major", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"major", you should undefine it after including <sys/types.h>.
sum_add_u64(&cs, major(st.st_rdev));
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Wed, 26 Apr 2017 01:22:41 +0000 (09:22 +0800)]
btrfs-progs: tests: fssum, fix memory leak
Free the alloced memory and close dir before exit.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 25 Apr 2017 08:01:17 +0000 (16:01 +0800)]
btrfs-progs: Fix memory leak when 0 sized block group item is found
When a 0 sized block group item is found, set_extent_bits() will not
really set any bits.
While set_state_private() still inserts allocated block group cache into
block group extent_io_tree.
So at close_ctree() time, we won't free the private block group cache
stored since we can't find any bit set for the 0 sized block group.
To fix it, at btrfs_read_block_groups() we skip any 0 sized block group,
so such leak won't happen.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Mon, 24 Apr 2017 13:29:40 +0000 (18:59 +0530)]
btrfs-progs: tests: add variable quotation to fsck-tests
Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 20 Apr 2017 16:50:02 +0000 (18:50 +0200)]
btrfs-progs: docs: update formatting and wording for btrfs(5)
- emphasize features in the list
- minor whitespace fixes
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Thu, 20 Apr 2017 08:07:57 +0000 (16:07 +0800)]
btrfs-progs: print-tree: add validation to print_chunk
In print_chunk, validate the value of uuid_offset when read the dev_uuid of
stripe.
Was triggered by misc-test/015-dump-super-garbage running indefinetelly.
Issue: #37
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Thu, 20 Apr 2017 08:07:56 +0000 (16:07 +0800)]
btrfs-progs: dump-super: check array_size in print_sys_chunk_array
Without validation of array_size, the dump-super may lead to a bad
memory access.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Evan Danaher [Tue, 11 Apr 2017 16:33:40 +0000 (12:33 -0400)]
btrfs-progs: send: always print a space after path in dump
I was shocked to discover that 'btrfs receive --dump' doesn't print a
space after long filenames, so it runs together into the metadata; for
example:
truncate ./20-00-03/this-name-is-32-characters-longsize=0
This is a trivial patch to add a single space unconditionally, so the
result is the following:
truncate ./20-00-03/this-name-is-32-characters-long size=0
I suppose this is technically a breaking change, but it seems unlikely
to me that anyone would depend on the existing behavior given how
unfriendly it is.
Signed-off-by: Evan Danaher <github@edanaher.net>
Reviewed-by: Noah Massey <noah.massey@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Sat, 15 Apr 2017 10:13:08 +0000 (15:43 +0530)]
btrfs-progs: misc-tests: Superblock corruption and recovery using backup
Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Sun, 16 Apr 2017 17:20:02 +0000 (19:20 +0200)]
btrfs-progs: docs: Fix newlines for man btrfstune
A bunch of newlines were missing, which resulted in only -S and -r to
show as option after xmlto is used to convert the documentation to a man
page.
The rest of the options would end up being appended to the explanation
of -r.
Signed-off-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 13 Apr 2017 21:56:55 +0000 (23:56 +0200)]
btrfs-progs: balance: minor wording adjustment for full balance warning
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Sat, 8 Apr 2017 22:37:47 +0000 (00:37 +0200)]
btrfs-progs: docs: Fix missing newline in man 5 btrfs
The text compress_lzo:: would show up directly after 'bigger than the
page size' on the same line.
Signed-off-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Christian Brauner [Mon, 3 Apr 2017 20:21:08 +0000 (22:21 +0200)]
btrfs-progs: tests: misc/018-receive use receive -e to terminate on end marker
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
[ enhance tests to take extra options and use for -e ]
Signed-off-by: David Sterba <dsterba@suse.com>
Christian Brauner [Mon, 3 Apr 2017 20:21:07 +0000 (22:21 +0200)]
btrfs-progs: fix btrfs send & receive with -e flag
The old check here tried to ensure that empty streams are not considered valid.
The old check however, will always fail when only one run through the while(1)
loop is needed and honor_end_cmd is set. So this:
btrfs send /some/subvol | btrfs receive -e /some/
will consistently fail because -e causes honor_cmd_to be set and
btrfs_read_and_process_send_stream() to correctly return 1. So the command will
be successful but btrfs receive will error out because the send - receive
concluded in one run through the while(1) loop.
If we want to exclude empty streams we need a way to tell the difference between
btrfs_read_and_process_send_stream() returning 1 because read_buf() did not
detect any data and read_and_process_cmd() returning 1 because honor_end_cmd was
set. Without introducing too many changes the best way to me seems to have
btrfs_read_and_process_send_stream() return -ENODATA in the first case. The rest
stays the same. We can then check for -ENODATA in do_receive() and report a
proper error in this case. This should also be backwards compatible to previous
versions of btrfs receive. They will fail on empty streams because a negative
value is returned. The only thing that they will lack is a nice error message.
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 7 Apr 2017 16:21:07 +0000 (12:21 -0400)]
btrfs-progs: docs: btrfs-replace, fix typo
Github pull request #39, from gabo77.
Author: 3psus <gabo77@me.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 3 Apr 2017 17:47:10 +0000 (19:47 +0200)]
btrfs-progs: README: sort bug reports means by preference
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 21 Nov 2016 12:40:43 +0000 (13:40 +0100)]
btrfs-progs: rework option parser to use getopt for global options
Preparatory work to support more global options. The current parser
abuses the subcommand table to understand help and version when
specified as options (--). These are now special case when processing
the global options.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 31 Mar 2017 17:40:05 +0000 (19:40 +0200)]
btrfs-progs: docs: mount options, enhance ssd/nossd
Reported-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 31 Mar 2017 15:29:12 +0000 (17:29 +0200)]
Btrfs progs v4.10.2
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 31 Mar 2017 12:20:21 +0000 (14:20 +0200)]
btrfs-progs: update CHANGES for v4.10.2
Signed-off-by: David Sterba <dsterba@suse.com>
Sergei Trofimovich [Mon, 27 Mar 2017 21:07:20 +0000 (22:07 +0100)]
btrfs-progs: fix missing __error symbol in libbtrfs.so.0
The easiest way to reproduce the error is to try to build
btrfs-progs with
$ make LDFLAGS=-Wl,--no-undefined
btrfs-list.o: In function `lookup_ino_path':
btrfs-list.c:(.text+0x7d2): undefined reference to `__error'
Noticed by Denis Descheneaux when snapper tool
stopped working after upgrade to btrfs-progs-4.10.
As soname didn't change in 4.9 -> 4.10 release
I assume it's just an object file omission
in library depends and not the API/ABI change
of the library error printing.
Cc: linux-btrfs@vger.kernel.org
Cc: Mike Gilbert <floppym@gentoo.org>
Reported-by: Denis Descheneaux
Bug: https://bugs.gentoo.org/show_bug.cgi?id=613890
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 17 Mar 2017 02:06:34 +0000 (10:06 +0800)]
btrfs-progs: tests: Add SHARED_DATA_REF test image for check lowmem mode
Introduce a new image, which contains external SHARED_DATA_REF items to
trigger a lowmem mode false alert.
The image only contains external SHARED_DATA_REF and no inlined data
backref.
Before the image, we only have inlined shared data ref, which is not
enough to trigger lowmem mode false alert.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 17 Mar 2017 02:06:33 +0000 (10:06 +0800)]
btrfs-progs: check: lowmem, fix false alert about backref lost for SHARED_DATA_REF
In check_extent_data_item(), after checking extent item of one data
extent, we search inlined data backref, then EXTENT_DATA_REF_KEY.
But we didn't search SHARED_DATA_REF, so if the backref is
SHARED_DATA_REF, then we will raise a false alert about backref lost.
Fix by also checking SHARED_DATA_REF_KEY in check_extent_data_item().
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 21 Mar 2017 01:00:55 +0000 (09:00 +0800)]
btrfs-progs: convert: Add missing return for HOLE mode when checking convert image
In check_convert_image(), for normal HOLE case, if the file extents are
smaller than image size, we set ret to -EINVAL and print error message.
But forget to return.
This patch adds the missing return to fix it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 27 Mar 2017 12:14:20 +0000 (14:14 +0200)]
btrfs-progs: add prefix to message helpers
The implementation of message helpers use very generic names so we
should at least use a prefix, as they're going to be usied from within
the library. The build fix will follow.
Reported-by: Mike Gilbert <floppym@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Mon, 27 Mar 2017 00:52:09 +0000 (08:52 +0800)]
btrfs-progs: Cleanup kernel-shared dir when execute make clean
Reported-by: Chris <coderight@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 27 Mar 2017 11:55:01 +0000 (13:55 +0200)]
btrfs-progs: library-test: add all exported headers
Explicitly include all headers exported to library, as a light build
check that the files are really present.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 17 Mar 2017 13:11:22 +0000 (14:11 +0100)]
Btrfs progs v4.10.1
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 16 Mar 2017 16:11:20 +0000 (17:11 +0100)]
btrfs-progs: update CHANGES for v4.10.1
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 16 Mar 2017 03:18:31 +0000 (11:18 +0800)]
btrfs-progs: convert: Make btrfs_reserved_ranges const
Since btrfs_reserved_ranges array is just used to store btrfs reserved
ranges, no one will nor should modify them at run time, make them static
and const will be better.
This also eliminates the use of immediate number 3.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ definition stays in source-fs.c ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 16 Mar 2017 03:18:30 +0000 (11:18 +0800)]
btrfs-progs: kerncompat: Fix re-definition of __bitwise
In latest linux api headers, __bitwise is already defined in
/usr/include/linux/types.h.
So kerncompat.h will re-define __bitwise, and cause gcc warning.
Fix it by checking if __bitwise is already define.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 14 Mar 2017 05:35:18 +0000 (13:35 +0800)]
btrfs-progs: dump-tree: Also output log root tree
In btrfs-dump-tree, we output any existing log tree, however we don't
output the log root tree, which records all root items for log trees.
This makes it confusing for any one who want to know where the log tree
comes from.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 14 Mar 2017 05:28:27 +0000 (13:28 +0800)]
btrfs-progs: dump-tree: Fix duplicated output when using -t option
When using -t option to output trees not in root tree (chunk/root/log
root), then we output the tree twice.
Fix it
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 15 Mar 2017 11:28:16 +0000 (12:28 +0100)]
btrfs-progs: tests: misc/019, use fssum
The fssum utility is now ready to use, test still passes.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 15 Mar 2017 11:21:50 +0000 (12:21 +0100)]
btrfs-progs: tests: fssum, switch from MD5 to SHA256
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 18:30:47 +0000 (19:30 +0100)]
btrfs-progs: tests: add SHA256
In order to drop dependency on SSL library to compute MD5 in fssum,
we'll use the reference implementation from RFC 6234.
The checksum is not in a cryptographically sensitive context, but we're
going to skip MD5 and SHA-1 anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 18:09:29 +0000 (19:09 +0100)]
btrfs-progs: tests: fssum, update includes
Remove unneeded and add headers that define time_te and int types.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 18:04:51 +0000 (19:04 +0100)]
btrfs-progs: tests: fssum, drop *_SOURCE defines, already provided via config.h
In file included from /usr/include/stdio.h:27:0,
from kerncompat.h:22,
from tests/fssum.c:25:
/usr/include/features.h:148:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
^~~~~~~
We've solved that long time ago and config.h now provides the macros.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 18:01:35 +0000 (19:01 +0100)]
btrfs-progs: tests: fssum, use our endianity helper
The function htonll is not provided by the standard library and we can
replace it by our cpu-to-XX helpers. This switches the endianity of the
checksummed value to LE, but this is not a problem.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 17:56:22 +0000 (18:56 +0100)]
btrfs-progs: tests: fssum, drop unnecessray build defines
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Mar 2017 17:53:29 +0000 (18:53 +0100)]
btrfs-progs: tests: add fssum utility
Copy from fstests, originally from
git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git
Needs libcrypto to link but this check is now missing in configure.
Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 17:07:28 +0000 (18:07 +0100)]
btrfs-progs: tests: cleanup use of IMAGE and replace by helpers
Nearly each use of IMAGE can be replaced by common helpers as there are
no specific requirements on the testing filesystem. There are still a
few left that need to be evaluated and converted eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 16:52:20 +0000 (17:52 +0100)]
btrfs-progs: tests: mkfs/002 and 003: use TEST_DEV instead of IMAGE
As we don't know what's the TEST_DEV like, use the explicit mkfs limit.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 16:48:55 +0000 (17:48 +0100)]
btrfs-progs: tests: use TEST_DEV in cli-tests
IMAGE points to the fallback image, tests should use TEST_DEV.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 16:43:05 +0000 (17:43 +0100)]
btrfs-progs: docs: add preliminary do's & dont's for tests
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 16:15:24 +0000 (17:15 +0100)]
btrfs-progs: tests: do test build of library-test.static
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 16:01:28 +0000 (17:01 +0100)]
btrfs-progs: library-test: update test to build outside of topdir
We need to build outside of the topdir so we can use the "btrfs/" prefix
for includes and not accidentally include other files.
Make magic is simple:
- build dependencies inside TOPDIR
- build inside temporary directory, link back to TOPDIR
- library-test.o not built anymore obviously
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 15:52:34 +0000 (16:52 +0100)]
btrfs-progs: library-test: update header inclusion
The library-test is supposed to compile and link as an external
program, so we should update the way the headers are included.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 13 Mar 2017 15:08:51 +0000 (16:08 +0100)]
btrfs-progs: library: distribute sizes.h, fix build
The patch "btrfs-progs: Introduce kernel sizes to cleanup large
intermediate number" (
a2203246ae483d09) was taken from kernel but not
properly ported so the build breaks because the header linux/sizes.h is
not exported.
The build tests of library do not cover the case when the macro
BTRFS_FLAT_INCLUDES is not defined (ie. an external build).
Reported-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Baruch Siach [Sun, 12 Mar 2017 17:46:52 +0000 (19:46 +0200)]
btrfs-progs: add missing limits headers
Build under musl libc fails because of missing PATH_MAX and XATTR_NAME_MAX
macro declarations. Add the required headers.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Qu Wenruo [Thu, 23 Feb 2017 08:21:14 +0000 (16:21 +0800)]
btrfs-progs: convert: Rework rollback
Rework rollback to a more easy to understand way.
New convert behavior makes us to have a more flex chunk layout, which
only data chunk containing old fs data will be at the same physical
location, while new chunks (data/meta/sys) can be mapped anywhere else.
This behavior makes old rollback behavior can't handle it.
As old behavior assumes all data/meta is mapped in a large chunk, which is
mapped 1:1 on disk.
So rework rollback to handle new convert behavior, enhance the check by
only checking all file extents of convert image, only to check if these
file extents and therir chunks are mapped 1:1.
This new rollback check behavior can handle both new and old convert
behavior, as the new behavior is a superset of old behavior.
Further more, introduce a simple rollback mechanisim:
1) Read reserved data (offset = file offset) from convert image
2) Write reserved data into disk (offset = physical offset)
Since old fs image is a valid fs, and we only need to rollback
superblocks (btrfs reserved ranges), then we just read out data in
reserved range, and write it back.
Due to the fact that all other file extents of converted image is mapped
1:1 on disk, we put the missing piece back, then the fs is as good as
old one.
Then what we do in btrfs is just another dream.
With this new rollback mechanisim, we can open btrfs read-only, so we
won't cause any damage to current btrfs, until the final piece (0~1M,
containing 1st super block) is put back.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ port to v4.10 ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Feb 2017 07:46:46 +0000 (15:46 +0800)]
btrfs-progs: convert: Introduce function to check if convert image is able to be rolled back
Introduce a function, check_convert_image() to check if that image is
rollback-able.
This means all file extents except one of the btrfs reserved ranges, must
be mapped 1:1 on disk.
1:1 mapped file extents must match the following conditions:
1) Their file_offset(key.offset) matches its disk_bytenr
2) The corresponding chunk must be mapped 1:1 on disk
That's to say, it's a SINGLE chunk, and chunk logical matches with
stripe physical.
Above 2 conditions ensured that file extent lies the exactly the same
position as in the old filesystem.
For data in reserved ranges of btrfs, they are relocated to new places,
and in that case, we use btrfs_read_file() to read out the content for
later rollback use.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 23 Feb 2017 07:17:16 +0000 (15:17 +0800)]
btrfs-progs: convert: Introduce function to read out btrfs reserved range
Introduce a new function, read_reserved_ranges(), to allow later
rollback to use these data to do rollback.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 23 Feb 2017 05:53:25 +0000 (13:53 +0800)]
btrfs-progs: file: Introduce function to read out file content
Introduce a new function, btrfs_read_file(), to read out data of a file
inode.
This function will iterate through EXTENT_DATA items and handle
inline/prealloc/hole file extents.
Compression is not supported yet.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Feb 2017 02:22:22 +0000 (10:22 +0800)]
btrfs-progs: convert: Use reserved ranges array to cleanup open code
Since we have reserved ranges array now, we can use them to skip all
these open codes.
And add some comment and asciidoc art for related part.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ port to v4.10 ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Feb 2017 08:46:18 +0000 (16:46 +0800)]
btrfs-progs: convert: Introduce simple range structure for convert reserved ranges
Introduce a new strucutre, simple_range, to present one contingous
range.
Also, use such structure to define btrfs_reserved_ranges(), which
convert and rollback will use.
Suggested-by: David Sterba <dsterba@suse.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ split hunks to new file structure ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 22 Feb 2017 07:50:32 +0000 (15:50 +0800)]
btrfs-progs: convert: Add comment for the overall convert design
Convert is now a little complex due to that fact we need to separate
metadata and data chunks for different profiles.
Add a comment with ascii art explaining the whole design and point
out the really complex part, so any newcomers interested in convert can
get a quick overview of it before digging into the hard to read code.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ wording and formatting adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 8 Mar 2017 14:51:57 +0000 (15:51 +0100)]
btrfs-progs: tests: port 019-receive-clones-on-munted-subvol
Port from fstests.
Signed-off-by: David Sterba <dsterba@suse.com>
Benedikt Morbach [Wed, 22 Feb 2017 23:07:35 +0000 (00:07 +0100)]
btrfs-progs: tests: correctly receive clones to mounted subvol
clone needs to resolve the paths of the involved subvolumes in the target
fs from their UUIDs. When doing so it might need to strip the prefix
that is mounted as the root of the fs from those paths.
It didn't do so correctly when processing the source of "clone" commands
This is a regression test for
btrfs-progs: receive: handle root subvol path in clone
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
[ copied the fstests version, will be updated later ]
Signed-off-by: David Sterba <dsterba@suse.com>
Benedikt Morbach [Wed, 22 Feb 2017 22:56:37 +0000 (23:56 +0100)]
btrfs-progs: receive: handle root subvol path in clone
testcase:
# ro subvol /src/parent
# rw subvol /src/foo
clone /src/parent/file /src/foo/file
subvol snapshot -r /src/foo /src/foo.snap
# generates a "clone parent/file -> foo.snap/file" send command
send -p /src/parent /src/foo.snap
# target fs:
# dest/
# |--- parent/...
# mounted with -o subvol=dest, such that "parent" is at <target>/parent
receive <target>
result:
ERROR: cannot open dest/parent/file: No such file or directory
expected:
"dest/" get's stripped from the clone source path to get the actual
path in the target fs, if reachable from the mount point/chroot.
This is exactly what process_snapshot does, which gets called on
_every_ incremental receive and I'm quite certain is correct in
doing so
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Benedikt Morbach [Wed, 22 Feb 2017 22:56:36 +0000 (23:56 +0100)]
btrfs-progs: receive: better error reporting for snapshots
Two fixes:
1)
Check that the parent subvol actually is reachable via our root path.
The previous code wouldn't catch
parent subvol: foo/bar
root path: bar (i.e. mounted with -o subvol=bar)
where the parent isn't reachable from the root path.
(but the original "strstr(parent, root_path) == NULL" check still doesn't hold)
Also check for the slash after "root_path", i.e. throw an error on
parent subvol: foobar
root path: foo
2)
If the parent subvol is the one that is mounted we obviously can't
receive into it, as it has to be read-only by definition.
We'd get a rather cryptic:
At subvol /tmp/test/dest.snap
At snapshot dest.snap
ERROR: creating snapshot / -> dest.snap failed: Invalid cross-device link
(not sure what it says if "/" isn't even a btrfs)
But with this we get
At subvol /tmp/test/dest.snap
At snapshot dest.snap
ERROR: creating snapshot . -> dest.snap failed: Read-only file system
which is both more helpful and more correct.
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 8 Mar 2017 13:23:24 +0000 (14:23 +0100)]
Btrfs progs v4.10
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 2 Mar 2017 13:36:25 +0000 (14:36 +0100)]
btrfs-progs: update CHANGES for v4.10
Signed-off-by: David Sterba <dsterba@suse.com>
Omar Sandoval [Fri, 3 Mar 2017 17:02:14 +0000 (09:02 -0800)]
btrfs-progs: report I/O errors when closing the filesystem
If the final fsync() on the Btrfs device fails, we just swallow the
error and don't alert the user in any way. This was uncovered by xfstest
generic/405, which checks that mkfs fails when it encounters EIO.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Mar 2017 16:16:46 +0000 (17:16 +0100)]
btrfs-progs: tests: use dd in 025-file-extents for creating files
Drop the dependency on xfs_io as it's not a standard tool, though it
provides convenience. We use a simple write here so dd can manage.
Signed-off-by: David Sterba <dsterba@suse.com>