platform/upstream/btrfs-progs.git
7 years agobtrfs-progs: remove extra newline from messages
David Sterba [Fri, 28 Oct 2016 16:47:35 +0000 (18:47 +0200)]
btrfs-progs: remove extra newline from messages

The common message helpers add the newline.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: fix NULL pointer dereference for possible memory allocation failure
Qu Wenruo [Mon, 24 Oct 2016 02:43:35 +0000 (10:43 +0800)]
btrfs-progs: check: fix NULL pointer dereference for possible memory allocation failure

We didn't check 'path' allocated in check_root_ref(), which can cause
NULL pointer dereference if the memory allocation failed.

Fix it by using stack memory, since the function should return error
bitmap not minus error code, we don't want memory allocation to be an
exception.

Resolves-Coverity-CID: 1372510
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: fsck: Fix patch allocation check and leak in check_fs_first_inode
Qu Wenruo [Mon, 24 Oct 2016 02:43:33 +0000 (10:43 +0800)]
btrfs-progs: fsck: Fix patch allocation check and leak in check_fs_first_inode

Allocated 'path' in check_fs_first_inode() is not checked and for
btrfs_search_slot() error, it will leak 'path'.

Fix it.

Resolves-Coverity-CID: 1374098
Resolves-Coverity-CID: 1374099
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: Enhance leaf traversal function to handle missing inode item
Qu Wenruo [Wed, 21 Sep 2016 02:37:27 +0000 (10:37 +0800)]
btrfs-progs: check: Enhance leaf traversal function to handle missing inode item

The leaf traversal function in lowmem mode will skip to the first inode
item of leaf and begin to check the inode.

That's designed to avoid checking overlapping part of a leaf.

But that will cause problem in fsck/010 test case, as in that case inode
item of the first inode(256) is missing.
So above traversal will check from inode 257 and found nothing wrong.

The fix is done in 2 part:
1) Manually check the first inode
   To avoid case like fsck/010

2) Check inode if ino changes from the first ino of a leaf
   To avoid missing inode_item problem.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: skip shared node or leaf check for low_memory mode
Wang Xiaoguang [Wed, 24 Aug 2016 07:49:17 +0000 (15:49 +0800)]
btrfs-progs: check: skip shared node or leaf check for low_memory mode

The basic idea is simple. Assume a middle tree node A is shared and
its referencing fs/file tree root ids are 5, 258 and 260, then we
only check node A in the tree who has the smallest root id. That means
in this case, when checking root tree(5), we check inode A, for root
tree 258 and 260, we can just skip it.

Notice even with this patch, we still may visit a shared node or leaf
multiple times. This happens when a inode metadata occupies multiple
leaves.

                 leaf_A     leaf_B
When checking inode item in leaf_A, assume inode[512] have file extents
in leaf_B, and leaf_B is shared. In the case, for inode[512], we must
visit leaf_B to have inode item checked. After finishing inode[512] check,
here we walk down tree root to leaf_B to check whether node or leaf
is shared, if some node or leaf is shared, we can just skip it and below
nodes or leaf's check.

I also fill a disk partition with linux source codes and create 3
snapshots in it. Before this patch, it averagely took 46s to finish one
btrfsck execution, with this patch, it averagely took 15s.

Signed-off-by: Wang Xiaoguang <wangxg.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: fix the return value bug of cmd_check()
Lu Fengqi [Tue, 20 Sep 2016 05:49:06 +0000 (13:49 +0800)]
btrfs-progs: check: fix the return value bug of cmd_check()

The function cmd_check() is called by the main function of btrfs.c, its
return value will be returned by exit(). Resulting in the loss of
significant bits in some cases, for example this value is greater than
0377. If use a bool value "err" to store all of the return value, this
will solve the problem.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce low_memory mode fs_tree check
Lu Fengqi [Tue, 26 Jul 2016 06:26:50 +0000 (14:26 +0800)]
btrfs-progs: check: introduce low_memory mode fs_tree check

Introduce a new function check_fs_roots_v2() for check fs_tree in
low_memory mode. It call check_fs_root_v2() to check fs_root, and call
check_root_ref() to check root_ref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check root ref
Lu Fengqi [Tue, 26 Jul 2016 05:12:11 +0000 (13:12 +0800)]
btrfs-progs: check: introduce function to check root ref

Introduce a new function check_root_ref() to check
root_ref/root_backref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check fs root
Lu Fengqi [Tue, 26 Jul 2016 03:18:40 +0000 (11:18 +0800)]
btrfs-progs: check: introduce function to check fs root

Introduce a new function check_fs_root_v2() to check fs root,
and call check_inode_item to check the items in the tree.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check inode item
Lu Fengqi [Tue, 26 Jul 2016 02:44:22 +0000 (10:44 +0800)]
btrfs-progs: check: introduce function to check inode item

Introduce a new function check_inode_item() to check INODE_ITEM and
related ITEMs that have the same inode id.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check file extent
Lu Fengqi [Tue, 26 Jul 2016 02:03:01 +0000 (10:03 +0800)]
btrfs-progs: check: introduce function to check file extent

Introduce a new function check_file_extent() to check file extent,
such as datasum, hole, size.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check dir_item
Lu Fengqi [Mon, 25 Jul 2016 06:57:36 +0000 (14:57 +0800)]
btrfs-progs: check: introduce function to check dir_item

Introduce a new function check_dir_item() to check DIR_ITEM/DIR_INDEX,
and call find_inode_ref() to find the related INODE_REF/INODE_EXTREF.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to find inode_ref
Lu Fengqi [Mon, 25 Jul 2016 05:12:35 +0000 (13:12 +0800)]
btrfs-progs: check: introduce function to find inode_ref

Introduce a new function find_inode_ref() to find
INODE_REF/INODE_EXTREF for the given key, and check it with the
specified DIR_ITEM/DIR_INDEX match.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check inode_extref
Lu Fengqi [Tue, 26 Jul 2016 06:00:01 +0000 (14:00 +0800)]
btrfs-progs: check: introduce function to check inode_extref

Introduce a new function check_inode_extref() to check INODE_EXTREF, and
call find_dir_item() to find the related DIR_ITEM/DIR_INDEX.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to check inode_ref
Lu Fengqi [Mon, 25 Jul 2016 02:24:05 +0000 (10:24 +0800)]
btrfs-progs: check: introduce function to check inode_ref

Introduce a new function check_inode_ref() to check INODE_REF,
and call find_dir_item() to find the related DIR_ITEM/DIR_INDEX.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: introduce function to find dir_item
Lu Fengqi [Thu, 21 Jul 2016 05:51:51 +0000 (13:51 +0800)]
btrfs-progs: check: introduce function to find dir_item

Introduce a new function find_dir_item() to find DIR_ITEM for the given
key, and check it with the specified INODE_REF/INODE_EXTREF match.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: volumes: Use new raid5_gen_result to calculate raid5 parity
Qu Wenruo [Fri, 30 Sep 2016 05:04:57 +0000 (13:04 +0800)]
btrfs-progs: volumes: Use new raid5_gen_result to calculate raid5 parity

Use thew raid5_gen_result() function to calculate raid5 parity.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: raid56: Introduce new function to calculate raid5 parity or data stripe
Qu Wenruo [Fri, 30 Sep 2016 05:04:56 +0000 (13:04 +0800)]
btrfs-progs: raid56: Introduce new function to calculate raid5 parity or data stripe

Introduce new function raid5_gen_result() to calculate parity or data
stripe.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs progs v4.8.5
David Sterba [Wed, 30 Nov 2016 12:06:08 +0000 (13:06 +0100)]
Btrfs progs v4.8.5

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update CHANGES for v4.8.5
David Sterba [Wed, 30 Nov 2016 12:04:34 +0000 (13:04 +0100)]
btrfs-progs: update CHANGES for v4.8.5

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: fix static build of library-test
David Sterba [Wed, 30 Nov 2016 11:37:47 +0000 (12:37 +0100)]
btrfs-progs: test: fix static build of library-test

The static variant of the test was broken, but not really used anyway.
Use the right compilation and linking flags.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add correct rpath to library-test
David Sterba [Wed, 30 Nov 2016 11:34:17 +0000 (12:34 +0100)]
btrfs-progs: tests: add correct rpath to library-test

Our library-test dynamically linked binary was not using the built
libbtrfs. To fix that, use -rpath.

Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: end of stream conditions
David Sterba [Tue, 29 Nov 2016 15:12:15 +0000 (16:12 +0100)]
btrfs-progs: tests: end of stream conditions

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: receive: properly detect end of stream conditions
David Sterba [Tue, 29 Nov 2016 15:07:19 +0000 (16:07 +0100)]
btrfs-progs: receive: properly detect end of stream conditions

Read buffer helper mistakenly reported end of data as an error. Next, we
have to check if the first stream exists as an empty file is not a valid
stream.

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: docs: fix typo in btrfs-man5
David Sterba [Fri, 25 Nov 2016 17:20:34 +0000 (18:20 +0100)]
btrfs-progs: docs: fix typo in btrfs-man5

Reported via IRC.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs progs v4.8.4
David Sterba [Fri, 25 Nov 2016 08:58:48 +0000 (09:58 +0100)]
Btrfs progs v4.8.4

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update CHANGES for v4.8.4
David Sterba [Thu, 24 Nov 2016 12:30:31 +0000 (13:30 +0100)]
btrfs-progs: update CHANGES for v4.8.4

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: link against local libraries
David Sterba [Thu, 24 Nov 2016 10:30:06 +0000 (11:30 +0100)]
btrfs-progs: build: link against local libraries

If there's a system provided libbtrfs.0, it will be used for the
library-test build and could miss linker errors, as reported.

Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: extend library-test coverage
David Sterba [Wed, 23 Nov 2016 09:39:05 +0000 (10:39 +0100)]
btrfs-progs: test: extend library-test coverage

Add test for linkage error of lookup_path_rootid. The built binary has
to be run to catch also run-time errors.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: move lookup_path_rootid to library utils
David Sterba [Wed, 23 Nov 2016 09:36:43 +0000 (10:36 +0100)]
btrfs-progs: move lookup_path_rootid to library utils

The refactoring in commit 1c85c3de5aab997ff66ea95cb0f2c9f79726ec40 has
broken use of libbtrfs that does not exhibit during build but at the run
time.

Fixes: 1c85c3de5aab997ff66ea95cb0f2c9f79726ec40
Bugzilla: https://bugs.gentoo.org/show_bug.cgi?id=600078
Reported-by: Mike Gilbert <floppymaster@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: fix format string on 32bit
David Sterba [Tue, 22 Nov 2016 17:19:34 +0000 (18:19 +0100)]
btrfs-progs: print-tree: fix format string on 32bit

We're passing a size_t, use the right format specifier.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: move bitops.h to kernel-lib
David Sterba [Tue, 22 Nov 2016 15:55:11 +0000 (16:55 +0100)]
btrfs-progs: move bitops.h to kernel-lib

Forgotten when the rest got moved.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: deprecate standalone btrfs-show-super
David Sterba [Tue, 22 Nov 2016 15:34:44 +0000 (16:34 +0100)]
btrfs-progs: deprecate standalone btrfs-show-super

The functionality is now provided by 'inspect dump-super'.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: deprecate standalone btrfs-calc-size
David Sterba [Tue, 22 Nov 2016 15:34:44 +0000 (16:34 +0100)]
btrfs-progs: deprecate standalone btrfs-calc-size

The functionality is now provided by 'inspect tree-stats'.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: move btrfs-image sources to own directory
David Sterba [Tue, 22 Nov 2016 15:15:24 +0000 (16:15 +0100)]
btrfs-progs: move btrfs-image sources to own directory

There are no pending patches to btrfs-image, this looks like a good time
to move it.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add support for additional command arguments
David Sterba [Tue, 22 Nov 2016 12:32:18 +0000 (13:32 +0100)]
btrfs-progs: tests: add support for additional command arguments

Add convenient support for extending command arguments, now implemented
for 'btrfs check' to cover the low-memory mode. If defined, arguments
are inserted to any 'btrfs check' command in tests. Exceptions could be
defined in common.local.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: more variable quoting in common scripts
David Sterba [Mon, 21 Nov 2016 13:51:40 +0000 (14:51 +0100)]
btrfs-progs: tests: more variable quoting in common scripts

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: drop redundant test for rwx number validity
David Sterba [Mon, 21 Nov 2016 13:44:06 +0000 (14:44 +0100)]
btrfs-progs: tests: drop redundant test for rwx number validity

The list of rwx permissions is now hardcoded but used to begenerated and
the invalid numbers filtered out. Not necessary anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: fix how to make test files in fsck-tests 013
Tsutomu Itoh [Mon, 21 Nov 2016 05:10:41 +0000 (14:10 +0900)]
btrfs-progs: test: fix how to make test files in fsck-tests 013

In my test environment, following error was occurred because the size
of /lib/modules/`uname -r`/* is larger than 1GB.

# make test-fsck
    [TEST]   fsck-tests.sh
    [TEST/fsck]   013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#

In this test case, 'generate_dataset small' is enough for making the
test files, so I will use 'generate_dataset' instead of 'cp'.

For this, move 'generate_dataset()' from 'common.convert' to 'common'.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Update README and other docs
David Sterba [Thu, 10 Nov 2016 17:31:53 +0000 (18:31 +0100)]
btrfs-progs: Update README and other docs

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: fix missing newlines
Omar Sandoval [Wed, 16 Nov 2016 20:54:32 +0000 (12:54 -0800)]
btrfs-progs: check: fix missing newlines

Also, the other progress messages go to stderr, so "checking extents"
probably should, as well.

Fixes: c7a1f66a205f ("btrfs-progs: check: switch some messages to common helpers")
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: qgroup: fix error in ASSERT condition expression
Tsutomu Itoh [Thu, 17 Nov 2016 05:01:31 +0000 (14:01 +0900)]
btrfs-progs: qgroup: fix error in ASSERT condition expression

Option -f, -F and --sort don't work because a conditional expression
of ASSERT is wrong.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add fallback to current directory for check_all_images
David Sterba [Fri, 18 Nov 2016 18:21:02 +0000 (19:21 +0100)]
btrfs-progs: tests: add fallback to current directory for check_all_images

Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: fix error of test target of Makefile
Tsutomu Itoh [Fri, 18 Nov 2016 05:44:12 +0000 (14:44 +0900)]
btrfs-progs: test: fix error of test target of Makefile

Add test-cli to test target of Makefile.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: document space_cache=v2 more thoroughly
Omar Sandoval [Mon, 14 Nov 2016 18:43:23 +0000 (10:43 -0800)]
btrfs-progs: document space_cache=v2 more thoroughly

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: implement btrfs check --clear-space-cache v2
Omar Sandoval [Mon, 14 Nov 2016 18:43:22 +0000 (10:43 -0800)]
btrfs-progs: implement btrfs check --clear-space-cache v2

Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
[ adjusted error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: add btrfs_clear_free_space_tree() from the kernel
Omar Sandoval [Mon, 14 Nov 2016 18:43:21 +0000 (10:43 -0800)]
btrfs-progs: add btrfs_clear_free_space_tree() from the kernel

Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: add OPEN_CTREE_INVALIDATE_FST flag
Omar Sandoval [Mon, 14 Nov 2016 18:43:20 +0000 (10:43 -0800)]
btrfs-progs: add OPEN_CTREE_INVALIDATE_FST flag

If this flag is passed to open_ctree(), we'll clear the
FREE_SPACE_TREE_VALID compat_ro bit. The kernel will then reconstruct
the free space tree the next time the filesystem is mounted.

Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: format FREE_SPACE_TREE{,_VALID} nicely in dump-super
Omar Sandoval [Mon, 14 Nov 2016 18:43:19 +0000 (10:43 -0800)]
btrfs-progs: format FREE_SPACE_TREE{,_VALID} nicely in dump-super

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: add the FREE_SPACE_TREE_VALID compat_ro bit definition
Omar Sandoval [Mon, 14 Nov 2016 18:43:18 +0000 (10:43 -0800)]
btrfs-progs: add the FREE_SPACE_TREE_VALID compat_ro bit definition

This is just the definition; we don't support it yet.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send: fix failure of fstests btrfs/038
Tsutomu Itoh [Tue, 15 Nov 2016 08:16:27 +0000 (17:16 +0900)]
btrfs-progs: send: fix failure of fstests btrfs/038

The following patch was imperfect, so xfstests btrfs/038 was failed.

  6d4fb3d  btrfs-progs: send: fix handling of multiple snapshots (-p option)

[before]
| # ./check btrfs/038
| FSTYP         -- btrfs
| PLATFORM      -- Linux/x86_64 luna 4.9.0-rc5
| MKFS_OPTIONS  -- /dev/sdb3
| MOUNT_OPTIONS -- /dev/sdb3 /test6
|
| btrfs/038 1s ... [failed, exit status 1] - output mismatch (see /For_RT/xfstests2/results//btrfs/038.out.bad)
|     --- tests/btrfs/038.out     2015-08-04 16:09:38.000000000 +0900
|     +++ /For_RT/xfstests2/results//btrfs/038.out.bad    2016-11-15 13:39:48.589435290 +0900
|     @@ -7,3 +7,5 @@
|      XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|      wrote 10000/10000 bytes at offset 300000
|      XXX Bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|     +failed: '/usr/local/bin/btrfs send -p /test6/mysnap1 -c /test6/clones_snap /test6/mysnap2 -f /tmp/tmp.aLpvAC7lsx/2.snap'
|     +(see /For_RT/xfstests2/results//btrfs/038.full for details)
|     ...
|     (Run 'diff -u tests/btrfs/038.out /For_RT/xfstests2/results//btrfs/038.out.bad'  to see the entire diff)
| Ran: btrfs/038
| Failures: btrfs/038

[after]
| # ./check btrfs/038
| FSTYP         -- btrfs
| PLATFORM      -- Linux/x86_64 luna 4.9.0-rc5
| MKFS_OPTIONS  -- /dev/sdb3
| MOUNT_OPTIONS -- /dev/sdb3 /test6
|
| btrfs/038 1s ... 1s
| Ran: btrfs/038
| Passed all 1 tests

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
[ Verified that this fixes both btrfs/038 and btrfs/117 ]
Tested-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add more tests to the CI build
David Sterba [Fri, 18 Nov 2016 16:46:12 +0000 (17:46 +0100)]
btrfs-progs: tests: add more tests to the CI build

Run more tests in the travis build environment. The mkfs tests are still
left out due to missing capabilities of losetup.

Other minor travis adjustments.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: enhance TEST_LOG features
David Sterba [Fri, 18 Nov 2016 14:18:02 +0000 (15:18 +0100)]
btrfs-progs: tests: enhance TEST_LOG features

Add new keyword to dump the log file after any test fails. Can be useful
for remote analysis of test failures.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: check for TEST_LOG values by a regex
David Sterba [Fri, 18 Nov 2016 14:09:50 +0000 (15:09 +0100)]
btrfs-progs: tests: check for TEST_LOG values by a regex

The set of possible values will be extended so check for existence of
the keyword.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: change options to list all loop devices
David Sterba [Fri, 18 Nov 2016 14:06:13 +0000 (15:06 +0100)]
btrfs-progs: tests: change options to list all loop devices

The option --list might not be available on older versions, the
equvalent is --all. Discovered via failed travis build.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: use the root helper for various info retrieval commands
David Sterba [Fri, 18 Nov 2016 14:02:50 +0000 (15:02 +0100)]
btrfs-progs: tests: use the root helper for various info retrieval commands

The current user might not be able to peek into the loop files directly,
use the helper. Discovered by running tests in travis.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: docs: fix typo in btrfs-scrub manual page
David Sterba [Fri, 18 Nov 2016 13:51:15 +0000 (14:51 +0100)]
btrfs-progs: docs: fix typo in btrfs-scrub manual page

Reported on IRC.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: track the read position in the stream
David Sterba [Tue, 15 Nov 2016 15:45:01 +0000 (16:45 +0100)]
btrfs-progs: send-stream: track the read position in the stream

Can be later used for more precise error reporting.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add basic checks for bad stream on the receive side
David Sterba [Tue, 15 Nov 2016 14:34:03 +0000 (15:34 +0100)]
btrfs-progs: tests: add basic checks for bad stream on the receive side

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: check number of read bytes from stream
David Sterba [Tue, 15 Nov 2016 14:22:42 +0000 (15:22 +0100)]
btrfs-progs: send-stream: check number of read bytes from stream

The read_buf does not verify that we've read the expected number of
bytes.  A corrupted of malformated stream will not be detdcted.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: document return values of some functions
David Sterba [Tue, 15 Nov 2016 14:09:08 +0000 (15:09 +0100)]
btrfs-progs: send-stream: document return values of some functions

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: use proper type for cmd in read_cmd
David Sterba [Tue, 15 Nov 2016 14:05:46 +0000 (15:05 +0100)]
btrfs-progs: send-stream: use proper type for cmd in read_cmd

We just read and assign the value, there's no reason to use different
types.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: don't use single letter variable in tlv_get
David Sterba [Tue, 15 Nov 2016 13:55:22 +0000 (14:55 +0100)]
btrfs-progs: send-stream: don't use single letter variable in tlv_get

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: use proper types for tlv header values
David Sterba [Tue, 15 Nov 2016 13:52:03 +0000 (14:52 +0100)]
btrfs-progs: send-stream: use proper types for tlv header values

Switch types to unsigned and adjust the checks.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: use proper type for read return value
David Sterba [Tue, 15 Nov 2016 13:44:46 +0000 (14:44 +0100)]
btrfs-progs: send-stream: use proper type for read return value

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: change length type to unsigned
David Sterba [Tue, 15 Nov 2016 13:41:54 +0000 (14:41 +0100)]
btrfs-progs: send-stream: change length type to unsigned

The command length is unsigned, use the right type, also to make the
length checks work.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: check command length before reading from stream
David Sterba [Tue, 15 Nov 2016 13:38:19 +0000 (14:38 +0100)]
btrfs-progs: send-stream: check command length before reading from stream

The command + header length could not fit to the intermediate buffer.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: pass char buffer to read_buf
David Sterba [Tue, 15 Nov 2016 13:26:44 +0000 (14:26 +0100)]
btrfs-progs: send-stream: pass char buffer to read_buf

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: fix size types passed to read_buf
David Sterba [Tue, 15 Nov 2016 13:19:30 +0000 (14:19 +0100)]
btrfs-progs: send-stream: fix size types passed to read_buf

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: rename single letter variable
David Sterba [Tue, 15 Nov 2016 13:15:23 +0000 (14:15 +0100)]
btrfs-progs: send-stream: rename single letter variable

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send-stream: switch to common message helpers
David Sterba [Tue, 15 Nov 2016 13:02:48 +0000 (14:02 +0100)]
btrfs-progs: send-stream: switch to common message helpers

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add more fuzzed images from bugzilla
David Sterba [Mon, 14 Nov 2016 18:06:40 +0000 (19:06 +0100)]
btrfs-progs: tests: add more fuzzed images from bugzilla

Fixing the problems by one does not scale now. Add more images despite
the fuzz tests will fail. They have been for some time already.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: ioctl: fix compilation if ioctl.h is included standalone
David Sterba [Mon, 14 Nov 2016 15:43:15 +0000 (16:43 +0100)]
btrfs-progs: ioctl: fix compilation if ioctl.h is included standalone

Definition of NULL is missing if ioctl.h is included standalone or when
no other include pulls the definition.

Reported-by: Sergei Trofimovich <slyich@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: return best entry, if it is the first one
Goldwyn Rodrigues [Thu, 10 Nov 2016 15:01:46 +0000 (09:01 -0600)]
btrfs-progs: return best entry, if it is the first one

The find_most_right_entry() tends to miss on the best entry if it is the
first one on the list and there are only two entries in the list.  So,
we assign both prev and best to entry.

To do this, the selection process (rather the rejection) has to be
performed earlier to skip on broken==count.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: utils: Fix NULL pointer derefernces in string_is_numerical
Qu Wenruo [Mon, 24 Oct 2016 02:43:34 +0000 (10:43 +0800)]
btrfs-progs: utils: Fix NULL pointer derefernces in string_is_numerical

In get_running_kernel_version() function, we directly pass return
pointer from strtok_r() to string_is_numerical().

Return pointer from strok_r() can be NULL, but string_is_numerical()
can't handle it and will cause NULL pointer derefernces.

Fix it by check if it's a NULL pointer first.

Resolves-Coverity-CID: 1374097
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs progs v4.8.3
David Sterba [Fri, 11 Nov 2016 15:28:18 +0000 (16:28 +0100)]
Btrfs progs v4.8.3

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update CHANGES for v4.8.3
David Sterba [Wed, 9 Nov 2016 14:07:35 +0000 (15:07 +0100)]
btrfs-progs: update CHANGES for v4.8.3

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: enhance run_mayfail description comment
David Sterba [Fri, 11 Nov 2016 12:36:27 +0000 (13:36 +0100)]
btrfs-progs: tests: enhance run_mayfail description comment

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add quotation around variables in support scripts
David Sterba [Fri, 11 Nov 2016 09:12:05 +0000 (10:12 +0100)]
btrfs-progs: tests: add quotation around variables in support scripts

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add quotation around variables in common.convert
David Sterba [Fri, 11 Nov 2016 09:12:05 +0000 (10:12 +0100)]
btrfs-progs: tests: add quotation around variables in common.convert

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add quotes around variables in common
David Sterba [Fri, 11 Nov 2016 09:04:26 +0000 (10:04 +0100)]
btrfs-progs: tests: add quotes around variables in common

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add path assertions to post-checks
David Sterba [Fri, 11 Nov 2016 07:40:21 +0000 (08:40 +0100)]
btrfs-progs: tests: add path assertions to post-checks

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add assertion helper
David Sterba [Fri, 11 Nov 2016 00:18:49 +0000 (01:18 +0100)]
btrfs-progs: tests: add assertion helper

Helper to extend sanity checks in various functions.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: refactor post-convert check helpers
David Sterba [Fri, 11 Nov 2016 00:27:39 +0000 (01:27 +0100)]
btrfs-progs: tests: refactor post-convert check helpers

Separate checksum test from convert_test_post_check and use it to fix
the broken test 005 as reported.

References: https://bugzilla.kernel.org/show_bug.cgi?id=177141
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: check if kernel has btrfs support
David Sterba [Thu, 10 Nov 2016 17:57:38 +0000 (18:57 +0100)]
btrfs-progs: tests: check if kernel has btrfs support

Add some sanity checks, reported among other issues via bugzilla.

References: https://bugzilla.kernel.org/show_bug.cgi?id=177141
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: print the new UUID
David Sterba [Thu, 10 Nov 2016 17:23:01 +0000 (18:23 +0100)]
btrfs-progs: mkfs: print the new UUID

Unless the uuid is specified as the command line option, it's not
printed in the summary.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: add names of matching sysfs feature names
David Sterba [Tue, 9 Feb 2016 22:45:38 +0000 (23:45 +0100)]
btrfs-progs: mkfs: add names of matching sysfs feature names

The mkfs parameters do not match file names exported through sysfs.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: enhance feature table
David Sterba [Thu, 10 Nov 2016 16:23:17 +0000 (17:23 +0100)]
btrfs-progs: mkfs: enhance feature table

Add string and numeric version for later use so we don't have to convert
all the time.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: extend mkfs features with compat, safe and default versions
David Sterba [Tue, 9 Feb 2016 18:36:42 +0000 (19:36 +0100)]
btrfs-progs: mkfs: extend mkfs features with compat, safe and default versions

Enhance the mkfs_features list with the minimum kernel versions that
will allow for turning on compatible and/or safe options.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: add temporary variable for fs features
David Sterba [Thu, 10 Nov 2016 16:11:09 +0000 (17:11 +0100)]
btrfs-progs: mkfs: add temporary variable for fs features

A minor cleanup before other changes.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Fix memory leak in write_raid56_with_parity
Qu Wenruo [Mon, 24 Oct 2016 02:43:32 +0000 (10:43 +0800)]
btrfs-progs: Fix memory leak in write_raid56_with_parity

Ebs and pointers are allocated, but if any of the allocation failed, we
should free the allocated memory.

Resolves-Coverity-CID: 1374101
Resolves-Coverity-CID: 1374100
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add checking of send multiple clone source option
Tsutomu Itoh [Wed, 9 Nov 2016 04:44:15 +0000 (13:44 +0900)]
btrfs-progs: tests: add checking of send multiple clone source option

Sending stream size of clone-src(-c) option is checked.
Fixed by "btrfs-progs: send: fix handling of -c option".

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send: check for output file existence before creating
David Sterba [Mon, 7 Nov 2016 15:51:20 +0000 (16:51 +0100)]
btrfs-progs: send: check for output file existence before creating

In some cases the root might not be able to create the output file (and
streaming to stdout is not an option). Make the output file creation two
step and let it work if the file is already created.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: send: fix handling of -c option
Tsutomu Itoh [Fri, 4 Nov 2016 08:33:58 +0000 (17:33 +0900)]
btrfs-progs: send: fix handling of -c option

When two or more -c options are specified, cannot find a suitable
parent. So, output stream is bigger than correct one.

[before]
At subvol Snap1
At subvol Snap2
At subvol ../SnapY
-rw------- 1 root root 3153 Oct 19 10:37 /tmp/data1

[after]
At subvol Snap1
At subvol Snap2
At subvol ../SnapY
-rw------- 1 root root 1492 Oct 19 10:39 /tmp/data1

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
[ constify subvol argument, renamed single letter variables ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: teach extract_image about packed streams
David Sterba [Wed, 9 Nov 2016 10:39:02 +0000 (11:39 +0100)]
btrfs-progs: tests: teach extract_image about packed streams

Packed streams for testing purposes should be packed with 'xz -9' and
use suffix .stream.xz .

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: use correct type for device id iteration in get_fs_info
David Sterba [Mon, 7 Nov 2016 14:38:24 +0000 (15:38 +0100)]
btrfs-progs: use correct type for device id iteration in get_fs_info

Coverity reports (CID 1374096) that there's return value overflow of
ret from get_device_info, but this most likely cannot happen. There's
another minor issue where we use int to iterate over devids.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: crc32: use fallback implementation for unaligned buffers
David Sterba [Mon, 7 Nov 2016 12:58:51 +0000 (13:58 +0100)]
btrfs-progs: crc32: use fallback implementation for unaligned buffers

ASAN reports that at some point the crc function gets an unaligned
buffer. It's the optimized intel version that casts char to ulong, the
buffer is the embedded filename in the directory items.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: teach scan-results about more errors
David Sterba [Mon, 7 Nov 2016 12:21:34 +0000 (13:21 +0100)]
btrfs-progs: tests: teach scan-results about more errors

ASAN detects use after free.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: fix search tree v2 ioctl detection
David Sterba [Mon, 7 Nov 2016 12:02:00 +0000 (13:02 +0100)]
btrfs-progs: fix search tree v2 ioctl detection

The result of the test applies per-filesystem, so we can't simply cache
it. The function hasn't been used yet.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: fix unaligned u64 access in btrfs_alloc_data_chunk
David Sterba [Fri, 4 Nov 2016 14:11:54 +0000 (15:11 +0100)]
btrfs-progs: fix unaligned u64 access in btrfs_alloc_data_chunk

The key.offset member is not well-aligned as the key is packed, use a
temporary variable to pass the argument. Reported by ASAN in misc test
002.

Signed-off-by: David Sterba <dsterba@suse.com>