platform/upstream/btrfs-progs.git
8 years agobtrfs-progs: tests: add fuzzed image for invalid chunk sectorsize
David Sterba [Sat, 3 Sep 2016 18:52:18 +0000 (20:52 +0200)]
btrfs-progs: tests: add fuzzed image for invalid chunk sectorsize

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: tests: add fuzzed image for invalid sys_array and stripe_len
David Sterba [Sat, 3 Sep 2016 18:52:18 +0000 (20:52 +0200)]
btrfs-progs: tests: add fuzzed image for invalid sys_array and stripe_len

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: tests: add fuzzed image for invalid sub_stripe value
David Sterba [Sat, 3 Sep 2016 18:47:21 +0000 (20:47 +0200)]
btrfs-progs: tests: add fuzzed image for invalid sub_stripe value

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: image: more verbose syntax erors for -t and -c
David Sterba [Sat, 3 Sep 2016 18:34:47 +0000 (20:34 +0200)]
btrfs-progs: image: more verbose syntax erors for -t and -c

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: print help test to stdout
David Sterba [Sat, 3 Sep 2016 18:30:50 +0000 (20:30 +0200)]
btrfs-progs: print help test to stdout

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: tests: add 015-dump-super-garbage
David Sterba [Thu, 1 Sep 2016 18:05:20 +0000 (20:05 +0200)]
btrfs-progs: tests: add 015-dump-super-garbage

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: dump-super: detect invalid checksum type
David Sterba [Thu, 1 Sep 2016 17:54:04 +0000 (19:54 +0200)]
btrfs-progs: dump-super: detect invalid checksum type

The helper btrfs_super_csum_size unconditionally bugs on a bogus value,
the dump-super is supposed to dump even corrupted superblocks so this
must not happen here. Group the checsum values in the output and be more
robust agains garbage values.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: drop permission arg from non-creating open
David Sterba [Thu, 1 Sep 2016 17:38:23 +0000 (19:38 +0200)]
btrfs-progs: drop permission arg from non-creating open

The permissions do not make sense without O_CREATE mode.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: docs: describe filesystem features
David Sterba [Thu, 1 Sep 2016 16:51:18 +0000 (18:51 +0200)]
btrfs-progs: docs: describe filesystem features

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: build: only install udev rules for udev >= 190
Jeff Mahoney [Wed, 31 Aug 2016 01:15:44 +0000 (21:15 -0400)]
btrfs-progs: build: only install udev rules for udev >= 190

Prior to udev v190, there was no btrfs builtin helper.  Installing it on
systems with an older udev will cause problems.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: tests README: fuzzed images
David Sterba [Tue, 30 Aug 2016 15:09:10 +0000 (17:09 +0200)]
btrfs-progs: tests README: fuzzed images

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fuzz-test: Add test case for unaligned extent item
Qu Wenruo [Tue, 30 Aug 2016 07:22:17 +0000 (15:22 +0800)]
btrfs-progs: fuzz-test: Add test case for unaligned extent item

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fsck: Avoid abort and BUG_ON in add_tree_backref
Qu Wenruo [Tue, 30 Aug 2016 07:22:16 +0000 (15:22 +0800)]
btrfs-progs: fsck: Avoid abort and BUG_ON in add_tree_backref

Add_tree_backref() can cause BUG_ON() and abort() in quite a lot of
cases, from the ENOMEM to existing tree backref records.

Change all these BUG_ON() and abort() to return proper values.
And modify all callers to handle such problems.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fsck: Check bytenr alignment for extent item
Qu Wenruo [Tue, 30 Aug 2016 07:22:15 +0000 (15:22 +0800)]
btrfs-progs: fsck: Check bytenr alignment for extent item

Check bytenr alignment for extent item to filter invalid items early.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fuzz-test: Add test case for invalid drop level
Qu Wenruo [Tue, 30 Aug 2016 07:22:14 +0000 (15:22 +0800)]
btrfs-progs: fuzz-test: Add test case for invalid drop level

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fsck: Check drop level before walking through fs tree
Qu Wenruo [Tue, 30 Aug 2016 07:22:13 +0000 (15:22 +0800)]
btrfs-progs: fsck: Check drop level before walking through fs tree

Exposed by fuzzed image from Lukas, which contains invalid drop level
(16), causing segfault when accessing path->nodes[drop_level].

This patch will check drop level against fs tree level and
BTRFS_MAX_LEVEL to avoid such problem.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fuzz-test: Add image for unaligned tree block ptr
Qu Wenruo [Tue, 30 Aug 2016 03:29:33 +0000 (11:29 +0800)]
btrfs-progs: fuzz-test: Add image for unaligned tree block ptr

Add test case image for unaligned tree block ptr.
It should lead to BUG_ON in free_extent_buffer().

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: do early check for read_tree_block
Qu Wenruo [Tue, 30 Aug 2016 03:29:32 +0000 (11:29 +0800)]
btrfs-progs: check: do early check for read_tree_block

Although we have enhanced read_tree_block() from a lot of different
aspects, it lacks the early bytenr/blocksize alignment check.

And the lack of such check can lead to strange use-after-free bugs, due
to the fact that alloc_extent_buffer() will free overlapping extent
buffers, and allocate new eb for the usage.

So we should not allow invalid bytenr/blocksize even passed to
btrfs_find_create_tree_block().

This patch will add such check so we won't trigger use-after-free bug
then.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: fuzz-test: Add image for wrong chunk item in root tree
Qu Wenruo [Tue, 30 Aug 2016 02:15:50 +0000 (10:15 +0800)]
btrfs-progs: fuzz-test: Add image for wrong chunk item in root tree

Reported by Lukas and the same image from him.

DATA_RELOC tree's key type is modifed to CHUNK_ITEM, causing btrfsck
interpret it as CHUNK_ITEM and cause 0 num_stripes.

Add the image to fuzz-test.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: Do extra chunk check before processing chunk item
Qu Wenruo [Mon, 29 Aug 2016 08:09:02 +0000 (16:09 +0800)]
btrfs-progs: Do extra chunk check before processing chunk item

Current we only do chunk validation check at mount time.

It's good for most case, but for fuzzed or manually crafted images, we
can insert a CHUNK_ITEM key into root tree.

Since mount time check will only check chunk tree, it will not check
CHUNK_ITEM in root tree.

Even with previous key type check against leaf owner, it is still
possible to modify the leaf owner to by-pass it.

So we still need to check chunk validation before processing it.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: ignore invalid key in invalid root
Qu Wenruo [Mon, 29 Aug 2016 08:09:01 +0000 (16:09 +0800)]
btrfs-progs: check: ignore invalid key in invalid root

Btrfs tree implies a lot of restriction on which key types are allowed
in specific roots.

Like CHUNK_ITEM keys are only valid in chunk root.

This patch will add such check at run_next_block() for original mode.

Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: Enhance and export print_key_type function
Qu Wenruo [Mon, 29 Aug 2016 08:09:00 +0000 (16:09 +0800)]
btrfs-progs: Enhance and export print_key_type function

Just the same thing done for print_objectid().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: Enhance and export print_objectid function
Qu Wenruo [Mon, 29 Aug 2016 08:08:59 +0000 (16:08 +0800)]
btrfs-progs: Enhance and export print_objectid function

This function is quite useful for a lot of error report.
Enhance it to support custom output other than stdout.
And export it for later btrfsck enhancement.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agoBtrfs progs v4.7.1
David Sterba [Thu, 25 Aug 2016 17:33:48 +0000 (19:33 +0200)]
Btrfs progs v4.7.1

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: update CHANGES for 4.7.1
David Sterba [Thu, 28 Jul 2016 12:18:59 +0000 (14:18 +0200)]
btrfs-progs: update CHANGES for 4.7.1

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: add rest of stack operations for btrfs_dir_item
David Sterba [Thu, 25 Aug 2016 17:02:18 +0000 (19:02 +0200)]
btrfs-progs: add rest of stack operations for btrfs_dir_item

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: switch to common error helpers in do_rollback
David Sterba [Tue, 23 Aug 2016 17:58:07 +0000 (19:58 +0200)]
btrfs-progs: convert: switch to common error helpers in do_rollback

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: reword message when conversion fails
David Sterba [Tue, 23 Aug 2016 17:43:20 +0000 (19:43 +0200)]
btrfs-progs: convert: reword message when conversion fails

The filesystem will not be finalized, no difference if it's before or
after the chunk mapping fixups.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: switch to message helpers in do_convert
David Sterba [Tue, 23 Aug 2016 17:40:32 +0000 (19:40 +0200)]
btrfs-progs: convert: switch to message helpers in do_convert

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: remove redundant check
David Sterba [Tue, 23 Aug 2016 17:33:05 +0000 (19:33 +0200)]
btrfs-progs: convert: remove redundant check

The check for sectorsize is properly done in do_convert, remove the
BUG_ON.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: improve error hanling of init_btrfs
David Sterba [Tue, 23 Aug 2016 17:24:48 +0000 (19:24 +0200)]
btrfs-progs: convert: improve error hanling of init_btrfs

Replace BUG_ONs and verbosely report the errors.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: improve error hanling of create_subvol
David Sterba [Tue, 23 Aug 2016 17:24:48 +0000 (19:24 +0200)]
btrfs-progs: convert: improve error hanling of create_subvol

Replace BUG_ONs and verbosely report the errors.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: improve error hanling of link_subvol
David Sterba [Tue, 23 Aug 2016 17:24:48 +0000 (19:24 +0200)]
btrfs-progs: convert: improve error hanling of link_subvol

Replace BUG_ONs and verbosely report the errors.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: make ext2 compilation honor the configure options
David Sterba [Tue, 23 Aug 2016 16:05:56 +0000 (18:05 +0200)]
btrfs-progs: convert: make ext2 compilation honor the configure options

Now ext2 conversion will be built in iff it's selected by --with-convert
option. Default is yes.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move prepare_system_chunk_sb
David Sterba [Tue, 23 Aug 2016 15:50:51 +0000 (17:50 +0200)]
btrfs-progs: convert: move prepare_system_chunk_sb

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move migrate_super_block
David Sterba [Tue, 23 Aug 2016 15:50:16 +0000 (17:50 +0200)]
btrfs-progs: convert: move migrate_super_block

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move init_btrfs
David Sterba [Tue, 23 Aug 2016 15:49:54 +0000 (17:49 +0200)]
btrfs-progs: convert: move init_btrfs

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move make_convert_data_block_groups
David Sterba [Tue, 23 Aug 2016 15:49:08 +0000 (17:49 +0200)]
btrfs-progs: convert: move make_convert_data_block_groups

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move create_subvol
David Sterba [Tue, 23 Aug 2016 15:48:44 +0000 (17:48 +0200)]
btrfs-progs: convert: move create_subvol

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move link_subvol
David Sterba [Tue, 23 Aug 2016 15:48:29 +0000 (17:48 +0200)]
btrfs-progs: convert: move link_subvol

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move create_image
David Sterba [Tue, 23 Aug 2016 15:48:07 +0000 (17:48 +0200)]
btrfs-progs: convert: move create_image

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move convert_read_used_space
David Sterba [Tue, 23 Aug 2016 15:46:25 +0000 (17:46 +0200)]
btrfs-progs: convert: move convert_read_used_space

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move calculate_available_space
David Sterba [Tue, 23 Aug 2016 15:45:54 +0000 (17:45 +0200)]
btrfs-progs: convert: move calculate_available_space

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move wipe_reserved_ranges
David Sterba [Tue, 23 Aug 2016 15:45:15 +0000 (17:45 +0200)]
btrfs-progs: convert: move wipe_reserved_ranges

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move wipe_one_reserved_range
David Sterba [Tue, 23 Aug 2016 15:44:36 +0000 (17:44 +0200)]
btrfs-progs: convert: move wipe_one_reserved_range

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move _expand_extent_cache
David Sterba [Tue, 23 Aug 2016 15:44:03 +0000 (17:44 +0200)]
btrfs-progs: convert: move _expand_extent_cache

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move migrate_reserved_ranges
David Sterba [Tue, 23 Aug 2016 15:43:21 +0000 (17:43 +0200)]
btrfs-progs: convert: move migrate_reserved_ranges

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move migrate_one_reserved_range
David Sterba [Tue, 23 Aug 2016 15:42:23 +0000 (17:42 +0200)]
btrfs-progs: convert: move migrate_one_reserved_range

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move create_image_file_range
David Sterba [Tue, 23 Aug 2016 15:42:01 +0000 (17:42 +0200)]
btrfs-progs: convert: move create_image_file_range

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move block_iterate_proc
David Sterba [Tue, 23 Aug 2016 15:41:20 +0000 (17:41 +0200)]
btrfs-progs: convert: move block_iterate_proc

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move record_file_blocks
David Sterba [Tue, 23 Aug 2016 15:41:00 +0000 (17:41 +0200)]
btrfs-progs: convert: move record_file_blocks

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move init_blk_iterate_data
David Sterba [Tue, 23 Aug 2016 15:40:08 +0000 (17:40 +0200)]
btrfs-progs: convert: move init_blk_iterate_data

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move csum_disk_extent
David Sterba [Tue, 23 Aug 2016 15:38:55 +0000 (17:38 +0200)]
btrfs-progs: convert: move csum_disk_extent

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move read_disk_extent
David Sterba [Tue, 23 Aug 2016 15:36:13 +0000 (17:36 +0200)]
btrfs-progs: convert: move read_disk_extent

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move convert_insert_dirent
David Sterba [Tue, 23 Aug 2016 15:35:05 +0000 (17:35 +0200)]
btrfs-progs: convert: move convert_insert_dirent

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: move intersect_with_sb
David Sterba [Tue, 23 Aug 2016 15:34:43 +0000 (17:34 +0200)]
btrfs-progs: convert: move intersect_with_sb

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: prefix all ext2-related helpers
David Sterba [Tue, 23 Aug 2016 15:31:34 +0000 (17:31 +0200)]
btrfs-progs: convert: prefix all ext2-related helpers

Add ext2_ prefix to all functions that seem to be ext2-specific.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: build: extend convert options
David Sterba [Tue, 23 Aug 2016 15:21:30 +0000 (17:21 +0200)]
btrfs-progs: build: extend convert options

Add --with-convert[=VALUE] option to configure. Accepts ext2, auto, yes,
or no, but will be extended to more in the future. The configure-time
defines are not used in the code, ext2 is built-in unconditionally.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: build: add per-tool cflags variable
David Sterba [Tue, 23 Aug 2016 16:04:37 +0000 (18:04 +0200)]
btrfs-progs: build: add per-tool cflags variable

The standalone tools are built from pattern rules, add support for
per-tool cflaags, like btrfs_something_clfags.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: cleanup, open_ctree to fs_info and use directly
David Sterba [Tue, 23 Aug 2016 14:18:33 +0000 (16:18 +0200)]
btrfs-progs: mkfs: cleanup, open_ctree to fs_info and use directly

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: kill print_version
David Sterba [Tue, 23 Aug 2016 14:12:36 +0000 (16:12 +0200)]
btrfs-progs: mkfs: kill print_version

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: clean up make_image
David Sterba [Mon, 22 Aug 2016 17:14:20 +0000 (19:14 +0200)]
btrfs-progs: mkfs: clean up make_image

The exact errors are printed, the removed message does not seem to be
necessary. Return proper errors.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: help and usage now to to stdout
David Sterba [Mon, 22 Aug 2016 16:18:54 +0000 (18:18 +0200)]
btrfs-progs: mkfs: help and usage now to to stdout

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: handle and report transaction commit failures
David Sterba [Mon, 22 Aug 2016 16:18:14 +0000 (18:18 +0200)]
btrfs-progs: mkfs: handle and report transaction commit failures

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: switch BUG_ON to error handling in traverse_directory
David Sterba [Mon, 22 Aug 2016 16:10:43 +0000 (18:10 +0200)]
btrfs-progs: mkfs: switch BUG_ON to error handling in traverse_directory

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: document all btrfs_open_ctree_flags
David Sterba [Mon, 22 Aug 2016 16:08:13 +0000 (18:08 +0200)]
btrfs-progs: document all btrfs_open_ctree_flags

Document and add unsigned type to the values.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: improve error handling in recow_roots
David Sterba [Mon, 22 Aug 2016 15:55:16 +0000 (17:55 +0200)]
btrfs-progs: mkfs: improve error handling in recow_roots

Return and handle errors in the callchain.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: improve error handling in main()
David Sterba [Mon, 22 Aug 2016 15:50:49 +0000 (17:50 +0200)]
btrfs-progs: mkfs: improve error handling in main()

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: return errors from block group creation functions
David Sterba [Mon, 22 Aug 2016 14:57:15 +0000 (16:57 +0200)]
btrfs-progs: mkfs: return errors from block group creation functions

No more BUG_ONs, we don't care about cleanup as the filesystem is
supposed to be marked as partial.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: two staged filesystem creation
David Sterba [Mon, 22 Aug 2016 14:32:24 +0000 (16:32 +0200)]
btrfs-progs: two staged filesystem creation

The filesystem existence on a device is manifested by the signature,
during the mkfs process we write it first and then create other
structures. Such filesystem is not valid and should not be registered
during device scan nor listed among devices from blkid.

This patch will introduce two staged creation. In the first phase, the
signature is wrong, but recognized as a partially created filesystem (by
open or scan helpers). Once we successfully create and write everything,
we fixup the signature. At this point automated scanning should find
a valid filesystem on all devices.

We can also rely on the partially created filesystem to do better error
handling during creation. We can just bail out and do not need to clean
up.

The partial signature is '!BHRfS_M', can be shown by

  btrfs inspect-internal dump-super -F image

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: do not scan partially initialized devices
David Sterba [Mon, 22 Aug 2016 14:31:11 +0000 (16:31 +0200)]
btrfs-progs: mkfs: do not scan partially initialized devices

We call scan ioctl on the devices too early, when most of the filesystem
structures are not yet created. Move the registration to the end, after
the filesystem gets closed.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: introduce signature for a partially set up filesystem
David Sterba [Fri, 19 Aug 2016 14:40:14 +0000 (16:40 +0200)]
btrfs-progs: introduce signature for a partially set up filesystem

Currently the superblock is created first, with a valid signaure, but
the rest of the filesystem is missing. When the creation process is
interrupted, the filesystem still might be considered as valid.

To prevent that, create the filesytem with an invalid signature that
would be still recognized during the mkfs process, and finalize at the
end.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: make superblock reading/scanning api more generic
David Sterba [Fri, 19 Aug 2016 14:36:40 +0000 (16:36 +0200)]
btrfs-progs: make superblock reading/scanning api more generic

We'll add more modes that affect scanning.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: pass OPEN_CTREE flags as unsigned
David Sterba [Fri, 19 Aug 2016 14:20:36 +0000 (16:20 +0200)]
btrfs-progs: pass OPEN_CTREE flags as unsigned

As we're passing a set of flags, the enum type is not appropriate.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: build: add more debugging features
David Sterba [Fri, 19 Aug 2016 14:06:41 +0000 (16:06 +0200)]
btrfs-progs: build: add more debugging features

Add options to show file and line or stack trace for error/warning
messages that use the common helpers. Possible to let any error stop
execution for ease of analysis and debugging.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: switch ternary op to an if in cmd_subvol_show
David Sterba [Fri, 19 Aug 2016 13:45:04 +0000 (15:45 +0200)]
btrfs-progs: switch ternary op to an if in cmd_subvol_show

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: print-tree: Print hex and human readable root flags
Qu Wenruo [Fri, 19 Aug 2016 08:13:08 +0000 (16:13 +0800)]
btrfs-progs: print-tree: Print hex and human readable root flags

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: print-tree: Print human readable inode flags
Qu Wenruo [Fri, 19 Aug 2016 08:13:07 +0000 (16:13 +0800)]
btrfs-progs: print-tree: Print human readable inode flags

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert-test: Check if the ext2_save/image is read only
Qu Wenruo [Fri, 19 Aug 2016 08:13:06 +0000 (16:13 +0800)]
btrfs-progs: convert-test: Check if the ext2_save/image is read only

Old convert codes uses both 0400 permission and INODE_READONLY flag to
make the converted ext2 image readonly.

While new convert treat the inode just as normal inode, with no special
inode flag and uses 0600 permission.
This makes user able to modify converted image unintentionally and make
rollback fails.

This test case will test the regression.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: convert: Fix a regression that ext2_save/image is not readonly
Qu Wenruo [Fri, 19 Aug 2016 08:13:05 +0000 (16:13 +0800)]
btrfs-progs: convert: Fix a regression that ext2_save/image is not readonly

The new convert treats the convert image as a normal file, without any
special flags and permissions.

This is different from original code:
1) Permission changed from 0400 to 0600
2) Inode lacks READONLY flag

This makes we can read-write mount the ext2 image and cause rollback
failure.

Follow old code behavior, use 0400 permission and add back READONLY
flag to fix it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: docs: refer to btrfs(5) from btrfs(8)
David Sterba [Thu, 18 Aug 2016 17:50:06 +0000 (19:50 +0200)]
btrfs-progs: docs: refer to btrfs(5) from btrfs(8)

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: docs: update btrfs-quota manual page, more sections
David Sterba [Thu, 18 Aug 2016 17:25:10 +0000 (19:25 +0200)]
btrfs-progs: docs: update btrfs-quota manual page, more sections

Add more overall sections.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: docs: update btrfs-quota manual page
David Sterba [Thu, 18 Aug 2016 17:25:10 +0000 (19:25 +0200)]
btrfs-progs: docs: update btrfs-quota manual page

Copy the intoductory and usecases from the text written by Arne Jansen,

https://git.kernel.org/cgit/linux/kernel/git/arne/qgroups-doc.git/

The graphics missing for now.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: mkfs: switch to new error message helpers
David Sterba [Thu, 18 Aug 2016 16:38:34 +0000 (18:38 +0200)]
btrfs-progs: mkfs: switch to new error message helpers

Do not use fprintf, adjust messages, add verbose errno or at least the
errorr code if there's no clear mapping to a string.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: no BUG_ON in close_ctree
David Sterba [Thu, 18 Aug 2016 15:44:18 +0000 (17:44 +0200)]
btrfs-progs: no BUG_ON in close_ctree

There's no reason for it.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: adjust command line options for the low-memory mode
David Sterba [Wed, 17 Aug 2016 17:33:24 +0000 (19:33 +0200)]
btrfs-progs: check: adjust command line options for the low-memory mode

Change the single-purpose option --low-memory to a generic option that
takes the mode. Currently supported are the original mode and the
low-memory in the same way.

Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce low memory mode
Lu Fengqi [Sun, 24 Apr 2016 07:47:12 +0000 (15:47 +0800)]
btrfs-progs: check: introduce low memory mode

Introduce a new fsck mode: low memory mode.

Old btrfsck is working efficiently but uses some memory for each extent
item.  This method will ensure extents are only iterated once at
extent/chunk tree check process.

But since it uses some memory for each extent item, for a large fs with
several TB metadata, this can easily eat up memory and cause OOM.

To handle such limitation and improve scalability, the new low-memory
mode will not use any heap memory to record which extent is checked.
Instead it will use extent backref to avoid most of uneeded checks on
shared fs/subvolume tree blocks.
And with the use forward and backward reference cross check, we can also
ensure every tree block is at least checked once.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce traversal function for fsck
Lu Fengqi [Sun, 24 Apr 2016 07:35:41 +0000 (15:35 +0800)]
btrfs-progs: check: introduce traversal function for fsck

Introduce a new function traverse_tree_block() to do pre-order
traversal, to co-operate with new fs/subvolume tree skip function.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to speed up fs tree check
Lu Fengqi [Sun, 24 Apr 2016 07:24:56 +0000 (15:24 +0800)]
btrfs-progs: check: introduce function to speed up fs tree check

Introduce function should_check() to reduced duplicated tree block check
for fs/subvolume tree.

The idea is, we only check the fs/subvolue tree block if we have the
lowest referencer rootid, according to extent tree.

In that case, we can skip a lot of fs/subvolume tree block checks if
there are a lot of snapshots.

Although we will do a lot of extent tree searches for it.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce main entry function for checking leaf items
Lu Fengqi [Sun, 24 Apr 2016 07:07:59 +0000 (15:07 +0800)]
btrfs-progs: check: introduce main entry function for checking leaf items

Introduce an entry function, check_leaf_items() to check all
known/valuable items and update related accounting like total_bytes and
csum_bytes.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check chunk item
Lu Fengqi [Sun, 24 Apr 2016 07:00:24 +0000 (15:00 +0800)]
btrfs-progs: check: introduce function to check chunk item

Introduce function check_chunk_item() to check a chunk item.
It will check all chunk stripes with dev extents and the corresponding
block group item.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check block group item
Lu Fengqi [Sun, 24 Apr 2016 06:41:41 +0000 (14:41 +0800)]
btrfs-progs: check: introduce function to check block group item

Introduce function check_block_group_item() to check a block group item.
It will check the referencer chunk and the used space accounting with
extent tree.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check dev used space
Lu Fengqi [Sun, 24 Apr 2016 06:17:10 +0000 (14:17 +0800)]
btrfs-progs: check: introduce function to check dev used space

Introduce function check_dev_item() to check used space with dev extent
items.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check dev extent item
Lu Fengqi [Sun, 24 Apr 2016 05:54:28 +0000 (13:54 +0800)]
btrfs-progs: check: introduce function to check dev extent item

Introduce function check_dev_extent_item() to find its referencer chunk.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check an extent
Lu Fengqi [Sun, 24 Apr 2016 05:45:28 +0000 (13:45 +0800)]
btrfs-progs: check: introduce function to check an extent

Introduce function check_extent_item() using previously introduced
functions.

With previous function to check referencer and backref, this function
can be quite easy.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check shared data backref
Lu Fengqi [Sun, 24 Apr 2016 05:33:47 +0000 (13:33 +0800)]
btrfs-progs: check: introduce function to check shared data backref

Introduce the function check_shared_data_backref() to check the
referencer of a given shared data backref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check referencer for data backref
Lu Fengqi [Sat, 23 Apr 2016 08:58:24 +0000 (16:58 +0800)]
btrfs-progs: check: introduce function to check referencer for data backref

Introduce new function check_extent_data_backref() to search referencer
for a given data backref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check shared block ref
Lu Fengqi [Sat, 23 Apr 2016 08:35:04 +0000 (16:35 +0800)]
btrfs-progs: check: introduce function to check shared block ref

Introduce function check_shared_block_backref() to check shared block
ref.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to check referencer of a backref
Lu Fengqi [Sat, 23 Apr 2016 08:08:40 +0000 (16:08 +0800)]
btrfs-progs: check: introduce function to check referencer of a backref

Introduce a new function check_tree_block_backref() to check if a
backref points to correct referencer.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
8 years agobtrfs-progs: check: introduce function to query tree block level
Lu Fengqi [Sat, 23 Apr 2016 07:56:50 +0000 (15:56 +0800)]
btrfs-progs: check: introduce function to query tree block level

Introduce function query_tree_block_level() to resolve tree block level
by the following method:
1) tree block backref level
2) tree block header level

And only when header level == backref level, and transid matches, it will
return the tree block level.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>