David Sterba [Thu, 20 Jul 2017 15:53:56 +0000 (17:53 +0200)]
btrfs-progs: tests: enhance API to request type of the converted filesystem
We'll use mount -t $fstype later on, extend the API so we don't have to
parse the type from other parameters.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 19 Jul 2017 19:01:30 +0000 (21:01 +0200)]
btrfs-progs: docs: adjust wording for subvol delete
Slightly update the text about deletion after the discussion on IRC.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 17 Jul 2017 15:58:50 +0000 (17:58 +0200)]
btrfs-progs: docs: enhance documentation of 'btrfs device ready'
There were questions raised about the purpose of the 'dev ready'
command.
Link: https://marc.info/?l=linux-btrfs&m=149944574207325
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 17 Jul 2017 14:44:02 +0000 (16:44 +0200)]
btrfs-progs: docs: move deprecated mount option to own section
- alloc_start
- recovery
- subvolrootid
Signed-off-by: David Sterba <dsterba@suse.com>
Anand Jain [Wed, 12 Jul 2017 22:47:11 +0000 (06:47 +0800)]
btrfs-progs: subvol show: add support to search subvolume by rootid or uuid
Unless the top level is mounted there is no way to know the
details of all the subvolume. For example:
mount -o subvol=sv1/newsv1 /dev/sdb /btrfs
btrfs su list /btrfs
ID 257 gen 12 top level 5 path sv1
ID 258 gen 9 top level 257 path sv1/snap
ID 259 gen 11 top level 257 path sv1/newsv1
You can't subvol show for sv1 and sv1/snap as its paths aren't
accessible to the user unless the its top level is mounted.
This patch adds two new options to the existing btrfs subvol show
cli. They are --rootid/-r or --uuid/-u, with this now the user will
be able to look for a subvolume using the rootid OR the uuid.
./btrfs su show -r 257 /btrfs
sv1
Name: sv1
UUID:
30129358-c69d-3e4a-a662-
29509cc69c95
Parent UUID: -
Received UUID: -
Creation time: 2017-07-11 20:32:57 +0800
Subvolume ID: 257
Generation: 12
Gen at creation: 7
Parent ID: 5
Top level ID: 5
Flags: -
Snapshot(s):
sv1/snap
Signed-off-by: Anand Jain <anand.jain@oracle.com>
[ minor adjustments in the help text ]
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Wed, 12 Jul 2017 20:04:49 +0000 (13:04 -0700)]
btrfs-progs: build: Enable ThreadSanitizer, using D=tsan
Tested with clang-3.9. We have to enable PIE,
(https://clang.llvm.org/docs/ThreadSanitizer.html)
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Justin Maggard [Fri, 9 Jun 2017 18:09:35 +0000 (11:09 -0700)]
btrfs-progs: Fix an infinite loop in btrfs_next_bg
I've run into a couple filesystems where btrfs-find-root would spin
indefinitely.
If the first cache extent start location is 0, we end up in an infinite
loop in btrfs_next_bg(). Fix it by checking for that situation, and
jumping to the next bg if necessary.
Fixes:
e2e0dae9 (btrfs-progs: volume: Fix a bug causing btrfs-find-root to skip first chunk)
Signed-off-by: Justin Maggard <jmaggard@netgear.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Mon, 26 Jun 2017 17:34:41 +0000 (11:34 -0600)]
Btrfs-progs: fix infinite loop in find_free_extent
If the found %ins is crossing a stripe len, ie. BTRFS_STRIPE_LEN, we'd
search again with a stripe-aligned %search_start. The current code
calculates %search_start by adding a wrong offset, in order to fix it, the
start position of the block group should be taken, otherwise, it'll end up
with looking at the same block group forever.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Mon, 10 Jul 2017 21:29:09 +0000 (14:29 -0700)]
btrfs-progs: Tighten integer types in print-tree
There are likely more places where the wrong size types are used, but
these tripped Clang's warnings because they eventually get passed to
printf.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Mon, 10 Jul 2017 21:29:08 +0000 (14:29 -0700)]
btrfs-progs: tests: Fix missing internal deps in check and misc tests
Doing a straight 'make test' would fail because some misc and fsck
tests require particular tools to already be built. Add dependencies
at the Makefile and shell-script level.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Anand Jain [Wed, 12 Jul 2017 07:20:27 +0000 (15:20 +0800)]
btrfs-progs: subvol show: fix the path use full_path as provided by the root info
This is a kind of preparatory patch for the patch which will add
--rootid and --uuid options for the btrfs subvol show command.
As of now btrfs subvol show is using the external user provided subvol
path to show in the output. Which is kind of confusing.
btrfs su show /btrfs
/btrfs <--
Name: <FS_TREE>
It will be even more confusing when proposed --uuid or --rootid
options are used.
btrfs su show --rootid 258 /btrfs
/btrfs <--
Name: snap <--
UUID:
9630a45f-e647-4242-bd19-
97590b4e20b2
Parent UUID:
30129358-c69d-3e4a-a662-
29509cc69c95
Received UUID: -
Creation time: 2017-07-12 12:43:28 +0800
Subvolume ID: 258
Generation: 9
Gen at creation: 9
Parent ID: 257
Top level ID: 257
Flags: -
Snapshot(s):
Now with this patch, it will only show what is provided by the root_info.
btrfs su show --rootid 258 /btrfs
sv1/snap <--
Name: snap
UUID:
9630a45f-e647-4242-bd19-
97590b4e20b2
Parent UUID:
30129358-c69d-3e4a-a662-
29509cc69c95
Received UUID: -
Creation time: 2017-07-12 12:43:28 +0800
Subvolume ID: 258
Generation: 9
Gen at creation: 9
Parent ID: 257
Top level ID: 257
Flags: -
Snapshot(s):
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Wed, 12 Jul 2017 22:10:13 +0000 (00:10 +0200)]
btrfs-progs: image: fix typos in messages
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
[ separated from the previous patch ]
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Wed, 12 Jul 2017 20:05:10 +0000 (13:05 -0700)]
btrfs-progs: image: Fix data races when reporting progress
Making the code data-race safe requires that reads *and* writes
happen under a mutex lock, if any of the access are writes. See
Dmitri Vyukov, "Benign data races: what could possibly go wrong?"
for more details.
The fix here was to put most of the main loop of restore_worker
under a mutex lock.
This race was detected using fsck-tests/012-leaf-corruption.
==================
WARNING: ThreadSanitizer: data race
Write of size 4 by main thread:
#0 add_cluster btrfs-progs/image/main.c:1931
#1 restore_metadump btrfs-progs/image/main.c:2566
#2 main btrfs-progs/image/main.c:2859
Previous read of size 4 by thread T6:
#0 restore_worker btrfs-progs/image/main.c:1720
Location is stack of main thread.
Thread T6 (running) created by main thread at:
#0 pthread_create <null>
#1 mdrestore_init btrfs-progs/image/main.c:1868
#2 restore_metadump btrfs-progs/image/main.c:2534
#3 main btrfs-progs/image/main.c:2859
SUMMARY: ThreadSanitizer: data race btrfs-progs/image/main.c:1931 in
add_cluster
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adam Buchbinder [Wed, 12 Jul 2017 20:05:38 +0000 (13:05 -0700)]
btrfs-progs: convert: Fix data race when reporting progress
The status display was reading the state while the task was updating
it. Use a mutex to prevent the race.
This race was detected using ThreadSanitizer and
misc-tests/005-convert-progress-thread-crash.
==================
WARNING: ThreadSanitizer: data race
Write of size 8 by main thread:
#0 ext2_copy_inodes btrfs-progs/convert/source-ext2.c:853
#1 copy_inodes btrfs-progs/convert/main.c:145
#2 do_convert btrfs-progs/convert/main.c:1297
#3 main btrfs-progs/convert/main.c:1924
Previous read of size 8 by thread T1:
#0 print_copied_inodes btrfs-progs/convert/main.c:124
Location is stack of main thread.
Thread T1 (running) created by main thread at:
#0 pthread_create <null>
#1 task_start btrfs-progs/task-utils.c:50
#2 do_convert btrfs-progs/convert/main.c:1295
#3 main btrfs-progs/convert/main.c:1924
SUMMARY: ThreadSanitizer: data race
btrfs-progs/convert/source-ext2.c:853 in ext2_copy_inodes
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:35 +0000 (17:19 +0800)]
btrfs-progs: Refactor chunk creation functions to use btrfs_fs_info
4 functions are involved in this refactor: btrfs_make_block_group()
btrfs_make_block_groups(), btrfs_alloc_chunk, btrfs_alloc_data_chunk().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:34 +0000 (17:19 +0800)]
btrfs-progs: Remove btrfs_read_super_device as there is no implementation
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:33 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_add_device() to use btrfs_fs_info
BTW, there is a duplicated definition of btrfs_add_device() in
volumes.h, also remove it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:32 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_chunk_readonly to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:31 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_add_system_chunk to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:30 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_read_sys_array/chunk_tree to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:29 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_find_device to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:28 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_check_chunk_valid to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:27 +0000 (17:19 +0800)]
btrfs-progs: Refactor write_all_supers and its callers to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:26 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_readahead_tree_block to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:25 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_find_tree_block to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:24 +0000 (17:19 +0800)]
btrfs-progs: Refactor read_extent_data to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:23 +0000 (17:19 +0800)]
btrfs-progs: btrfstune: Refactor change_devices_uuid to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:22 +0000 (17:19 +0800)]
btrfs-progs: Refactor write_tree_block to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:21 +0000 (17:19 +0800)]
btrfs-progs: Refactor write_and_map_eb to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:20 +0000 (17:19 +0800)]
btrfs-progs: Refactor csum_tree_block to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:19 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_next_bg and its callers to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:18 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_num_copies to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 13 Jun 2017 09:19:17 +0000 (17:19 +0800)]
btrfs-progs: Refactor btrfs_map_block and its variants to use btrfs_fs_info
Just to keep the 1st paramter the same as kernel.
We can also save a few lines since the parameter is shorter now.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 31 May 2017 05:56:09 +0000 (13:56 +0800)]
btrfs-progs: lowmem check: Fix false alert on missing chunk or dev extent
When checking chunk or dev extent, lowmem mode uses chunk length as dev
extent length, and if they mismatch, report missing chunk or dev extent
like:
------
ERROR: chunk[256
4324327424) stripe 0 did not find the related dev extent
ERROR: chunk[256
4324327424) stripe 1 did not find the related dev extent
ERROR: chunk[256
4324327424) stripe 2 did not find the related dev extent
------
However, only for Single/DUP/RAID1 profiles chunk length is the same as
dev extent length.
For other profiles, this will cause tons of false alert.
Fix it by using correct stripe length when checking chunk and dev extent
items.
This fixes the mkfs test failure when using lowmem mode check.
Reported-by: Kai Krakow <hurikhan77@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 31 May 2017 05:56:08 +0000 (13:56 +0800)]
btrfs-progs: Introduce function to get correct stripe length
Introduce a new function, btrfs_get_chunk_stripe_len() to get correct
stripe length.
This is very handy for lowmem mode, which checks the mapping between
device extent and chunk item.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 31 May 2017 05:56:07 +0000 (13:56 +0800)]
btrfs-progs: check: Reuse btrfs_check_chunk_valid in lowmem mode
Before this patch, btrfs check lowmem mode manually checks found chunk
item, even we already have the generic chunk validation checker,
btrfs_check_chunk_valid().
This patch will use btrfs_check_chunk_valid() to replace open-coded
chunk validation checker in check_chunk_item().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 31 May 2017 05:56:06 +0000 (13:56 +0800)]
btrfs-progs: Enhance chunk item validation check
btrfs_check_chunk_valid() doesn't check if
1) chunk flag has conflicting flags
For example chunk type DATA|METADATA|RAID1|RAID10 is completely
invalid, while current check_chunk_valid() can't detect it.
2) num_stripes is invalid for RAID10
Num_stripes 5 is not valid for RAID10.
This patch will enhance btrfs_check_chunk_valid() to handle above cases.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 25 May 2017 06:21:50 +0000 (14:21 +0800)]
btrfs-progs: Introduce wrapper to recover raid56 data
Introduce a wrapper to recover raid56 data.
The logical is the same with kernel one, but with different interfaces,
since kernel ones cares the performance while in btrfs we don't care
that much.
And the interface is more caller friendly inside btrfs-progs.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 25 May 2017 06:21:49 +0000 (14:21 +0800)]
btrfs-progs: raid56: Allow raid6 to recover data and P
Copied from kernel lib/raid6/recov.c.
Minor modifications includes:
- Rename from raid6_datap_recov_intx() to raid5_recov_datap()
- Rename parameter from faila to dest1
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 25 May 2017 06:21:48 +0000 (14:21 +0800)]
btrfs-progs: raid56: Allow raid6 to recover 2 data stripes
Copied from kernel lib/raid6/recov.c raid6_2data_recov_intx1() function.
With the following modification:
- Rename to raid6_recov_data2() for shorter name
- s/kfree/free/g modification
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 25 May 2017 06:21:47 +0000 (14:21 +0800)]
btrfs-progs: raid56: Introduce tables for RAID6 recovery
Use kernel RAID6 galois tables for later RAID6 recovery.
Galois tables file, kernel-lib/tables.c is generated by user space
program, mktable.
Galois field tables declaration, in kernel-lib/raid56.h, is completely
copied from kernel.
The mktables.c is copied from kernel with minor header/macro
modification, to ensure the generated tables.c works well in
btrfs-progs.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 25 May 2017 06:21:46 +0000 (14:21 +0800)]
btrfs-progs: raid56: Introduce raid56 header for later recovery usage
Introduce a new header, kernel-lib/raid56.h, for later raid56 works.
It contains 2 functions, from original btrfs-progs code:
void raid6_gen_syndrome(int disks, size_t bytes, void **ptrs);
int raid5_gen_result(int nr_devs, size_t stripe_len, int dest, void **data);
Will be expanded later and some part of it(RAID6 recover part) may keep
sync with kernel later.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ unify gpl header, rename header macro ]
Signed-off-by: David Sterba <dsterba@suse.com>
Philipp Hahn [Mon, 15 May 2017 17:00:23 +0000 (19:00 +0200)]
btrfs-progs: Fix slot >= nritems
Running "btrfsck --repair /dev/sdd2" crashed as it can happen in
(corrupted) file systems, that slot > nritems:
> (gdb) bt full
> #0 0x00007ffff7020e71 in __memmove_sse2_unaligned_erms () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x0000000000438764 in btrfs_del_ptr (trans=<optimized out>, root=0x6e4fe0, path=0x1d17880, level=0, slot=7)
> at ctree.c:2611
> parent = 0xcd96980
> nritems = <optimized out>
> __func__ = "btrfs_del_ptr"
> #2 0x0000000000421b15 in repair_btree (corrupt_blocks=<optimized out>, root=<optimized out>) at cmds-check.c:3539
> key = {objectid =
77990592512, type = 168 '\250', offset = 16384}
> trans = 0x8f48c0
> path = 0x1d17880
> level = 0
> #3 check_fs_root (wc=<optimized out>, root_cache=<optimized out>, root=<optimized out>) at cmds-check.c:3703
> corrupt = 0x1d17880
> corrupt_blocks = {root = {rb_node = 0x6e80c60}}
> path = {nodes = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, slots = {0, 0, 0, 0, 0, 0, 0, 0}, locks = {0, 0,
> 0, 0, 0, 0, 0, 0}, reada = 0, lowest_level = 0, search_for_split = 0, skip_check_block = 0}
> nrefs = {bytenr = {
271663104,
271646720,
560021504, 0, 0, 0, 0, 0}, refs = {1, 1, 1, 0, 0, 0, 0, 0}}
> wret =
215575372
> root_node = {cache = {rb_node = {__rb_parent_color = 0, rb_right = 0x0, rb_left = 0x0}, objectid = 0,
> start = 0, size = 0}, root_cache = {root = {rb_node = 0x0}}, inode_cache = {root = {
> rb_node = 0x781c80}}, current = 0x819530, refs = 0}
> status =
215575372
> rec = 0x1
> #4 check_fs_roots (root_cache=0xcd96b6d, root=<optimized out>) at cmds-check.c:3809
> path = {nodes = {0x6eed90, 0x6a2f40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}, slots = {18, 2, 0, 0, 0, 0, 0, 0},
> locks = {0, 0, 0, 0, 0, 0, 0, 0}, reada = 0, lowest_level = 0, search_for_split = 0,
> skip_check_block = 0}
> key = {objectid = 323, type = 132 '\204', offset =
18446744073709551615}
> wc = {shared = {root = {rb_node = 0x0}}, nodes = {0x0, 0x0, 0x7fffffffe428, 0x0, 0x0, 0x0, 0x0, 0x0},
> active_node = 2, root_level = 2}
> leaf = 0x6e4fe0
> tmp_root = 0x6e4fe0
> #5 0x00000000004287c3 in cmd_check (argc=
215575372, argv=0x1d17880) at cmds-check.c:11521
> root_cache = {root = {rb_node = 0x98c2940}}
> info = 0x6927b0
> bytenr = 6891440
> tree_root_bytenr = 0
> uuidbuf = "
f65ff1a1-76ef-456e-beb5-
c6c3841e7534"
> num =
215575372
> readonly =
218080104
> qgroups_repaired = 0
> #6 0x000000000040a41f in main (argc=3, argv=0x7fffffffebe8) at btrfs.c:243
> cmd = 0x689868
> bname = <optimized out>
> ret = <optimized out>
in that case the count of remaining items (nritems - slot - 1) gets
negative. That is then casted to (unsigned long len), which leads to the
observed crash.
Change the tests before the move to handle only the non-corrupted case,
were slow < nritems.
This does not fix the corruption, but allows btrfsck to finish without
crashing.
Signed-off-by: Philipp Hahn <hahn@univention.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 May 2017 03:11:55 +0000 (11:11 +0800)]
btrfs-progs: Refactor read_node_slot function to get rid of btrfs_root
parameter
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:51:08 +0000 (10:51 +0800)]
btrfs-progs: Refactor read_tree_block to get rid of btrfs_root
The only reasom read_tree_block() needs a btrfs_root parameter is to get
its node/sector size.
And long ago, I have already introduced a compactible interface,
read_tree_block_fs_info() to pass btrfs_fs_info instead of btrfs_root.
Since we have cleaned up all root->sector/node/stripesize users, we
should be OK to refactor read_tree_block() function.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:36:19 +0000 (10:36 +0800)]
btrfs-progs: Refactor btrfs_root paramters in btrfs-corrupt-block.c
Refactor the following functions and its callers to get rid of
incorrectly passed btrfs_root parameters:
1) corrupt_keys()
2) corrupt_metadata_block()
The only reason passing btrfs_root for them is to get block sizes.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:22:22 +0000 (10:22 +0800)]
btrfs-progs: Remove block size members in btrfs_root
Finally, we can get rid of per tree block size members now.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:21:34 +0000 (10:21 +0800)]
btrfs-progs: Refactor sectorsize users in cmds-inspect-dump-tree.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:14:19 +0000 (10:14 +0800)]
btrfs-progs: Refactor sectorsize users in convert/source-ext2.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:13:27 +0000 (10:13 +0800)]
btrfs-progs: Refactor sectorsize users in convert/main.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:11:46 +0000 (10:11 +0800)]
btrfs-progs: Refactor sectorsize in convert/source-fs.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:09:53 +0000 (10:09 +0800)]
btrfs-progs: Refactor sectorsize users in free-space-tree.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:09:10 +0000 (10:09 +0800)]
btrfs-progs: Refactor sectorsize users in volumes.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:06:40 +0000 (10:06 +0800)]
btrfs-progs: Refactor sectorsize users in file.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:05:22 +0000 (10:05 +0800)]
btrfs-progs: Refactor sectorsize users in free-space-cache.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:02:24 +0000 (10:02 +0800)]
btrfs-progs: Refactor sectorsizes users in file-item.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 02:00:01 +0000 (10:00 +0800)]
btrfs-progs: Refactor sectorsize users in mkfs/main.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:54:49 +0000 (09:54 +0800)]
btrfs-progs: Refactor nodesize users in cmds-inspect-tree-stats.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:51:55 +0000 (09:51 +0800)]
btrfs-progs: Refactor nodesize users in qgroup-verify.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:50:51 +0000 (09:50 +0800)]
btrfs-progs: Refactor nodesize user in print-tree.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:48:52 +0000 (09:48 +0800)]
btrfs-progs: Refactor block sizes users in extent-tree.c
And in the refactoring, also refactors a open-coded round_up().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:40:25 +0000 (09:40 +0800)]
btrfs-progs: Refactor nodesize users in utils.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:39:03 +0000 (09:39 +0800)]
btrfs-progs: Refactor nodesize user in btrfstune.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:38:03 +0000 (09:38 +0800)]
btrfs-progs: Refactor block sizes users in cmds-check.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:29:54 +0000 (09:29 +0800)]
btrfs-progs: Refactor nodesize users in image/main.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:27:32 +0000 (09:27 +0800)]
btrfs-progs: Refactor nodesize user in extent_io.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:26:48 +0000 (09:26 +0800)]
btrfs-progs: Refactor block sizes users in cmds-restore.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:24:55 +0000 (09:24 +0800)]
btrfs-progs: Refactor block sizes users in backref.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:23:59 +0000 (09:23 +0800)]
btrfs-progs: Refactor block sizes users in chunk-recover.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:21:05 +0000 (09:21 +0800)]
btrfs-progs: Refactor block sizes users in btrfs-map-logical.c
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:17:41 +0000 (09:17 +0800)]
btrfs-progs: Refactor block sizes users in ctree.c and ctree.h
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Thu, 18 May 2017 01:11:20 +0000 (09:11 +0800)]
btrfs-progs: Refactor block sizes users in btrfs-corrupt-block.c
Convert root->sectorsize/nodesize users in btrfs-corrupt-block.
This provides the basis to further refactor incorrect btrfs_root
parameter to btrfs_fs_info parameter.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Wed, 17 May 2017 09:17:56 +0000 (17:17 +0800)]
btrfs-progs: Refactor block sizes users in disk-io.c
Since we have cached block sizes in fs_info, there is no need to specify
these sizes in btrfs_setup_root() function.
And refactor all root->sector/node/stripesize users in disk-io.c.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Wed, 17 May 2017 08:57:35 +0000 (16:57 +0800)]
btrfs-progs: Introduce sectorsize nodesize and stripesize members for
btrfs_fs_info
Just like what we do in kernel, since we will not support different
leaf/node/stripe size per tree, there is no need to store these block
sizes in btrfs_root.
This patch will introduce these block size members into btrfs_fs_info
structure, allowing us to convert such usage in later patches.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Wed, 17 May 2017 08:42:50 +0000 (16:42 +0800)]
btrfs-progs: Remove deprecated leafsize usage
Leafsize is deprecated for a long time, and kernel has already updated
ctree.h to rename sb->leafsize to sb->__unused_leafsize.
This patch will remove normal users of leafsize:
1) Remove leafsize member from btrfs_root structure
Now only root->nodesize and root->sectorisze.
No longer root->leafsize.
2) Remove @leafsize parameter from btrfs_setup_root() function
Since no root->leafsize, no need for @leafsize parameter.
The remaining user of leafsize will be:
1) btrfs inspect-internal dump-super
Reformat the "leafsize" output to "leafsize (deprecated)" and
use le32_to_cpu() to do the cast manually.
2) mkfs
We still need to set sb->__unused_leafsize to nodesize.
Do the manual cast too.
3) convert
Same as mkfs, these two superblock setup should be merged later
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Mon, 15 May 2017 08:27:38 +0000 (16:27 +0800)]
btrfs-progs: Cleanup open-coded btrfs_chunk_item_size
In btrfs_check_chunk_valid() we calculate chunk item using open code,
use an existing helper btrfs_chunk_item_size() instead.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 18 May 2017 16:49:00 +0000 (18:49 +0200)]
btrfs-progs: docs: document conventions
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 30 Jun 2017 16:34:24 +0000 (18:34 +0200)]
Btrfs progs v4.11.1
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 31 Mar 2017 12:20:21 +0000 (14:20 +0200)]
btrfs-progs: update CHANGES for v4.11.1
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Mon, 26 Jun 2017 23:55:02 +0000 (17:55 -0600)]
Btrfs-progs: convert: do not clear header rev
So btrfs_set_header_flags() vs btrfs_set_header_flag, the difference is
sort of similar to "=" vs "|=", when creating and initialising a new
extent buffer, convert uses the former one which clears header_rev by
accident.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Wed, 28 Jun 2017 21:29:49 +0000 (15:29 -0600)]
Btrfs-progs: convert: do not clear nodatasum flag in inode item
With the current btrfs-convert, if we convert a ext4 without data checksum,
it'd not set nodatasum flag in inode item, nor create csum item, reading
file ends up with checksum errors.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Liu Bo [Tue, 27 Jun 2017 21:25:45 +0000 (15:25 -0600)]
Btrfs-progs: man: nodesize must be power of 2 now
This updates mkfs.btrfs's man page with the new limitation that nodesize must
be a power of 2 as well.
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Thu, 22 Jun 2017 08:12:56 +0000 (16:12 +0800)]
btrfs-progs: lowmem check: Fix false alert about file extent interrupt
As Qu mentioned in this thread
(https://www.spinics.net/lists/linux-btrfs/msg64469.html), compression
can cause regular extent to co-exist with an inlined extent. This
coexistence makes things confusing. Since it is currently allowed and
can appear in a filesystem, fix btrfsck to prevent a bunch of error
reports to appear that will make user feel uneasy.
When checking a file extent, record the extent_end of the regular extent
to check if there is a gap between the regular extents. Normally there
is only one inlined extent, so the extent_end of inlined extent is
useless. However, if a regular extent can co-exist with an inlined
extent, the extent_end of the inlined extent also needs to be recorded.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 16 Jun 2017 01:43:23 +0000 (09:43 +0800)]
btrfs-progs: tests: Add test case to check file hole extents with NO_HOLES flag
Add test case which we have NO_HOLES incompat flag while still have
hole file extent.
This can be created by enabling NO_HOLES feature on an existing
filesystem, which lowmem mode would cause false alert for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 16 Jun 2017 01:43:22 +0000 (09:43 +0800)]
btrfs-progs: check: Fix false alert about EXTENT_DATA that shouldn't be a hole
Since the incompat feature NO_HOLES still allows us to have an explicit
hole file extent, current check is too strict and will cause false
alerts like:
root 5 EXTENT_DATA[257, 0] shouldn't be hole
Fix it by removing the strict file hole extent check.
Link: https://www.spinics.net/lists/linux-btrfs/msg66374.html
Reported-by: Henk Slager <eye1tm@gmail.com>
Tested-by: Henk Slager <eye1tm@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Sat, 3 Jun 2017 09:57:45 +0000 (15:27 +0530)]
btrfs-progs: convert: Add larger device support
With larger file system (in this case its 22TB), ext2fs_open() returns
EXT2_ET_CANT_USE_LEGACY_BITMAPS error message with
ext2fs_read_block_bitmap().
To overcome this issue,
(a) we need pass EXT2_FLAG_64BITS flag with ext2fs_open.
(b) use 64-bit functions like ext2fs_get_block_bitmap_range2,
ext2fs_inode_data_blocks2,ext2fs_read_ext_attr2
(c) use 64bit types with btrfs_convert_context fields
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=194795
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Mon, 26 Jun 2017 12:43:31 +0000 (14:43 +0200)]
btrfs-progs: convert: widen int types in convert context
The u32 types in the convert context might not be enough for some very
large filesytems (20TB). Use 64bit types to be safe.
Signed-off-by: Lakshmipathi.G <lakshmipathi.g@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Lakshmipathi.G [Thu, 22 Jun 2017 07:57:53 +0000 (13:27 +0530)]
btrfs-progs: Fix 'btrfs device stats --check' cli option
"Bug 194961 - btrfs device stats --check <folder> does not work"
The long option --check is not recognized as it's missing from the
option table.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=194961
Reported-by: Tomas Thiemel<thiemel@centrum.cz>
Signed-off-by: Lakshmipathi.G <Lakshmipathi.G@giis.co.in>
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Thu, 22 Jun 2017 17:49:55 +0000 (12:49 -0500)]
btrfs-progs: docs: replace <dev> with <device>
As discussed in #50, use 'device' everywhere a path do a device is
expected.
Pull request: #52
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Thu, 22 Jun 2017 01:52:01 +0000 (20:52 -0500)]
btrfs-progs: docs: Wrong section in ref to manpage
Pull request: #51
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 19 Jun 2017 16:21:44 +0000 (18:21 +0200)]
btrfs-progs: build: sync recent makefile changes to android.mk
Signed-off-by: David Sterba <dsterba@suse.com>
Kasijjuf [Wed, 14 Jun 2017 05:26:08 +0000 (00:26 -0500)]
btrfs-progs: docs: Expand confusing abbreviation in documentation
While the command interpreter may be able to disambiguate the meaning,
the reader is not helped by being forced to do so.
Pull request: #48
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 19 Jun 2017 13:08:50 +0000 (15:08 +0200)]
btrfs-progs: docs: update note about device deletion
User Kasijjuf points out the VFS initialism is not explained anywhere.
While this could be fixed, the whole note about inability to delete the
device by which the filesystem has been mounted, is wrong.
Issue: #49
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Tue, 6 Jun 2017 13:00:58 +0000 (15:00 +0200)]
btrfs-progs: send operates on ro snapshots only
While talking to another btrfs user on IRC today, it became clear that a
major point of confusion in the btrfs send manual is that it's not
telling the user soon enough that send/receive solely operates on
subvolume snapshots instead of the original (read/write) subvolumes.
So, change the first few lines to explicitly mention snapshots instead.
Technically, snapshots are also just subvolumes, but requiring this
level of technical detailed knowledge doesn't help the user who is just
trying out things.
Signed-off-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 5 Jun 2017 14:39:33 +0000 (16:39 +0200)]
btrfs-progs: docs: fix sentence for no-dump file attribute
User Kasijjuf found a nonsensical sentence in the maunal page.
Issue: #45
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 31 May 2017 11:49:08 +0000 (13:49 +0200)]
btrfs-progs: docs: update formatting of btrfs-property
Reported by a wiki user, that there are formatting artifacts in the
'get' section:
in html rendered as "The -t <em><type></em> option can be..."
This is probably due to the nesting '' and <>. We don't need the <> in
the explanation, as this is only to describe the command line syntax.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 31 May 2017 11:49:08 +0000 (13:49 +0200)]
btrfs-progs: docs: update formatting of btrfs-rescue
Reported by a wiki user, that the 'zero-log' section indentaion is
wrong.
Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 17 May 2017 01:41:31 +0000 (02:41 +0100)]
btrfs-progs: test for restoring multiple devices fs into a single device
Test that we are able to create an image from a multiple devices fs, that
we are able to restore that image into a single device and finally that we
are able to mount it.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
[ added shell quotation and chmod a+w so testsuite on NFS works ]
Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Wed, 17 May 2017 01:41:21 +0000 (02:41 +0100)]
btrfs-progs: Fix restoring image from multi devices fs into single device
We correctly build an image from a multiple devices filesystem but when
restoring the image into a single device we were missing updating the
number of devices in the superblock to the value 1 (we already took care
of setting the number of stripes to 1 for each chunk item and setting
the device id for each chunk item to match the device id from the super
block).
This missing update of the number of devices makes it impossible to mount
the restored filesystem on recent kernels, more specifically since the
linux kernel commit
99e3ecfcb9f4ca35192d20a5bea158b81f600062
("Btrfs: add more validation checks for superblock"), that produce the
following message in the dmesg/syslog:
[21097.542047] BTRFS error (device sdi): super_num_devices 2 mismatch with num_devices 1 found here
[21097.543972] BTRFS error (device sdi): failed to read chunk tree: -22
[21097.720360] BTRFS error (device sdi): open_ctree failed
So fix this by updating the number of devices to 1 in the superblock.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Tsutomu Itoh [Tue, 16 May 2017 03:01:53 +0000 (12:01 +0900)]
btrfs-progs: tests: remove variable quotation from convert-tests
In btrfs-progs-v4.11-rc1, the following convert-tests failed.
[TEST/conv] 008-readonly-image
[TEST/conv] readonly image test, btrfs defaults
failed: mke2fs -t ext4 -b 4096 -F /Build/btrfs-progs-v4.11-rc1/tests/test.img
test failed for case 008-readonly-image
Makefile:271: recipe for target 'test-convert' failed
make: *** [test-convert] Error 1
[TEST/conv] 009-common-inode-flags
[TEST/conv] common inode flags test, btrfs defaults
failed: mke2fs -t ext4 -b 4096 -F /Build/btrfs-progs-v4.11-rc1/tests/test.img
test failed for case 009-common-inode-flags
Makefile:271: recipe for target 'test-convert' failed
make: *** [test-convert] Error 1
So, remove quotes from $default_mke2fs.
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>