Zhao Lei [Mon, 27 Jul 2015 12:24:26 +0000 (20:24 +0800)]
btrfs-progs: Fix wrong address accessing by subthread in btrfs-convert
btrfs-convert sometimes show 'Assertion failed' in converting a nearly blank
file system, as:
create btrfs filesystem:
blocksize: 4096
nodesize: 16384
features: extref, skinny-metadata (default)
creating btrfs metadata.
creating ext2fs image file.
trans 7 running 5
ctree.c:363: btrfs_cow_block: Assertion `1` failed.
btrfs-convert(btrfs_cow_block+0x92)[0x40acaf]
btrfs-convert(btrfs_search_slot+0x1cb)[0x40c50f]
btrfs-convert(btrfs_csum_file_block+0x20f)[0x41d83a]
btrfs-convert[0x43422d]
btrfs-convert[0x4342cd]
btrfs-convert[0x4345ca]
btrfs-convert[0x434767]
btrfs-convert[0x435770]
btrfs-convert[0x439748]
btrfs-convert(main+0x13f8)[0x43b09d]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x335e01ecdd]
btrfs-convert[0x407649]
Reason is complex:
1: main thread allocated a block of memory,
shared with sub thread
2: main thread killed sub thread, and free above memory
3: main thread malloc a new one(in same address),
and use it
4: sub thread(which is not really quit), write into
this address, and caused this bug.
By adding some debug lines into code, we can see following output:
create btrfs filesystem:
blocksize: 4096
nodesize: 16384
features: extref, skinny-metadata (default)
creating btrfs metadata.
1: ctx(0x7ffe1abde230)->info=0xc65b80
2: task_period_start: will create periodic.timer_fd
3: task_stop: info->periodic.timer_fd = NULL
4: task_stop: begin pthread_cancel info->id=-
1746053376
5: task_stop: done pthread_cancel ret=0
6: task_stop: begin info->postfn
7: task_period_stop: periodic.timer_fd NULL
8: task_stop: done info->postfn
9: task_stop: done all
10: creating ext2fs image file.
trans 7 running 5
11: task_period_start: create periodic.timer_fd done info->periodic.timer_fd(0xc65b80)=7
12: btrfs_cow_block: root->fs_info->generation(0xc63568) = 5 trans->transid(0xc65b80)=7
13: ctree.c:368: btrfs_cow_block: Assertion `1` failed.
./btrfs-convert(btrfs_cow_block+0xda)[0x40ad37]
./btrfs-convert(btrfs_search_slot+0x1cb)[0x40c5b4]
./btrfs-convert(btrfs_insert_empty_items+0xac)[0x40d9f6]
./btrfs-convert(btrfs_record_file_extent+0xc0)[0x4183fe]
./btrfs-convert[0x435796]
./btrfs-convert[0x439b0c]
./btrfs-convert(main+0x13f8)[0x43b45d]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x335e01ecdd]
./btrfs-convert[0x407689]
Conclusion:
a: subthread should exit before step 5, but it is still running
in step 11
b: task_stop() hadn't close periodic.timer_fd in step3,
because periodic.timer_fd is not initialized yet.
c. address of 0xc65b80 is overwrited by subthread in step 11,
but this address is freed and re-malloc by main thread
before step 10, and used for trans->transid.
d: trans->transid which is overwrite by subthread caused error
in step 13.
Fix:
pthread_cancel() only send a cancellation request to the thread,
thread will quit in next cancellation point by default.
To make sub thread quit in time, this patch add pthread_join()
after pthread_cancel() call.
And to make pthread_join() works, pthread_detach() is removed.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Zhao Lei [Mon, 27 Jul 2015 13:01:50 +0000 (21:01 +0800)]
btrfs-progs: tests: Add -o loop to fsck-tests/012-leaf-corruption
To avoid following mount error in test:
mount: /root/btrfs/progs/tests/fsck-tests/012-leaf-corruption/test.img
is not a block device (maybe try `-o loop'?)
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Jul 2015 09:18:10 +0000 (17:18 +0800)]
btrfs-progs: convert: Avoid allocating metadata extent crossing stripe boundary
As convert implement its own alloc extent, avoid such metadata problem
too.
Reported-by: Chris Murphy <lists@colorremedies.com>
Reported-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Jul 2015 09:18:09 +0000 (17:18 +0800)]
btrfs-progs: extent-tree: avoid allocating tree block that crosses stripe boundary
Now find_free_extent() function won't return a metadata extent that
crosses stripe boundary.
Reported-by: Chris Murphy <lists@colorremedies.com>
Reported-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Jul 2015 09:18:08 +0000 (17:18 +0800)]
btrfs-progs: fsck: Check if a metadata tree block crossing stripe boundary
Kernel btrfs_map_block() function has a limitation that it can only
map BTRFS_STRIPE_LEN size.
That will cause scrub fails to scrub tree block which crosses strip
boundary, causing BUG_ON().
Normally, it's OK as metadata is always in metadata chunk and
BTRFS_STRIPE_LEN can always be divided by node/leaf size.
So without mixed block group, tree block won't cross stripe boundary.
But for mixed block group, especially for btrfs converted from ext4,
it's almost sure one or more tree blocks are not aligned with node size
and cross stripe boundary.
Causing bug with kernel scrub.
This patch will report the problem, although we don't have a good idea
how to fix it in user space until we add the ability to relocate tree
block in user space.
Also, kernel code should also be checked for such tree block alloc
problems.
Reported-by: Chris Murphy <lists@colorremedies.com>
Reported-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 23 Jul 2015 09:18:07 +0000 (17:18 +0800)]
btrfs-progs: print-tree: print stripe len of a chunk
Although it is fixed to BTRFS_STRIPE_LEN(64K) now, it's still used in a
lot of code, just output it for user who wants to trace the source of
stripe_len in btrfs_map_bio() code.
Reported-by: Chris Murphy <lists@colorremedies.com>
Reported-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Zhao Lei [Wed, 22 Jul 2015 07:58:06 +0000 (15:58 +0800)]
btrfs-progs: Accurate errormsg for resize operation on no-enouth-free-space case
btrfs progs output following error message when doing resize on
no-enouth-free-space case:
# btrfs filesystem resize +10g /mnt/btrfs_5gb
Resize '/mnt/btrfs_5gb' of '+10g'
ERROR: unable to resize '/mnt/btrfs_5gb' - File too large
#
It is not a good description for users, and this patch changed it to:
# ./btrfs filesystem resize +10G /mnt/tmp1
Resize '/mnt/tmp1' of '+10G'
ERROR: unable to resize '/mnt/tmp1' - no enouth free space
#
Reported-by: Taeha Kim <kthguru@gmail.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Patrik Lundquist [Fri, 24 Jul 2015 10:21:02 +0000 (12:21 +0200)]
btrfs-progs: defrag: remove unused variable
A leftover from when recursive defrag was added.
Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Patrik Lundquist [Fri, 24 Jul 2015 08:35:02 +0000 (10:35 +0200)]
btrfs-progs: defrag: fix threshold overflow again
Commit
dedb1ebeee847e3c4d71e14d0c1077887630e44a broke commit
96cfbbf0ea9fce7ecaa9e03964474f407f6e76ab.
Casting thresh value greater than (u32)-1 simply truncates bits while
desired value is (u32)-1 for max defrag threshold.
I.e. "btrfs fi defrag -t 4g" is trimmed/truncated to 0
and "-t 5g" to
1073741824.
Also added a missing newline.
Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 22:18:21 +0000 (00:18 +0200)]
btrfs-progs: mkfs: catch errors after transaction start
Replace missing or BUG_ON in main().
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 22:38:07 +0000 (00:38 +0200)]
btrfs-progs: tests: support testname glob
To run a given test set the variable TEST like
$ make test TEST=002-bad-transid
$ make test TEST=002-*
and only tests matching the value will be run. The pattern is glob and
pased to 'find -name'.
The convert tests do not follow the fsck and misc layout and are skipped
if TEST is set.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 22:31:02 +0000 (00:31 +0200)]
btrfs-progs: fsck tests: move code to a function
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 22:07:05 +0000 (00:07 +0200)]
btrfs-progs: prop: use macro terminator for command list
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 22:05:05 +0000 (00:05 +0200)]
btrfs-progs: unify naming of command handlers
Use cmd_ + group + command schema.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 20 Jul 2015 15:31:43 +0000 (17:31 +0200)]
btrfs-progs: inspect: add command min-dev-size
Previously in 'filesystem resize get_min_size', now
'inspect-internal min-dev-size'. We'd like to avoid cluttering the
'resize' syntax further.
The test has been updated to exercise the new option.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 20 Jul 2015 15:29:24 +0000 (17:29 +0200)]
btrfs-progs: move min-resize implementation to inspect-internal
Signed-off-by: David Sterba <dsterba@suse.com>
Filipe Manana [Thu, 16 Jul 2015 15:47:13 +0000 (16:47 +0100)]
Btrfs-progs: add feature to get mininum size for resizing a fs/device
Currently there is not way for a user to know what is the minimum size a
device of a btrfs filesystem can be resized to. Sometimes the value of
total allocated space (sum of all allocated chunks/device extents), which
can be parsed from 'btrfs filesystem show' and 'btrfs filesystem usage',
works as the minimum size, but sometimes it does not, namely when device
extents have to relocated to holes (unallocated space) within the new
size of the device (the total allocated space sum).
This change adds the ability to reliably compute such minimum value and
extents 'btrfs filesystem resize' with the following syntax to get such
value:
btrfs filesystem resize [devid:]get_min_size
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Jul 2015 16:51:52 +0000 (18:51 +0200)]
btrfs-progs: utils: missing newline in error messages when checking dup
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:28 +0000 (16:15 +0800)]
btrfs-progs: mkfs: Cleanup temporary chunk to avoid strange balance behavior.
[BUG]
# mkfs.btrfs /dev/sdb /dev/sdd -m raid0 -d raid0
# mount /dev/sdb /mnt/btrfs
# btrfs balance start /mnt/btrfs
# btrfs fi df /mnt/btrfs
Data, single: total=1.00GiB, used=320.00KiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, RAID0: total=256.00MiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
Only metadata stay RAID0. Data and system goes from RAID0 to single.
[REASON]
The problem is caused by the temporary single chunk.
In mkfs, it will always create single data/metadata/sys chunk and them
add device into the temporary btrfs.
When doing all chunk balance, for data and syschunk, they are almost
empty, so balance will move them into the single chunk and remove the
old RAID0 chunk.
For metadata, it has more data and will kick the metadata chunk pre
alloc, so new RAID0 chunk is allocated and the old metadata is move
there. Old RAID0 and single chunks are removed.
[FIX]
Now we add a new function to cleanup the temporary chunks at the end of
mkfs routine.
It will cleanup the chunks which is empty and its profile differs from
the mkfs profile.
So in balance, btrfs will always alloc a new chunk to keep the profile,
other than moving data into the single chunk.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Wang Yanfeng [Tue, 7 Jul 2015 02:12:16 +0000 (10:12 +0800)]
Documentation: update btrfs-replace manual to support RAID5/6
Man manual need to be updated since RAID5/6 has been supported
by btrfs-replace.
Signed-off-by: Wang Yanfeng <wangyf-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Jul 2015 11:45:36 +0000 (13:45 +0200)]
Btrfs progs v4.1.2
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 14 Jul 2015 02:13:01 +0000 (10:13 +0800)]
Revert "btrfs-progs: mkfs: create only desired block groups for single device"
This reverts commit
5f8232e5c8f0b0de0ef426274911385b0e877392.
This commit causes a regression:
$ mkfs.btrfs -f /dev/sda6
$ btrfsck /dev/sda6
Checking filesystem on /dev/sda6
UUID:
2ebb483c-1986-4610-802a-
c6f3e6ab4b76
checking extents
Chunk[256, 228, 0]: length(4194304), offset(0), type(2) mismatch with
block group[0, 192, 4194304]: offset(4194304), objectid(0), flags(34)
Chunk[256, 228, 4194304]: length(8388608), offset(4194304), type(4)
mismatch with block group[4194304, 192, 8388608]: offset(8388608),
objectid(4194304), flags(36)
Block group[0, 4194304] (flags = 34) didn't find the relative chunk.
Block group[4194304, 8388608] (flags = 36) didn't find the relative
chunk.
......
The commit has the following bug causing the problem.
1) Typo forgets to add meta/data_profile for alloc_chunk.
Only meta/data_profile is added to allocate a block group, but not
chunk.
2) Type for the first system chunk is impossible to modify yet.
The type for the first chunk and its stripe is hard coded into
make_btrfs() function.
So even we try to modify the type of the block group, we are unable to
change the type of the first chunk.
Causing the chunk type mismatch problem.
The 1st bug can be fixed quite easily but the second is not.
The good news is, the last patch "btrfs-progs: mkfs: Cleanup temporary
chunk to avoid strange balance behavior." from my patchset can handle it
quite well alone.
So just revert the patch.
New bug fix for btrfsck(err is 0 even chunk/extent tree is corrupted) and
new test cases for mkfs will follow soon.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 10 Jul 2015 13:27:49 +0000 (15:27 +0200)]
Btrfs progs v4.1.1
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:27 +0000 (16:15 +0800)]
btrfs-progs: extent-tree: Introduce btrfs_free_block_group function
This function will be used to free a empty chunk.
This provides the basis for later temp chunk cleanup.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:26 +0000 (16:15 +0800)]
btrfs-progs: extent-tree: Introduce functions to free in-memory block group cache
Introduce two functions, free_space_info and free_block_group_cache.
The former will free the space of a empty block group.
The latter will free the in memory block group cache along with its
space in space_info and device space.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:25 +0000 (16:15 +0800)]
btrfs-progs: extent-tree: Introduce functions to free chunk items
Introduce two functions, free_chunk_item and free_system_chunk_item.
First one will free chunk item in chunk tree.
The latter one will free a system chunk in super block.
They are used for later chunk/block group free function.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:24 +0000 (16:15 +0800)]
btrfs-progs: extent-tree: Introduce functions to free dev extents in a chunk
Introduce two functions, free_dev_extent_item and
free_chunk_dev_extent_items, to free dev extent items in a chunk.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:23 +0000 (16:15 +0800)]
btrfs-progs: extent-tree: Introduce free_block_group_item function
This function is used to free a block group item. It must be called
with all the space in the block group pinned. Or there is a possibility
that tree blocks be allocated into the range.
The function is used for later block group/chunk free function.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 7 Jul 2015 08:15:22 +0000 (16:15 +0800)]
btrfs-progs: disk-io: Support commit transaction on chunk tree
As chunk tree is only stored in super block, chunk tree commit doesn't
need to go through tree root update.
Or a BUG_ON will be triggered.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 3 Jul 2015 16:07:17 +0000 (18:07 +0200)]
btrfs-progs: let corrupt-block kill nbytes
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 2 Jul 2015 06:36:04 +0000 (14:36 +0800)]
btrfs-progs: tests: Add test case for I_ERR_FILE_WRONG_NBYTES repair
Add a new test case for I_ERR_FILE_WRONG_NBYTES.
The new btrfs-image dump image contains a file in 12K size.
But nbytes in its inode item is a random number.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Qu Wenruo [Fri, 19 Jun 2015 03:43:40 +0000 (11:43 +0800)]
btrfs-progs: fsck: Add repair function for I_ERR_FILE_WRONG_NBYTES
Some unknown kernel bug makes inode nbytes modification out of sync with
file extent update.
But it's quite easy to fix in btrfs-progs anyway.
So just fix it by adding a new function repair_inode_nbytes by using the
found_size in inode_record.
Reported-by: Christian <cdysthe@gmail.com>
Reported-by: Chris Murphy <lists@colorremedies.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 2 Jul 2015 22:15:18 +0000 (00:15 +0200)]
btrfs-progs: mkfs: create only desired block groups for single device
The filesystem creation has to solve some chicken-egg problems and
creates some temporary objects. In our case it's an extra single/single
pair of block groups that's not used unless the user asks that
explicitly.
Example:
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=153.56MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
Even with a single device filesystem and defaults, there's single
block group for metadata and system. The single device case is easy to
fix, we'll simply create the right type from the beginning.
Example:
Data, single: total=8.00MiB, used=64.00KiB
System, DUP: total=4.00MiB, used=16.00KiB
Metadata, DUP: total=136.00MiB, used=112.00KiB
GlobalReserve, single: total=16.00MiB, used=0.00B
Filesystem on top of multiple devices still leaves the single/single
groups behind.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 2 Jul 2015 17:23:27 +0000 (19:23 +0200)]
btrfs-progs: drop unused argument from create_raid_groups
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 17:19:05 +0000 (19:19 +0200)]
btrfs-progs: split data block group creation out of make_root_dir
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 17:15:35 +0000 (19:15 +0200)]
btrfs-progs: move transaction start/commit out of make_root_dir
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 17:12:38 +0000 (19:12 +0200)]
btrfs-progs: split metadata group creation out of make_root_dir
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 15:54:43 +0000 (17:54 +0200)]
btrfs-progs: drop unused parameter from make_btrfs
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 15:49:21 +0000 (17:49 +0200)]
btrfs-progs: move make_btrfs arguments to a struct
No functional change, just introduce the structure and switch current
users.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 13:39:59 +0000 (15:39 +0200)]
btrfs-progs: add 'device remove' alias to completion
Signed-off-by: David Sterba <dsterba@suse.com>
Tsutomu Itoh [Tue, 30 Jun 2015 23:56:52 +0000 (08:56 +0900)]
btrfs-progs: doc: fix short explanation of restore in btrfs
Short explanation of restore is wrong. Fix it.
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Wed, 1 Jul 2015 13:20:23 +0000 (15:20 +0200)]
btrfs-progs: check for item end outside of leaf
Enhance leaf check to verify item ends that looks otherwise fine but
would exceed leaf. Same check is done in kernel.
Reported-by: Robert Marklund <robbelibobban@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 30 Jun 2015 13:05:07 +0000 (15:05 +0200)]
btrfs-progs: drop argument from attribute deprecated
The optional argument to attribute 'deprecated' has been introduced in
gcc 4.5, and does not build on 4.4 which is still in use. The
recommended replacements are mentioned in the comment, not absolutely
necessary to repeat it via the attribute.
Reported-by: Amr El-Sharnoby <amr.elsharnoby@horizontechs.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Fri, 19 Jun 2015 03:49:19 +0000 (11:49 +0800)]
btrfs-progs: Add nbytes output for print-tree and reformat inode output
The original implementation doesn't output the nbytes for an inode.
Add the output and since the output is too long, reformat it to multi
lines.
This is very handy to debug related bugs.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 26 Jun 2015 14:44:29 +0000 (16:44 +0200)]
btrfs-progs: no extra newline between aliased commands in help text
Signed-off-by: David Sterba <dsterba@suse.cz>
Omar Sandoval [Wed, 24 Jun 2015 16:09:17 +0000 (09:09 -0700)]
btrfs-progs: alias btrfs device delete to btrfs device remove
There's an awkward asymmetry between btrfs device add and btrfs device
delete. Resolve this by aliasing delete to remove.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Omar Sandoval [Wed, 24 Jun 2015 16:09:16 +0000 (09:09 -0700)]
btrfs-progs: replace struct cmd_group->hidden with flags
We're also going to want to support aliases, so rather than adding
another member, replace "hidden" with a "flags" member.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Patrik Lundquist [Fri, 26 Jun 2015 08:43:56 +0000 (10:43 +0200)]
btrfs-progs: inspect: Fix out of bounds string termination.
Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 26 Jun 2015 14:23:00 +0000 (16:23 +0200)]
btrfs-progs: doc: update defrag page
- update wording for -t
- add optional argument to -c
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Thu, 25 Jun 2015 17:36:06 +0000 (19:36 +0200)]
btrfs-progs: defrag, check target extent earlier
Print a warning if the target extent size (option -t) is larger than 4G.
Signed-off-by: David Sterba <dsterba@suse.cz>
Patrik Lundquist [Wed, 24 Jun 2015 14:21:06 +0000 (16:21 +0200)]
btrfs-progs: fix defrag threshold overflow
btrfs fi defrag -t 1T overflows the u32 thresh variable and default,
instead of max, threshold is used.
Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Adam Borowski [Tue, 23 Jun 2015 22:46:13 +0000 (00:46 +0200)]
btrfs-progs: doc: mkfs.btrfs: document -O^
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 22 Jun 2015 14:48:41 +0000 (16:48 +0200)]
Btrfs progs v4.1
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Mon, 22 Jun 2015 14:13:14 +0000 (16:13 +0200)]
btrfs-progs: docs: new size options for fi show
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Thu, 18 Jun 2015 06:46:11 +0000 (14:46 +0800)]
btrfs-progs: Allow "filesystem show" command to handle different units
Now "filesystem show" command can handle different units now.
This is handy for higher level programs to get accurate output from "fi
show" command.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Sun, 21 Jun 2015 23:39:27 +0000 (01:39 +0200)]
btrfs-progs: INSTALL: fix typos
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Sun, 21 Jun 2015 23:13:48 +0000 (01:13 +0200)]
btrfs-progs: configure: fix typo in summary
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Thu, 18 Jun 2015 17:47:28 +0000 (19:47 +0200)]
btrfs-progs: btrfstune: print correct current uuid during rewrite
uuid_unparse is supposed to take the ASCII representation of a UUID, so
we have to pass the raw fsid buffer.
Reported-by: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Thu, 18 Jun 2015 17:40:01 +0000 (19:40 +0200)]
btrfs-progs: btrfstune: print lowercase uuid during uuid rewrite
We're using lowercase everywhere else.
Reported-by: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Thu, 18 Jun 2015 17:35:58 +0000 (19:35 +0200)]
btrfs-progs: tests: verify btrfstune output during uuid-rewrite
The 'Current fsid:' value does not match the real fsid.
Reported-by: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Wed, 17 Jun 2015 07:49:04 +0000 (15:49 +0800)]
btrfs-progs: map-logical: Rework map-logical logics
[BUG]
The original map-logical has the following problems:
1) Assert if we pass any tree root bytenr.
The problem is easy to trigger, here the number
29622272 is the bytenr of tree root:
# btrfs-map-logical -l
29622272 /dev/sda6
mirror 1 logical
29622272 physical
38010880 device /dev/sda6
mirror 2 logical
29622272 physical
1111752704 device /dev/sda6
extent_io.c:582: free_extent_buffer: Assertion `eb->refs < 0` failed.
btrfs-map-logical[0x41c464]
btrfs-map-logical(free_extent_buffer+0xc0)[0x41cf10]
btrfs-map-logical(btrfs_release_all_roots+0x59)[0x40e649]
btrfs-map-logical(close_ctree+0x1aa)[0x40f51a]
btrfs-map-logical(main+0x387)[0x4077c7]
/usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f80a5562790]
btrfs-map-logical(_start+0x29)[0x4078f9]
The problem is that, btrfs-map-logical always use sectorsize as default
block size to call alloc_extent_buffer.
And when it failes to find the block with the same size, it will free
the extent buffer in a incorrect method(Free and create a new one with
refs == 1).
2) Will return map result for non-exist extent.
# btrfs-map-logical -l 1 -b 123456 /dev/sda6
mirror 1 logical 1 physical 1 device /dev/sda6
mirror 1 logical 4097 physical 4097 device /dev/sda6
mirror 1 logical 8193 physical 8193 device /dev/sda6
...
Normally, before bytenr
12582912, there should be no extent as that's
the mkfs time temp metadata/system chunk.
But map-logical will still map them out.
Not to mention the 1 offset among all results.
[FIX]
This patch will rework the whole map logical by the following methods:
1) Always do things inside a extent
Even under the following case, map logical will only return covered
range in existing extents.
|<------ range given ------->|
|<-Extent A->| |<-Extent B->| |<---Extent C->|
Result:
|<-->| |<---------->| |<-->|
So with this patch, we will search extent tree to ensure all operation
are inside a extent before we do some stupid things.
2) No direct call on alloc_extent_buffer function.
That low-level function shouldn't be called at such high level.
It's only designed for low-level tree operation.
So in this patch we will only use safe high level functions avoid such
problem.
[RESULT]
With this patch, no assert will be triggered and better handle on
non-exist extents.
# btrfs-map-logical -l
29622272 /dev/sda6
mirror 1 logical
29622272 physical
38010880 device /dev/sda6
mirror 2 logical
29622272 physical
1111752704 device /dev/sda6
# btrfs-map-logical -l 1 -b 123456 /dev/sda6
No extent found at range [1,123457)
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Wed, 17 Jun 2015 07:49:03 +0000 (15:49 +0800)]
Btrfs-progs: map-logical: introduce write_extent_content function
This function will write extent content info desired file.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Wed, 17 Jun 2015 07:49:02 +0000 (15:49 +0800)]
Btrfs-progs: map-logical: introduce print_mapping_info function
The new function will print the mapping info of given range
[logical, logical+len).
Note, caller must ensure the ranges are completely inside an extent.
Or btrfs_map_block can return -ENOENT.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Wed, 17 Jun 2015 07:49:01 +0000 (15:49 +0800)]
btrfs-progs: map-logical: introduce map_one_extent function
Introduce the function to get accurate extent length based on extent
tree search.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Qu Wenruo [Wed, 17 Jun 2015 07:49:00 +0000 (15:49 +0800)]
btrfs-progs: export read_extent_data function
Export it for later btrfs-map-logical cleanup.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 17 Jun 2015 10:44:16 +0000 (12:44 +0200)]
Btrfs progs v4.1-rc1
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 15:01:06 +0000 (17:01 +0200)]
btrfs-progs: Makefile: fix typo, dependencies for cmds-* files
The dependency generation skipped all cmds-*.c files due to a typo
in variable name, since day one.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 14:37:18 +0000 (16:37 +0200)]
btrfs-progs: send utils: deprecate path_cat and path_cat3
The functios do no allocation error handling, use _out variants instead.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 13:56:37 +0000 (15:56 +0200)]
btrfs-progs: receive: use static buffer for cur_subvol path
The embedded 'path' is a pointera and we can't make it a path buffer due
to API constraints. Use a separate buffer and sto using the unsafe
path_cat interface.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 13:36:42 +0000 (15:36 +0200)]
btrfs-progs: send utils: switch callbacks to patch_cat3_out
Also switch full_subvol_path to a static buffer as the interface
semantics change.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 13:28:44 +0000 (15:28 +0200)]
btrfs-progs: receive: use static buffer for write_path
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 16 Jun 2015 13:15:51 +0000 (15:15 +0200)]
btrfs-progs: receive: use static buffer for root_subvol_path
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 15 Jun 2015 12:40:42 +0000 (14:40 +0200)]
btrfs-progs: unify naming of key types in print-tree
Drop the BTRFS_ prefix and _KEY suffix where applicable.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 16:18:40 +0000 (18:18 +0200)]
btrfs-progs: send utils: switch callbacks to new helpers
Use a local buffer for full path(s) and avoid dynamic allocations that
happend to be unchecked.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 15:06:48 +0000 (17:06 +0200)]
btrfs-progs: and new path_cat helpers to send utils
Add versions of path_cat and path_cat3 that do not allocate the memory.
The unhandled memory allocations are still there.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 14:42:41 +0000 (16:42 +0200)]
btrfs-progs: send: use static buffer for output file name
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 14:37:31 +0000 (16:37 +0200)]
btrfs-progs: receive: use static buffer for source file
We should copy the optarg anyway.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 13:58:55 +0000 (15:58 +0200)]
btrfs-progs: receive: use static buffer for mount point option argument
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 14:09:33 +0000 (16:09 +0200)]
btrfs-progs: add helper for copying paths
Check the source path length and do the copy.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 13:40:07 +0000 (15:40 +0200)]
btrfs-progs: receive: use static buffer for cur_subvol
Get rid of the allocation. The logic is changed so that a NULL
cur_subvol::path means that no subvolume/snapshot has been found so far.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 12:36:51 +0000 (14:36 +0200)]
btrfs-progs: properly set up ioctl arguments
At some places we do not clear the whole ioctl structure and could
pass garbage to kernel. Zero the ioctl vol_args and use a helper for
copying the path.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 11:18:44 +0000 (13:18 +0200)]
btrfs-progs: use PATH_MAX instead of BTRFS_PATH_NAME_MAX
The path bufferes should be PATH_MAX but BTRFS_PATH_NAME_MAX is shorter
due to embedding in 4k aligned structures.
The only reason to use BTRFS_PATH_NAME_MAX is for the respective
structures btrfs_ioctl_vol_args::name.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 12 Jun 2015 11:15:41 +0000 (13:15 +0200)]
btrfs-progs: send: add option to for the no-data mode
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Thu, 11 Jun 2015 17:31:01 +0000 (19:31 +0200)]
btrfs-progs: receive: implement the update_extent callback
The callback for the NO_FILE_DATA send flag is missing and would crash
if such stream is received.
Signed-off-by: David Sterba <dsterba@suse.cz>
Josef Bacik [Wed, 10 Jun 2015 19:05:51 +0000 (15:05 -0400)]
btrfs-progs: make receive work inside of subvolumes
Kind of a big feature of btrfs is being able to have a default subvol. However
the receive code generates the paths to the subvols from the root of the fs,
even in the case of a default subvol. So instead figure out if we're inside of
a subvol, either because we have a different default or we've chroot'ed and are
using -m. Then strip this extra path off of the subvol we find so we can look
up our parent properly. Thanks
Reported-by: Neil Horman <nhorman@redhat.com>
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Fri, 27 Feb 2015 18:37:24 +0000 (19:37 +0100)]
btrfs-progs: use less memory for pretty_size_mode buffers
Anand reports that the static buffers used for pertty size strings cause
a stack overflow on SPARC. Zach proposed to change the printf format to
wrap the number and the suffix into a macro. This would require to
change all callsites of pretty_size* and is not very convienient to
write.
This patch replaces the per-call-site static buffers with a limited
number for slots that would be used on each invokation of pretty_size
and wrap around. The number of array slots shall be 10 for now, in
current codebase there are no more than 2 calls to pretty_size in a
single argument list.
Reported-by: Anand Jain <Anand.Jain@oracle.com>
CC: Zach Brown <zab@zabbo.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 10 Jun 2015 22:51:15 +0000 (00:51 +0200)]
btrfs-progs: print error within test_num_disk_vs_raid
The error string buffer passed as an argument is of a fixed size, though
we could print up to PATH_MAX + something bytes. Print the error message
directly.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 10 Jun 2015 22:46:30 +0000 (00:46 +0200)]
btrfs-progs: print error within test_dev_for_mkfs
The error string buffer passed as an argument is of a fixed size, though
we could print up to PATH_MAX + something bytes. Print the error message
directly.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 10 Jun 2015 22:04:19 +0000 (00:04 +0200)]
btrfs-progs: accept --help as option in the standalone utilities
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 10 Jun 2015 00:21:42 +0000 (02:21 +0200)]
btrfs-progs: add missing includes to header files
Add includes that let the header files compile or add explicit include
of kerncompat if the uXX types are used.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Wed, 8 Apr 2015 16:15:57 +0000 (18:15 +0200)]
btrfs-progs: add helper to wait for subvolume cleaning
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 7 Apr 2015 22:28:48 +0000 (00:28 +0200)]
btrfs-progs: subvol: let sync check only current deletions
So far the subvol sync command takes a shortcut and looks if there are
any deleted subvols at all. It does not print the deleted subvolumes as
they get cleaned. Arguably this is what the user would like to see and
has to do
$ btrfs subvol sync /path $(btrfs subvol list -d /path | "extract the ids")
to see the progress.
Make it look for all currently deleted subvolumes automatically and
print the progress as if the ids were listed manually.
This is a slight change in the semantics of the command. Previously, any
new subvol deletion would prevent subvol sync to return. To simulate the
old behaviour, run 'subvol sync' in a loop until it returns 0.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 9 Jun 2015 13:24:51 +0000 (15:24 +0200)]
btrfs-progs: cleanup after errors in open_file_or_dir3
We should zero the dirstream in case we close it ourselves, so the
caller does not do it again. Most current callers do not do that and
exit immediatelly.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 9 Jun 2015 12:58:43 +0000 (14:58 +0200)]
btrfs-progs: receive: fix minor resource leak
Resolves-coverity-id: 1302985
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 8 Jun 2015 16:54:04 +0000 (18:54 +0200)]
btrfs-progs: print compact help for btrfs
Running 'btrfs' without arguments will print complete help that spans
a lot of lines and is really helpful. Print only subcommand group
names with short descriptions, similar to what 'git' does.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Tue, 9 Jun 2015 12:22:05 +0000 (14:22 +0200)]
btrfs-progs: add command group info strings
They're printed in the 'btrfs' command group summary.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 8 Jun 2015 21:30:21 +0000 (23:30 +0200)]
btrfs-progs: fix argv0_buf handling
The variable argv0_buf was duplicated and the changes done in utils.c
were not propagated to help.c. So if an unknown commandline token was
found, the error message did not contain the known part:
$ btrfs scrub test
: unknown token 'test'
instead of
$ btrfs scrub test
btrfs scrub: uknown token 'test'
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 8 Jun 2015 16:36:58 +0000 (18:36 +0200)]
btrfs-progs: remove stray deprecation notice
The 1st level balance subcommand is the right one but it accidentally
inherited the notice about 'filesystem balance' deprecation.
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 8 Jun 2015 14:26:54 +0000 (16:26 +0200)]
btrfs-progs: mkfs: make the summary more compact
Single device example:
btrfs-progs v4.0.1-39
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID:
a88bfc85-b454-4a32-8de7-
276c01f04d58
Node size: 16384
Sector size: 4096
Filesystem size: 2.00GiB
Block group profiles:
Data: single 8.00MiB
Metadata: single 8.00MiB
System: single 4.00MiB
SSD detected: no
Incompat features: extref, skinny-metadata
Number of devices: 1
Devices:
ID SIZE PATH
1 2.00GiB /dev/sda
Multiple devices:
btrfs-progs v4.0.1-39
See http://btrfs.wiki.kernel.org for more information.
Label: (null)
UUID:
9db282ef-55f3-4070-a59f-
012a654614b2
Node size: 16384
Sector size: 4096
Filesystem size: 8.00GiB
Block group profiles:
Data: RAID6 417.50MiB
Metadata: RAID6 417.50MiB
System: RAID6 20.00MiB
SSD detected: no
Incompat features: extref, raid56, skinny-metadata
Number of devices: 4
Devices:
ID SIZE PATH
1 2.00GiB /dev/sda
2 2.00GiB /dev/sdb
3 2.00GiB /dev/sdc
4 2.00GiB /dev/sdd
Signed-off-by: David Sterba <dsterba@suse.cz>
David Sterba [Mon, 8 Jun 2015 13:06:17 +0000 (15:06 +0200)]
btrfs-progs: mkfs: drop mixed from summary
The mixed-bg incompat feature should be enough.
Signed-off-by: David Sterba <dsterba@suse.cz>