platform/upstream/btrfs-progs.git
9 years agobtrfs-progs: move test_isdir() to utils.c
Satoru Takeuchi [Fri, 1 Aug 2014 02:58:00 +0000 (11:58 +0900)]
btrfs-progs: move test_isdir() to utils.c

Since test_isdir() is a utility function, it's better to
move it to utils.c. In addition, "const char *" is
more appropriate type as its "path" argument because
this argument is not changed in this function.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Cc: David Sterba <dsterba@suse.cz>
Cc: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: introduce test_issubvolname() for simplicity
Satoru Takeuchi [Fri, 1 Aug 2014 02:44:21 +0000 (11:44 +0900)]
btrfs-progs: introduce test_issubvolname() for simplicity

There are many duplicated codes to check if the given string is
correct subvolume name. Introduce test_issubvolname() for this
purpose for simplicity.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Cc: David Sterba <dsterba@suse.cz>
Cc: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add mount status check for btrfs-image
Gui Hecheng [Mon, 7 Jul 2014 01:56:51 +0000 (09:56 +0800)]
btrfs-progs: add mount status check for btrfs-image

When btrfs-image run on a mounted filesystem,
the undergoing fs operations may change what you have imaged a while ago.
In this case, give a warning to remind the user that he may not
get a consistent image he wants.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
[tweaked the message]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: remove unnecessary NULL check after get_df()
Satoru Takeuchi [Wed, 30 Jul 2014 07:26:32 +0000 (16:26 +0900)]
btrfs-progs: remove unnecessary NULL check after get_df()

If get_df() returns 0, "sargs" surely points to malloc'ed region.
So NULL check of sargs is not necessary.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix improper return value check for is_existing_blk_or_reg_file
Gui Hecheng [Thu, 31 Jul 2014 03:23:44 +0000 (11:23 +0800)]
btrfs-progs: fix improper return value check for is_existing_blk_or_reg_file

The function @is_existing_blk_or_reg_file has a return value of -errno,
which indicate the @stat call fails with non-ENOENT errors.
In this condition, we should not continue the following work.

But -errno evaluates to true and will let the following work go.
So we should judge more accurately whether the return value of
@is_existing_blk_or_reg_file is > 0 or not to decide our behavior.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: remove unnecessary return value check in btrfs-property
Gui Hecheng [Thu, 31 Jul 2014 03:23:43 +0000 (11:23 +0800)]
btrfs-progs: remove unnecessary return value check in btrfs-property

The function @parse_prop() returns either -1 or 0, no need to check
for other values. Just return the unnecessary check.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: init variables which are checked later in btrfs-property
Gui Hecheng [Thu, 31 Jul 2014 03:23:42 +0000 (11:23 +0800)]
btrfs-progs: init variables which are checked later in btrfs-property

The local variable @object etc. in btrfs-property get/set functions
are to be checked whether to be NULL or not, but the @parse_args()
don't guarantee to assign a value to it, so it is better to init
it to NULL.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: mkfs: remove experimental tag
David Sterba [Thu, 31 Jul 2014 12:16:50 +0000 (14:16 +0200)]
btrfs-progs: mkfs: remove experimental tag

Make it consistent with kernel status and documentation.

Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add always option to restore's looping prompt
Justin Maggard [Tue, 29 Jul 2014 01:55:09 +0000 (18:55 -0700)]
btrfs-progs: add always option to restore's looping prompt

If you are using btrfs restore to try to recover a very large or
fragmented file, you may encounter _lots_ of prompts requiring
you to press 'y' to continue because we are looping a lot.

Add the option to press 'a', to supress these prompts for the rest
of the file.

Signed-off-by: Justin Maggard <jmaggard10@gmail.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fix some build warnings on 32bit platform
Wang Shilong [Sat, 26 Jul 2014 16:49:55 +0000 (00:49 +0800)]
Btrfs-progs: fix some build warnings on 32bit platform

Fix following build warnings on 32bit platform:

...
utils.c:1708:3: warning: left shift count >= width of
type [enabled by default]
   if (x << i & (1UL << 63))
   ^
qgroup-verify.c:393:9: warning: cast to pointer from integer
of different size [-Wint-to-pointer-cast]
  return (struct tree_block *)unode->aux;
         ^
qgroup-verify.c:407:38: warning: cast from pointer to integer
of different size [-Wpointer-to-int-cast]
   if (ulist_add(tree_blocks, bytenr, (unsigned long long)block, 0) >= 0)
                                      ^
cmds-restore.c:120:4: warning: format %lu expects argument of type
long unsigned int, but argument 3 has type size_t [-Wformat=]
    fprintf(stderr, "bad compress length %lu\n", in_len);
...

BTW, this patch also switches other castings with new helpers.

Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Unify the messy error message formats
Satoru Takeuchi [Tue, 29 Jul 2014 02:18:17 +0000 (11:18 +0900)]
btrfs-progs: Unify the messy error message formats

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: remove author and copyright info from man page
Wang Shilong [Thu, 24 Jul 2014 09:30:15 +0000 (17:30 +0800)]
Btrfs-progs: remove author and copyright info from man page

>From:
http://man7.org/linux/man-pages/man7/man-pages.7.html
...
AUTHORS lists authors of the documentation or program.Use of
an AUTHORS section is strongly discouraged. Generally,
it is better not to clutter every page with a list of
(over time potentially numerous) authors; if you write
or significantly amend a page, add a copyright notice
as a comment in the source file.  If you are the author
of a device driver and want to include an address for
reporting bugs, place this under the BUGS section.
...

Suggested-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Check fstype in find_mount_root()
Qu Wenruo [Wed, 23 Jul 2014 05:47:35 +0000 (13:47 +0800)]
btrfs-progs: Check fstype in find_mount_root()

When calling find_mount_root(), caller in fact wants to find the mount
point of *BTRFS*.

So also check ent->fstype in find_mount_root() and do special error
string output in caller.

This will suppress a lot of "Inapproiate ioctl for device" error
message.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Remove fprintf() in find_mount_root().
Qu Wenruo [Wed, 23 Jul 2014 05:47:34 +0000 (13:47 +0800)]
btrfs-progs: Remove fprintf() in find_mount_root().

find_mount_root() function in utils.c should not print error string.
Caller should be responsible to print error string.

This patch will remove the only fprintf in find_mount_root() and modify
the caller a little to use strerror() to prompt users.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fix wrong manpage of defrag command
Liu Bo [Thu, 17 Jul 2014 10:46:01 +0000 (18:46 +0800)]
Btrfs-progs: fix wrong manpage of defrag command

'btrfs filesystem defrag' has an option '-t', whose manpage says

"Any extent bigger than threshold given by -t option, will be
considered already defragged. Use 0 to take the kernel default, and
use 1 to say every single extent must be rewritten."

Here 'use 0' still works, it refers to the default value(256K), however,
'use 1' is an obvious typo, it should be -1, which means the largest value
it can be.

Right now, we use parse_size() which no more allow value '-1', so in
order to keep the manpage correct, this updates it to only keep value '0'.

If you want to make sure every single extent is rewritten, please use a fairly
large size, say 1G.

Reported-by: Sebastian Ochmann <ochmann@informatik.uni-bonn.de>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: use check_argc_* to check arg number for all tools
Gui Hecheng [Wed, 16 Jul 2014 03:59:46 +0000 (11:59 +0800)]
btrfs-progs: use check_argc_* to check arg number for all tools

Since this patch:
btrfs-progs: move the check_argc_* functions into utils.c

All tools including the independent tools(e.g. btrfs-image, btrfs-convert)
can share the convenience of the check_argc_* functions, so this patch
adopt the argc check functions globally.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add uninstall targets to Makefiles.
Nils Steinger [Fri, 25 Jul 2014 13:41:57 +0000 (15:41 +0200)]
btrfs-progs: Add uninstall targets to Makefiles.

Signed-off-by: Nils Steinger <git@n-st.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: replace a confusing raw number with a macro
Gui Hecheng [Thu, 17 Jul 2014 02:40:38 +0000 (10:40 +0800)]
btrfs-progs: replace a confusing raw number with a macro

The raw number 36 for the uuid string length is somewhat confusing,
use a macro to define replace it.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
[Use BTRFS_UUID_UNPARSED_SIZE]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix wrong message about forcing the mixed group profile
Anand Jain [Tue, 15 Jul 2014 08:26:20 +0000 (16:26 +0800)]
btrfs-progs: fix wrong message about forcing the mixed group profile

When the added disk is small, the function btrfs_prepare_device() would
print message indicating that mixed group is forced, apparently thats
not true

reproducer:

mkfs.btrfs -f -draid1 -mraid1 /dev/sdd /dev/sdc
::
SMALL VOLUME: forcing mixed metadata/data groups <---- WRONG

mount /dev/sdd /btrfs

btrfs fi df /btrfs
Data, RAID1: total=126.25MiB, used=128.00KiB
Data, single: total=8.00MiB, used=0.00B
System, RAID1: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, RAID1: total=126.25MiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix build of static target
David Sterba [Wed, 23 Jul 2014 13:13:46 +0000 (15:13 +0200)]
btrfs-progs: fix build of static target

A user repoted that static buid fails with

utils-lib.static.o: In function `arg_strtou64':
/home/dsterba/labs/btrfs-progs/utils-lib.c:17: multiple definition of `arg_strtou64'
utils-lib.static.o:/home/dsterba/labs/btrfs-progs/utils-lib.c:17: first defined here

utils-lib.o was mistakenly added to linker twice.

Signed-off-by: David Sterba <dsterba@suse.cz>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
9 years agobtrfs-progs: correct manpage option description for scrub
Gui Hecheng [Wed, 30 Jul 2014 07:32:05 +0000 (15:32 +0800)]
btrfs-progs: correct manpage option description for scrub

The -f option of scrub means to
    "force starting new scrub even if a scrub is already running"
*not*
    "force to check whether scrub has started or resumed in userspace"
as described originally.

So replace the orignal description in the manpage and code.

Also, add description of the potential failure as follows
    "If a scrub is already running, it fails."

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Cc: David Sterba <dsterba@suse.cz>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: remove unnecessary judgment for fd in scrub
Gui Hecheng [Thu, 17 Jul 2014 02:40:37 +0000 (10:40 +0800)]
btrfs-progs: remove unnecessary judgment for fd in scrub

The scrub_read_file function is always on a branch,
which has (fd >= 0), so there is not need to judgment
the pasted in arg.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: adjust the return values for scrub
Gui Hecheng [Thu, 17 Jul 2014 02:40:36 +0000 (10:40 +0800)]
btrfs-progs: adjust the return values for scrub

o Return 0 to indicate success,
  when detected errors were corrected during scrubbing.
  P.s. This is also to facilitate scripting when return value
       is to be checked.
o Warn the users if there are uncorrectable errors detected.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: mkfs: don't zero extend small files
Zach Brown [Wed, 16 Jul 2014 21:02:37 +0000 (14:02 -0700)]
btrfs-progs: mkfs: don't zero extend small files

mkfs can try to write outside of small devices.  The zeroing code
doesn't test the device size and runs before mkfs tests for small
devices and exits.

Testers experienced this as small regular files being extended as mkfs
failed:

 $ truncate -s 1m /tmp/some-file
 $ strace -epwrite ./mkfs.btrfs /tmp/some-file
 SMALL VOLUME: forcing mixed metadata/data groups

 WARNING! - Btrfs v3.14.2 IS EXPERIMENTAL
 WARNING! - see http://btrfs.wiki.kernel.org before using

 pwrite(3, ..., 2097152, 0) = 2097152
 pwrite(3, ..., 4096, 65536) = 4096
 pwrite(3 ..., 2097152, 18446744073708503040) = -1 EINVAL (Invalid argument)
 ERROR: failed to zero device '/tmp/some-file' - Input/output error

 $ ls -lh /tmp/some-file
 -rw-rw-r--. 1 zab zab 2.0M Jul 16 13:49 /tmp/some-file

This simple fix adds a helper that clamps a region to be zeroed to the
size of the device.  It doesn't address the larger questions of whether
to modify the device before the size test or whether or zero regions
that have been trimmed.

Finally, the error handling mess after the zeroing calls is cleaned up.
zero_blocks() and its callers only return -errno.

Signed-off-by: Zach Brown <zab@zabbo.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: define BTRFS_MKFS_SMALL_VOLUME_SIZE for small volume
Anand Jain [Tue, 15 Jul 2014 08:02:10 +0000 (16:02 +0800)]
btrfs-progs: define BTRFS_MKFS_SMALL_VOLUME_SIZE for small volume

mkfs cut of size '1024 * 1024 * 1024' to mark dev as small volume so to
force mixed group. Use a define for that.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: dont break the string
Anand Jain [Tue, 15 Jul 2014 08:02:09 +0000 (16:02 +0800)]
btrfs-progs: dont break the string

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: doc: fix incorrect subvol name
Tsutomu Itoh [Mon, 14 Jul 2014 02:27:43 +0000 (11:27 +0900)]
btrfs-progs: doc: fix incorrect subvol name

Subvolume name is wrong. Fix it.

Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: ignore orphaned qgroups by default
Mark Fasheh [Tue, 8 Jul 2014 20:41:57 +0000 (13:41 -0700)]
btrfs-progs: ignore orphaned qgroups by default

qgroup items are not deleted by btrfs when the underlying subvolume goes
away. As a result, btrfsck will print those as inconsistent. This can
clutter up the printout so we ignore them by default. They are still printed
if a full report (via --qgroup-report) is requested.

This patch and the ones it depends on (to do qgroup verification) can be
found at:

https://github.com/markfasheh/btrfs-progs-patches/tree/qgroup-verify

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fix Segmentation fault of btrfs-convert
Liu Bo [Mon, 7 Jul 2014 14:35:53 +0000 (22:35 +0800)]
Btrfs-progs: fix Segmentation fault of btrfs-convert

Recently we merge a memory leak fix, which fails xfstests/btrfs/012,
the cause is that it only frees @fs_devices but leaves it on the global
fs_uuid list, which cause a 'Segmentation fault' over running command
btrfs-convert.  This fixes the problem.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Allow btrfs_read_dev_super() to read all 3 super for super_recover.
Qu Wenruo [Thu, 3 Jul 2014 09:36:36 +0000 (17:36 +0800)]
btrfs-progs: Allow btrfs_read_dev_super() to read all 3 super for super_recover.

Btrfs-progs superblock checksum check is somewhat too restricted for
super-recover, since current btrfs-progs will only read the 1st
superblock and if you need super-recover the 1st superblock is
possibly already damaged.

The fix is introducing super_recover parameter for
btrfs_read_dev_super() and callers to allow scan backup superblocks if
needed.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: show extent state for a subvolume
Mark Fasheh [Wed, 2 Jul 2014 20:34:41 +0000 (13:34 -0700)]
btrfs-progs: show extent state for a subvolume

The qgroup verification code can trivially be extended to provide
extended information on the extents which a subvolume root
references. Along with qgroup-verify, I have found this tool to be
invaluable when tracking down extent references.

The patch adds a switch to the check subcommand '--subvol-extents'
which takes as args a single subvolume id. When run with the switch,
we'll print out each extent that the subvolume references. The extent
printout gives standard extent info you would expect along with
information on which other roots reference it.

Sample output follows - this is a few lines from a run on a subvolume
I've been testing qgroup changes on:

Print extent state for subvolume 281 on /dev/vdb2
UUID: 8203ca66-9858-4e3f-b447-5bbaacf79c02
Offset Len Root Refs Roots
12582912 20480 12 257 279 280 281 282 283 284 285 286 287 288 289
12603392 8192 12 257 279 280 281 282 283 284 285 286 287 288 289
12611584 12288 12 257 279 280 281 282 283 284 285 286 287 288 289
 <snip a bunch of extents to show some variety>
124583936 16384 4 281 282 283 280
125075456 16384 4 280 281 282 283
126255104 16384 11 257 280 281 282 283 284 285 286 287 288 289
4763508736 4096 3 279 280 281

In case it wasn't clear, this applies on top of my qgroup verify patch:
"btrfs-progs: add quota group verify code"

A branch with all this can be found on github:

https://github.com/markfasheh/btrfs-progs-patches/tree/qgroup-verify

Please apply,

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: limit minimal num of args for btrfs-image
Gui Hecheng [Mon, 30 Jun 2014 03:54:12 +0000 (11:54 +0800)]
btrfs-progs: limit minimal num of args for btrfs-image

The btrfs-image requires at least 2 args to run,
one for the source dev/file, the other for the target dev/file.

This patch depends on patch:
btrfs-progs: move the check_argc_* functions into utils.c

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: move the check_argc_* functions into utils.c
Gui Hecheng [Mon, 30 Jun 2014 03:54:11 +0000 (11:54 +0800)]
btrfs-progs: move the check_argc_* functions into utils.c

To let the independent tools(e.g. btrfs-image, btrfs-convert, etc.)
share the convenience of check_argc_* functions, just move it into
utils.c.
Also add a new function "set_argv0" to set the correct tool name:
*btrfs-image*: too few arguments

The original btrfs* tools work as before.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
[moved argv0 and check_argc to utils.*]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: balance filter: add limit of processed chunks
David Sterba [Wed, 7 May 2014 15:38:00 +0000 (17:38 +0200)]
btrfs-progs: balance filter: add limit of processed chunks

Add more control to the balance behaviour.

Usage filter may not be finegrained enough and can lead to moving too
many chunks at once. Another example use is in connection with
drange+devid or vrange filters that allow to work with a specific chunk
or even with a chunk on a given device.

The limit filter applies last, the value of 0 means no limiting.

CC: Ilya Dryomov <idryomov@gmail.com>
CC: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add X inode flag to btrfs(5)
David Sterba [Wed, 2 Jul 2014 11:25:11 +0000 (13:25 +0200)]
btrfs-progs: add X inode flag to btrfs(5)

After the discussion in
http://thread.gmane.org/gmane.comp.file-systems.btrfs/36334

the 'X' will be mentioned in the manpage because new e2fsprogs/lsattr
will display it and represents the NOCOMPRESS bit of an inode.

Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: cleanup unnecessary free if malloc fails in btrfs-image
Gui Hecheng [Thu, 26 Jun 2014 02:53:03 +0000 (10:53 +0800)]
btrfs-progs: cleanup unnecessary free if malloc fails in btrfs-image

Don't bother free the buffer if the malloc failed.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fsck: reduce memory usage for extents check
Wang Shilong [Fri, 20 Jun 2014 11:09:07 +0000 (19:09 +0800)]
Btrfs-progs: fsck: reduce memory usage for extents check

Steps to reproduce:

 # mkfs.btrfs -f /dev/sda9 -b 2g
 # mount /dev/sda9 /mnt
 # dd if=/dev/zero of=/mnt/data bs=4k oflag=direct
 # btrfs file df /mnt

    Data, single: total=1.66GiB, used=1.66GiB
    System, single: total=4.00MiB, used=16.00KiB
    Metadata, single: total=200.00MiB, used=67.88MiB

For a filesystem without snapshots, 70M metadata, extent
checking eats max memory about 110M, this is a nightmare
for some system with low memory.

It is very likely that extent record can be freed quickly
for a filesystem without snapshots, improve this by trying
if it can free memory after adding data/tree backrefs.

This patch reduces max memory cost from 110M to 40M for
extents checking for the above case.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix btrfs-image old_restore fsck failure
Gui Hecheng [Thu, 26 Jun 2014 02:53:01 +0000 (10:53 +0800)]
btrfs-progs: fix btrfs-image old_restore fsck failure

Steps to reproduce:
# mkfs.btrfs -f <dev1>
# btrfs-image <dev1> <image_file>
# btrfs-image -r -o <image_file> <dev2>
# btrfs check <dev2>

btrfs check output:
: read block failed check_tree_block
: Couldn't read tree root
: Couldn't open file system

The btrfs-image should not mess with the chunk tree under the old_restore way.

The new restore way was introduced by:
commit d6f7e3da0dae7b60cb7565f8a47c3b9045c52d1d
Btrfs-progs: make btrfs-image restore with a valid chunk tree V2
...
And the following commit enhanced the new restore on the valid chunk tree
building stuff:
commit ef2a8889ef813ba77061f6a92f4954d047a78932
Btrfs-progs: make image restore with the original device offsets
...

But the second commit should not effect the old_restore way since the
old_restore way doesn't try to build a valid chunk tree at all.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Avoid double-free of fs_devices->list
Satoru Takeuchi [Tue, 24 Jun 2014 07:30:21 +0000 (16:30 +0900)]
btrfs-progs: Avoid double-free of fs_devices->list

I found the following patch is insufficient.

===============================================================================
commit 6e6b32ddf58db54f714d0f263c2589f4859e8b5e
Author: Adam Buchbinder <abuchbinder@google.com>
Date:   Fri Jun 13 16:43:56 2014 -0700

    btrfs-progs: Fix a use-after-free in the volumes code.
===============================================================================

"btrfs filesystem show <dev>" with this patch causes segmentation fault
if "<dev>" is a not-mounted Btrfs filesystem.

===============================================================================
Label: none  uuid: <cut here>
        Total devices 1 FS bytes used 112.00KiB
        devid    1 size 59.12GiB used 2.04GiB path /dev/sdd1

Segmentation fault (core dumped)
===============================================================================

It's due to double-free of fs_devices->list as follows.

===============================================================================
cmd_show
  -> list_del(&fs_devices->list)     # 1st one.
  -> btrfs_close_devices(fs_devices)
    -> list_del(&fs_devices->list)   # <- 2nd one introduced at 6e6b32dd.
                                          Double-free happens here.
===============================================================================

First list_del() can safely be removed because fs_devices->list will be
deleted by second one, soon.

Signed-off-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Cc: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: use BTRFS_SUPER_INFO_SIZE to replace raw 4096 in btrfs-image
Gui Hecheng [Thu, 26 Jun 2014 02:53:06 +0000 (10:53 +0800)]
btrfs-progs: use BTRFS_SUPER_INFO_SIZE to replace raw 4096 in btrfs-image

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: cleanup possible silent failure in btrfs-image
Gui Hecheng [Thu, 26 Jun 2014 02:53:04 +0000 (10:53 +0800)]
btrfs-progs: cleanup possible silent failure in btrfs-image

If the malloc above fails, the btrfs-image will exit directly
without any error messages.
Now just return the ENOMEM errno and let the caller prompt the
error message.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: deal with malloc failure in btrfs-image
Gui Hecheng [Thu, 26 Jun 2014 02:53:02 +0000 (10:53 +0800)]
btrfs-progs: deal with malloc failure in btrfs-image

Handle the malloc failure for dump_worker in the same way as
the restore worker.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: replace BTRFS_NUM_MIRRORS with BTRFS_MAX_MIRRORS
Gui Hecheng [Wed, 25 Jun 2014 10:04:17 +0000 (18:04 +0800)]
btrfs-progs: replace BTRFS_NUM_MIRRORS with BTRFS_MAX_MIRRORS

The chunk-recover.c/BTRFS_NUM_MIRRORS in the userspace means
the same thing as ctree.h/BTRFS_MAX_MIRRORS in the kernelspace,
so to stay consistent with the kernelspace, just make this movement
in the userspace:
chunk-recover.c/BTRFS_NUM_MIRRORS
===>
ctree.h/BTRFS_MAX_MIRRORS

This provides convenience for future use.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add supported attr flags to btrfs(5)
Eric Sandeen [Thu, 26 Jun 2014 20:38:33 +0000 (15:38 -0500)]
btrfs-progs: add supported attr flags to btrfs(5)

The chattr(1) manpage suffers from the same problems mount(1)
had: many options listed, not kept up to date for various
filesystems.

I've submitted a manpage update for chattr(1) which says to
refer to filesystem-specific manpages for supported attributes;
this patch updates btrfs(5) to list the attributes supported
by btrfs.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[added some asciidoc markups, adjusted formatting]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add minimum device size check
Qu Wenruo [Fri, 4 Jul 2014 07:29:17 +0000 (15:29 +0800)]
btrfs-progs: Add minimum device size check

Btrfs has global block reservation, so even mkfs.btrfs can execute
without problem, there is still a possibility that the filesystem can't
be mounted.
For example when mkfs.btrfs on a 8M file on x86_64 platform, kernel will
refuse to mount due to ENOSPC, since system block group takes 4M and
mixed block group takes 4M, and global block reservation will takes all
the 4M from mixed block group, which makes btrfs unable to create uuid
tree.

This patch will add minimum device size check before actually mkfs.
The minimum size calculation uses a simplified one:
minimum_size_for_each_dev = 2 * (system block group + global block rsv)
and global block rsv = leafsize << 10

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: check: Fix wrong level access
Hugo Mills [Mon, 5 May 2014 17:07:49 +0000 (18:07 +0100)]
btrfs-progs: check: Fix wrong level access

There's no reason to assume that the bad key order is in a leaf block,
so accessing level 0 of the path is going to be an error if it's actually
a node block that's bad.

Reported-by: Chris Mason <clm@fb.com>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fsck: add tests for extent tree rebuilding
Wang Shilong [Thu, 29 May 2014 10:01:44 +0000 (18:01 +0800)]
Btrfs-progs: fsck: add tests for extent tree rebuilding

We need test to verify extent tree rebuilding work, this test
create a strange filesystem with some snapshots, destroy
extent root node, and run fsck with "--init-extent-tree".

Since this tests need btrfs internal tool(btrfs-corrupt-block),so
i add this test into btrfs-progs.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: delete invalid output file when btrfs-image failed
Gui Hecheng [Tue, 24 Jun 2014 03:16:45 +0000 (11:16 +0800)]
btrfs-progs: delete invalid output file when btrfs-image failed

When btrfs-image failed to create an image, the invalid output file
had better be deleted to prevent being used mistakenly in the future.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: deal with invalid option combinations for btrfs-image
Gui Hecheng [Tue, 24 Jun 2014 03:16:12 +0000 (11:16 +0800)]
btrfs-progs: deal with invalid option combinations for btrfs-image

For btrfs-image,
dump  may not come with option '-o'
-r may not come with option '-c', '-s', '-w', dev_cnt != 1
-m may not come with dev_cnt < 2
All of the above should be regarded as invalid combinations,
and the usage will show up.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: update manpage for btrfs-image with -m option added
Gui Hecheng [Thu, 19 Jun 2014 01:46:03 +0000 (09:46 +0800)]
btrfs-progs: update manpage for btrfs-image with -m option added

The btrfs-image support multiple devices with -m specified.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: cleanup duplicate assignment of variable leaf for btrfs-image
Gui Hecheng [Thu, 19 Jun 2014 01:46:00 +0000 (09:46 +0800)]
btrfs-progs: cleanup duplicate assignment of variable leaf for btrfs-image

The value of variable leaf in while loop don't have to be set
for every round. Just move it outside.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: document updates
Wang Shilong [Thu, 19 Jun 2014 01:27:09 +0000 (09:27 +0800)]
btrfs-progs: restore: document updates

Add some missing options, also improve some confusing
expressions.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: don't allow users to specify -r and -f at the same time
Wang Shilong [Thu, 19 Jun 2014 01:27:08 +0000 (09:27 +0800)]
btrfs-progs: restore: don't allow users to specify -r and -f at the same time

These two options are used for same purpose, but they are exclusive with
each other. Make it clear to common users.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: output resason why it fails to read root
Wang Shilong [Thu, 19 Jun 2014 01:27:07 +0000 (09:27 +0800)]
btrfs-progs: restore: output resason why it fails to read root

Previously if restore could not read users specified fs root, it would
output following message:

 Error reading root

With this patch, it will output message like:

 Fail to read root 1000: No such file or directory

Signed-off-byr Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: fix wrong return value if it fails to read specified fs root
Wang Shilong [Thu, 19 Jun 2014 01:27:06 +0000 (09:27 +0800)]
btrfs-progs: restore: fix wrong return value if it fails to read specified fs root

Steps to reproduce:
 # mkfs.btrfs -f /dev/sda9
 # btrfs restore -f 1 -o /tmp /dev/sda9
 # echo $?

Fix to return 1 in this failure path.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: check if specified root is fs/file tree firstly
Wang Shilong [Thu, 19 Jun 2014 01:27:05 +0000 (09:27 +0800)]
btrfs-progs: restore: check if specified root is fs/file tree firstly

Steps to reproduce:
 # mkfs.btrfs -f /dev/sda9
 # mount /dev/sda9 /mnt
 # dd if=/dev/zero of=/mnt/data bs=1M count=1
 # btrfs restore -r /dev/sda9 -r 2 -o /tmp

If users don't input a valid fs/file root objectid, btrfs restore still
continue and don't restore anything, this is unfriendly, we could
check it firstly.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fsck: switch to is_fstree()
Wang Shilong [Thu, 19 Jun 2014 01:27:04 +0000 (09:27 +0800)]
Btrfs-progs: fsck: switch to is_fstree()

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: restore: check lzo compress length
Vincent Stehlé [Wed, 18 Jun 2014 16:51:19 +0000 (18:51 +0200)]
btrfs-progs: restore: check lzo compress length

When things go wrong for lzo-compressed btrfs, feeding lzo1x_decompress_safe()
with corrupt data during restore can lead to crashes. Reduce the risk by adding
a check on the input length.

Signed-off-by: Vincent Stehlé <vincent.stehle@laposte.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Properly size the leafsize field in the mdrestore_struct struct
Adam Buchbinder [Thu, 12 Jun 2014 22:57:33 +0000 (15:57 -0700)]
btrfs-progs: Properly size the leafsize field in the mdrestore_struct struct

It's 32 bits as defined in ctree.h, but the struct had it as 64 bits.

Found using MemorySanitizer.

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Use sparse files for filesystem conversion tests
Adam Buchbinder [Thu, 12 Jun 2014 16:08:33 +0000 (09:08 -0700)]
btrfs-progs: Use sparse files for filesystem conversion tests

On my system, this brings the FS conversion test suite's runtime from over
ten seconds down to under two.

Thanks to Julien Muchembled for the suggestion.

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: clean fsck noise for free inode generation 0
Gui Hecheng [Wed, 18 Jun 2014 05:28:22 +0000 (13:28 +0800)]
btrfs-progs: clean fsck noise for free inode generation 0

When we write a btrfs to full and then we have no space left for
free space cache.
The btrfs check will output msg as follows which is noise indeed:
# free space inode generation (0) did not match
  free space cache generation (XXX)

When the free space cache is not written out normally,
the free inode generation will be 0.
In this condition, no noise should be outputed.
Also, check 0-sized inode eariler together with 0-generationed inode.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Improve the parse_size() error message
Qu Wenruo [Fri, 13 Jun 2014 02:55:57 +0000 (10:55 +0800)]
btrfs-progs: Improve the parse_size() error message

When using parse_size(), even non-numeric value is passed, it will only
give error message "ERROR: size value is empty", which is quite
confusing for end users.

This patch will introduce more meaningful error message for the
following new cases
1) Invalid size string (non-numeric string)
2) Minus size value (like "-1K")

Also this patch will take full use of endptr returned by strtoll() to
reduce unneeded loop.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: show meaningful msgs for replace cmd upon raid56
Gui Hecheng [Thu, 5 Jun 2014 02:02:52 +0000 (10:02 +0800)]
btrfs-progs: show meaningful msgs for replace cmd upon raid56

This depends on the kernel patch:
[PATCH] btrfs:replace EINVAL with EOPNOTSUPP for dev_replace

This catches the EOPNOTSUPP and output msg that says dev_replace raid56
is not currently supported. Note that the msg will only be shown when
run dev_replace not in background.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix max mirror number error for chunk-recover
Gui Hecheng [Thu, 12 Jun 2014 02:25:00 +0000 (10:25 +0800)]
btrfs-progs: fix max mirror number error for chunk-recover

When run chunk-recover on a health btrfs(data profile raid0, with
plenty of data), the program has a chance to abort on the number
of mirrors of an extent.

According to the kernel code, the max mirror number of an extent
is 3 not 2:
ctree.h:  BTRFS_MAX_MIRRORS 3
chunk-recover.c : BTRFS_NUM_MIRRORS 2
just change BTRFS_NUM_MIRRORS to 3, and everything goes well.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix missing parity stripe for raid6 in chunk-recover
Gui Hecheng [Thu, 12 Jun 2014 02:24:59 +0000 (10:24 +0800)]
btrfs-progs: fix missing parity stripe for raid6 in chunk-recover

When deal with the p & q stripes for data profile raid6, chunk-recover
forgets to insert them into the chunk record. Just insert them back
freely.
Also wrap the insert procedure into a new function, fill_chunk_up.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add some simple end-to-end tests for btrfs-convert
Adam Buchbinder [Wed, 21 May 2014 17:20:27 +0000 (10:20 -0700)]
btrfs-progs: Add some simple end-to-end tests for btrfs-convert

These use the system's mke2fs, and don't require loop devices
or root privileges.

They don't pick up anything with the default flags right now,
but they do pick up some sanitizer issues when the tools are
compiled with any of -fsanitize={address,memory,thread}.

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add human readable flags output for chunk/block group type.
Qu Wenruo [Thu, 12 Jun 2014 07:12:32 +0000 (15:12 +0800)]
btrfs-progs: Add human readable flags output for chunk/block group type.

Current btrfs-debug-tree output chunk/block group type as numbers,
which makes it hard to understand and need to check the source to
understand the meaning.

This patch will convert numeric type output to human readable strings.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add human readable flags output string for extent flags.
Qu Wenruo [Thu, 12 Jun 2014 07:12:31 +0000 (15:12 +0800)]
btrfs-progs: Add human readable flags output string for extent flags.

Current btrfs-debug-tree outputs extent flags as numbers,
which makes it hard to understand and need to check the source to
understand the meaning.

This patch will convert numberic flags output to human readable strings.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: update manpage for btrfs resize support size unit t/p/e
Gui Hecheng [Tue, 1 Apr 2014 10:01:10 +0000 (18:01 +0800)]
btrfs-progs: update manpage for btrfs resize support size unit t/p/e

btrfs resize now support size unit parse of k/m/g/t/p/e in kernel space,
adopt the changes in userspace manpage.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Fix a use-after-free in the volumes code.
Adam Buchbinder [Fri, 13 Jun 2014 23:43:56 +0000 (16:43 -0700)]
btrfs-progs: Fix a use-after-free in the volumes code.

When a struct btrfs_fs_devices was being torn down by
btrfs_close_devices(), there was an invalidated pointer in the global
list fs_uuids which still pointed to it; if a device was closed and
then reopened (which btrfs-convert does), freed memory would be
accessed.

This was found using ThreadSanitizer (pretty much doing what
AddressSanitizer would, but not exiting after the first failure).
To reproduce, build with -fsanitize=thread and run 'make test'.
Representative output is below.

This change makes the current tests TSan-clean.

WARNING: ThreadSanitizer: heap-use-after-free (pid=29161)
  Read of size 8 at 0x7d180000eee0 by main thread:
    #0 memcmp ??:0
    #1 find_fsid .../volumes.c:81
    #2 device_list_add .../volumes.c:95
    #3 btrfs_scan_one_device .../volumes.c:259
    #4 btrfs_scan_fs_devices .../disk-io.c:1002
    #5 __open_ctree_fd .../disk-io.c:1090
    #6 open_ctree_fd .../disk-io.c:1191
    #7 do_convert .../btrfs-convert.c:2317
    #8 main .../btrfs-convert.c:2745

  Previous write of size 8 at 0x7d180000eee0 by main thread:
    #0 free ??:0
    #1 btrfs_close_devices .../volumes.c:191
    #2 close_ctree .../disk-io.c:1401
    #3 do_convert .../btrfs-convert.c:2300
    #4 main .../btrfs-convert.c:2745

  Location is heap block of size 96 at 0x7d180000eee0 allocated by main thread:
    #0 calloc ??:0 (exe+0x00000002acc6)
    #1 device_list_add .../volumes.c:97
    #2 btrfs_scan_one_device .../volumes.c:259
    #3 btrfs_scan_fs_devices .../disk-io.c:1002
    #4 __open_ctree_fd .../disk-io.c:1090
    #5 open_ctree_fd .../disk-io.c:1191
    #6 do_convert .../btrfs-convert.c:2256
    #7 main .../btrfs-convert.c:2745

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Fix undefined behavior in radix-tree.c.
Adam Buchbinder [Fri, 13 Jun 2014 21:18:50 +0000 (14:18 -0700)]
btrfs-progs: Fix undefined behavior in radix-tree.c.

When running with UndefinedBehaviorSanitizer, the tests produce the following
error:

  radix-tree.c:836:30: runtime error: shift exponent 18446744073709551613
  is too large for 64-bit type 'unsigned long'

(That's a negative shift exponent represented as an unsigned long.)

Even though the value is discarded in those cases, it's still undefined
behavior; see the C99 standard, section 6.5.7, paragraph three: "If the
value of the right operand is negative [...] the behavior is undefined."

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: rename btrfs-mount.5 to btrfs.5
David Sterba [Wed, 18 Jun 2014 15:17:38 +0000 (17:17 +0200)]
btrfs-progs: rename btrfs-mount.5 to btrfs.5

We'd like to keep the name of category 5 page same as the filesystem
to be consistent with the rest.

Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add mount options to btrfs-mount.5
Eric Sandeen [Wed, 11 Jun 2014 22:14:55 +0000 (17:14 -0500)]
btrfs-progs: add mount options to btrfs-mount.5

This is a straight cut and paste from the util-linux
mount manpage into btrfs-mount.5

It's pretty much impossible for util-linux to keep up
with every filesystem out there, and Karel has more than
once expressed a wish that mount options move into fs-specific
manpages.

So, here we go.

The way btrfs asciidoc is generated, there's not a trivial
way to have both btrfs(5) and btrfs(8) so I named it btrfs-mount(5)
for now.  A bit ick and I'm open to suggestions.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: cleanup unused assignment for chunk-recover
Gui Hecheng [Thu, 12 Jun 2014 02:25:01 +0000 (10:25 +0800)]
btrfs-progs: cleanup unused assignment for chunk-recover

The 'num_unordered' will be recounted after 'goto out',
just remove it.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: canonicalize pathnames for device commands
Jeff Mahoney [Wed, 4 Jun 2014 20:43:11 +0000 (16:43 -0400)]
btrfs-progs: canonicalize pathnames for device commands

mount(8) will canonicalize pathnames before passing them to the kernel.
Links to e.g. /dev/sda will be resolved to /dev/sda. Links to /dev/dm-#
will be resolved using the name of the device mapper table to
/dev/mapper/<name>.

Btrfs will use whatever name the user passes to it, regardless of whether
it is canonical or not. That means that if a 'btrfs device ready' is
issued on any device node pointing to the original device, it will adopt
the new name instead of the name that was used during mount.

Mounting using /dev/sdb2 will result in df:
/dev/sdb2      209715200 39328 207577088   1% /mnt

lrwxrwxrwx 1 root root 4 Jun  4 13:36 /dev/whatever-i-like -> sdb2
/dev/whatever-i-like 209715200 39328 207577088   1% /mnt

Likewise, mounting with /dev/mapper/whatever and using /dev/dm-0 with a
btrfs device command results in df showing /dev/dm-0. This can happen with
multipath devices with friendly names enabled and doing something like
'partprobe' which (at least with our version) ends up issuing a 'change'
uevent on the sysfs node. That *always* uses the dm-# name, and we get
confused users.

This patch does the same canonicalization of the paths that mount does
so that we don't end up having inconsistent names reported by ->show_devices
later.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[use PATH_MAX in canonicalize_dm_name]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: make pretty Documentation/ build match the rest
Eric Sandeen [Fri, 30 May 2014 16:27:14 +0000 (11:27 -0500)]
btrfs-progs: make pretty Documentation/ build match the rest

This is the most important patch ever.  ;)

I found this to be less aesthetically pleasing than it was before:

     [CC]     btrfstune.o
 Making all in Documentation
     ASCIIDOC btrfs-convert.xml
     [LD]     btrfstune
     XMLTO btrfs-convert.8
     [CC]     btrfs-show-super.o
     GZIP btrfs-convert.8.gz
     [LD]     btrfs-show-super
     ASCIIDOC btrfs-debug-tree.xml
     XMLTO btrfs-debug-tree.8

so I shortened the pretty-text to match what we had before.

Also, make clean "quiet" like it is in the top dir.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[Changed to ASCII and XMLTO]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Documentation: Add filter section for btrfs-balance.
Qu Wenruo [Tue, 3 Jun 2014 06:20:08 +0000 (14:20 +0800)]
btrfs-progs: Documentation: Add filter section for btrfs-balance.

Man page for 'btrfs-balance' mentioned <filters> but does not explain
them, which make end users hard to use '-d', '-m' or '-s options.

This patch will use the explanations from
https://btrfs.wiki.kernel.org/index.php/Balance_Filters to enrich the
man page.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: update man page for btrfs-show-super
Gui Hecheng [Thu, 8 May 2014 03:03:58 +0000 (11:03 +0800)]
btrfs-progs: update man page for btrfs-show-super

Add '-f' option for btrfs-show-super manpage,
This option implies that sys chunk array and backup roots info
will show up.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fsck: add an option to check data csums
Wang Shilong [Wed, 28 May 2014 11:22:40 +0000 (19:22 +0800)]
Btrfs-progs: fsck: add an option to check data csums

This patch adds an option '--check-data-csum' to verify data checksums.
fsck won't check data csums unless users specify this option explictly.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: fsck: fix wrong check for btrfs_read_fs_root()
Wang Shilong [Wed, 28 May 2014 11:20:41 +0000 (19:20 +0800)]
Btrfs-progs: fsck: fix wrong check for btrfs_read_fs_root()

When encountering a corrupted fs root node, fsck hit following message:

Check tree block failed, want=29360128, have=0
Check tree block failed, want=29360128, have=0
Check tree block failed, want=29360128, have=0
Check tree block failed, want=29360128, have=0
Check tree block failed, want=29360128, have=0
read block failed check_tree_block
Checking filesystem on /dev/sda9
UUID: 0d295d80-bae2-45f2-a106-120dbfd0e173
checking extents
Segmentation fault (core dumped)

This is because in btrfs_setup_all_roots(), we check
btrfs_read_fs_root() return value by verifing whether it is
NULL pointer, this is wrong since btrfs_read_fs_root() return
PTR_ERR(ret), fix it.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix blindly goto failure for chunk-recover
Gui Hecheng [Wed, 28 May 2014 10:49:38 +0000 (18:49 +0800)]
btrfs-progs: fix blindly goto failure for chunk-recover

If the csum of one stripe is not able to judge the order of two
device extents, the stripe may happen to belong to the device extent
that is already kicked out as ordered.
Take this condition into consideration, don't report failure and
give more tries with the stripes following.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix uninitialized number count in chunk-recover
Gui Hecheng [Wed, 28 May 2014 10:49:37 +0000 (18:49 +0800)]
btrfs-progs: fix uninitialized number count in chunk-recover

When count the number of unordered device extents in chunk-recover,
the counter should be reinitialized to be used.
Also, introduce a new function for the counting job.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add quota group verify code
Mark Fasheh [Wed, 7 May 2014 20:07:17 +0000 (13:07 -0700)]
btrfs-progs: add quota group verify code

This patch adds functionality (in qgroup-verify.c) to compute bytecounts in
subvolume quota groups. The original groups are read in and stored in memory
so that after we compute our own bytecounts, we can compare them with those
on disk. A print function is provided to do this comparison and show the
results on the console.

A 'qgroup check' pass is added to btrfsck. If any subvolume quota groups
differ from what we compute, the differences for them are printed.  We also
provide an option '--qgroup-report' which will run only the quota check code
and print a report on all quota groups.  Other than making it possible to
verify that our qgroup changes work correctly, this mode can also be used in
xfstests for automated checking after qgroup tests.

This patch does not address the following:
- compressed counts are identical to non compressed, because kernel doesn't
  make the distinction yet.  Adding the code to verify compressed counts
  shouldn't be hard at all though once kernel can do this.
- It is only concerned with subvolume quota groups (like most of
  btrfs-progs).

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: import ulist
Mark Fasheh [Wed, 7 May 2014 20:07:16 +0000 (13:07 -0700)]
btrfs-progs: import ulist

qgroup-verify.c wants this for walking root refs.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: print qgroup excl as unsigned
Mark Fasheh [Wed, 7 May 2014 20:07:15 +0000 (13:07 -0700)]
btrfs-progs: print qgroup excl as unsigned

It's unsigned in the structure definition.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: receive, allow to continue after errors happen
Filipe David Borba Manana [Fri, 23 May 2014 19:14:56 +0000 (20:14 +0100)]
Btrfs-progs: receive, allow to continue after errors happen

Due to either bugs in send (kernel) that generate a command against
a wrong path for example, or transient errors on the receiving side,
we stopped processing the send stream immediately and exited with
an error.

It's often desirable to continue processing the send stream even if an
error happens while processing a single command from the send stream.

This change just adds a --max-errors <N> parameter, whose default value
is 1 (preserving current behaviour), that allows to tolerate N errors
before stopping. A value of 0 means to never stop no matter how many
errors we get into while processing the send stream. Regardless of its
value, errors are always printed to stderr when they happen, just like
before this change.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-image: Fix a data race in build_chunk_tree.
Adam Buchbinder [Mon, 19 May 2014 05:40:42 +0000 (22:40 -0700)]
btrfs-image: Fix a data race in build_chunk_tree.

A mdrestore_struct was being written to without its mutex being held.
This race was found with ThreadSanitizer; the relevant part of the report
looks like this:

WARNING: ThreadSanitizer: data race (pid=18828)
  Write of size 8 at 0x7fffffc3d088 by main thread:
    #0 build_chunk_tree .../btrfs-progs/btrfs-image.c:2233
    #1 __restore_metadump .../btrfs-progs/btrfs-image.c:2294
    #2 restore_metadump .../btrfs-progs/btrfs-image.c:2345
    #3 main .../btrfs-progs/btrfs-image.c:2545

  Previous read of size 8 at 0x7fffffc3d088 by thread T1 (mutexes: write M0):
    #0 restore_worker .../btrfs-progs/btrfs-image.c:1636

  Location is stack of main thread.

  Mutex M0 created at:
    #0 pthread_mutex_init ??:0
    #1 mdrestore_init .../btrfs-progs/btrfs-image.c:1766
    #2 __restore_metadump .../btrfs-progs/btrfs-image.c:2286
    #3 restore_metadump .../btrfs-progs/btrfs-image.c:2345
    #4 main .../btrfs-progs/btrfs-image.c:2545

  Thread T1 (tid=18830, running) created by main thread at:
    #0 pthread_create ??:0
    #1 mdrestore_init .../btrfs-progs/btrfs-image.c:1784
    #2 __restore_metadump .../btrfs-progs/btrfs-image.c:2286
    #3 restore_metadump .../btrfs-progs/btrfs-image.c:2345
    #4 main .../btrfs-progs/btrfs-image.c:2545

Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: mkfs: allow UUID specification at mkfs time
Eric Sandeen [Wed, 14 May 2014 17:39:07 +0000 (12:39 -0500)]
btrfs-progs: mkfs: allow UUID specification at mkfs time

Allow the specification of the filesystem UUID at mkfs time.

Non-unique unique IDs are rejected.  This includes attempting
to re-mkfs with the same UUID; if you really want to do that,
you can mkfs with a new UUID, then re-mkfs with the one you
wanted.

(Implemented only for mkfs.btrfs, not btrfs-convert).

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[converted help to asciidoc]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix compiler warning
Christian Hesse [Tue, 3 Jun 2014 11:37:45 +0000 (13:37 +0200)]
btrfs-progs: fix compiler warning

gcc 4.9.0 gives a warning: format ‘%d’ expects argument of type ‘int’,
but argument 2 has type ‘u64’

Using %llu and casting to unsigned long long (same as bytenr) fixes this.

Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix btrfs-fragments build
Eric Sandeen [Fri, 30 May 2014 16:29:19 +0000 (11:29 -0500)]
btrfs-progs: fix btrfs-fragments build

When I made all the btrfs-foo.c targets generic, I somehow
managed to break the libs definition for btrfs-fragments by
dropping the "s" off the end.

Fix that, although apparently nobody is building this tool.  :)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add dev uuid output for print_dev_item().
Qu Wenruo [Fri, 30 May 2014 02:19:04 +0000 (10:19 +0800)]
btrfs-progs: Add dev uuid output for print_dev_item().

The original print_dev_item() only prints device id,total bytes and
bytes used.

When it comes to debug duplicated device id, dev uuid
is needed to distinguish different devices since device id is not
reliable.
(Although the current dev replace implement will reuse the dev uuid,
so not really helpful)

This patch adds dev uuid output for print_dev_item().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: clarify manpage for btrfstune seeding option
Gui Hecheng [Wed, 21 May 2014 08:03:57 +0000 (16:03 +0800)]
btrfs-progs: clarify manpage for btrfstune seeding option

The btrfstune -S option accepts a positive value to enable seeding,
and a zero to disable seeding, negtive is not allowed.

Add "positive, zero, negative" sentences to btrfstune manpage.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: clarify mkfs.btrfs --alloc-start option
Eric Sandeen [Thu, 15 May 2014 17:14:40 +0000 (19:14 +0200)]
btrfs-progs: clarify mkfs.btrfs --alloc-start option

Fedora had a bug where a poor user thought that --alloc-start
meant that the filesystem would be created at that offset into
the device, rather than just starting allocations at that offset.
A subtle difference, but worth clarifying, because the manpage
is misleading on this point.

The original commit log for this option says:

    Add mkfs.btrfs -A offset to control allocation start on devices

    This is a utility option for the resizer, it makes sure to allocate
    at offset bytes in the disk or higher.  It ensures the resizer will have
    something to move when testing it.

so allude to that intended use in the manpage.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[converted to asciidoc]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: Add information about libblkid requirement to INSTALL file
Augusto Mecking Caringi [Sat, 31 May 2014 01:05:10 +0000 (22:05 -0300)]
btrfs-progs: Add information about libblkid requirement to INSTALL file

Signed-off-by: Augusto Mecking Caringi <augustocaringi@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: doc: fix symlink target for btrfsck.8
David Sterba [Mon, 5 May 2014 14:54:00 +0000 (16:54 +0200)]
btrfs-progs: doc: fix symlink target for btrfsck.8

The manpage of btrfsck.8 is supposed to link to btrfs-check.8 .

Reported-by: WorMzy Tykashi <wormzy.tykashi@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: fix build, manpage compression command
Christian Hesse [Tue, 3 Jun 2014 06:09:25 +0000 (08:09 +0200)]
btrfs-progs: fix build, manpage compression command

man pages for btrfs-progs are compressed by gzip by default. In Makefile
the variable GZIP is use, this evaluates to 'gzip gzip' on my system.
>From man gzip:

> The environment variable GZIP can hold a set of default options for
> gzip. These options are interpreted first and can be overwritten by
> explicit command line parameters.

So using any other variable name fixes this.

Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agoBtrfs-progs: debug-tree, add option to dump a single tree
Filipe David Borba Manana [Fri, 23 May 2014 13:44:29 +0000 (14:44 +0100)]
Btrfs-progs: debug-tree, add option to dump a single tree

Very often while debugging filesystems with many subvolumes and/or
snapshots, specially when they are large, I want to see only the
content of one of the trees. So this change just adds an option
to btrfs-debug-tree to allow to specify the id of the tree we're
interesting in dumping to stdout.

Example:  btrfs-debug-tree -t 257 /dev/sdc

Will only dump the tree of the first snapshot or subvolume that was
created.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-show-super: don't try to print not-superblocks
Eric Sandeen [Wed, 14 May 2014 02:03:04 +0000 (21:03 -0500)]
btrfs-show-super: don't try to print not-superblocks

If we point btrfs-show-super at a not-btrfs-device and
try to print all superblocks, bad things are apt to happen:

superblock: bytenr=274877906944, device=/dev/sdc2
---------------------------------------------------------
btrfs-show-super: ctree.h:1984: btrfs_super_csum_size: \
Assertion `!(t >= (sizeof(btrfs_csum_sizes) / sizeof((btrfs_csum_sizes)[0])))' failed.
csum 0xAborted

Don't try to print superblocks that don't look like superblocks,
and add an "-f" (force) option to try anyway, if the user
really wants to give it a shot.

Fix some spelling & capitalization while we're at it.

The manpage says that if any problem happens, 1 will
be returned, but that's already not true today LOL, so
I didn't bother to make it true when we detect bad
sb magic, either...

I figure it's worth continuing and trying all superblocks
in case just one has a corrupt magic.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
[renamed -f to -F due to clash with existing option, converted
relevant docs to asciidoc]
Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: use check_argc_exact to check arg number of btrfs-rescue
Gui Hecheng [Thu, 15 May 2014 01:29:09 +0000 (09:29 +0800)]
btrfs-progs: use check_argc_exact to check arg number of btrfs-rescue

The btrfs-rescue accepts exactly one arg for both
chunk-recover & super-recover, use check_argc_exact clearly.

Signed-off-by: David Sterba <dsterba@suse.cz>
9 years agobtrfs-progs: add sys_chunk_array and backup roots info to show-super
Gui Hecheng [Fri, 16 May 2014 01:23:37 +0000 (09:23 +0800)]
btrfs-progs: add sys_chunk_array and backup roots info to show-super

Add sys chunk array and backup roots info if the new option '-f'
if specified.
This may be useful for debugging sys_chunk related issues.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>