platform/upstream/btrfs-progs.git
11 years agobtrfs-progs: use IEC units for sizes
David Sterba [Wed, 10 Jul 2013 17:30:51 +0000 (19:30 +0200)]
btrfs-progs: use IEC units for sizes

As implemented now, we use 1024 based units but reporting 1000 based,
let's finally fix that and add optional unit bases later.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: update usage message for cmds-restore
Filipe David Borba Manana [Wed, 10 Jul 2013 08:36:34 +0000 (09:36 +0100)]
Btrfs-progs: update usage message for cmds-restore

Mention that a target path argument is mandatory unless
the -l option is supplied. Also mention about the existence
of the -l option, which was previously not announced.

$ btrfs restore -v /dev/sdb3
usage: btrfs restore [options] <device>

    Try to restore files from a damaged filesystem (unmounted)

    -s              get snapshots
    -v              verbose
    -i              ignore errors
    -o              overwrite
    -t              tree location
    -f <offset>     filesystem location
    -u <block>      super mirror
    -d              find dir
$ echo $?
129

After specifying a target path, the command works as expected:

$ btrfs restore -v /dev/sdb3 files2/
Restoring files2/file1
Done searching
$ echo $?
0

V2: Updated command synopsis by suggestion of Anand Jain.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: use reentrant localtime
David Sterba [Tue, 9 Jul 2013 16:39:24 +0000 (18:39 +0200)]
btrfs-progs: use reentrant localtime

localtime may return NULL (when an error is detected eg. after setting
tzname), followed by a segfault when the values is about to be used.
localtime_r works, does not set tzname and does not return NULL.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: use accessor macro for otime
David Sterba [Tue, 9 Jul 2013 16:38:55 +0000 (18:38 +0200)]
btrfs-progs: use accessor macro for otime

The root_item->otime value must be accessed via the macro otherwise the
value is incorrect on big endian machines.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: fix duplicate __[su]* typedefs on ppc64
Michal Marek [Tue, 9 Jul 2013 16:38:46 +0000 (18:38 +0200)]
btrfs-progs: fix duplicate __[su]* typedefs on ppc64

The <ext2fs/ext2_types.h> header does attempt to avoid conflicts with
<linux/types.h>, but on ppc64, <asm-generic/int-ll64.h> gets somehow
included by other headers.

Include <linux/types.h> explicitly, so that <ext2fs/ext2_types.h>
notices it. The proper fix would be to fix <ext2fs/ext2_types.h> to not
use its own typedefs.

Originally observed in btrfs-convert, put the include into kerncompat.h
to avoid future problems.

Signed-off-by: Michal Marek <mmarek@suse.cz>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: Fix getopt on arm/ppc platforms
David Sterba [Tue, 9 Jul 2013 16:38:29 +0000 (18:38 +0200)]
btrfs-progs: Fix getopt on arm/ppc platforms

(same as commit bb0eabc383e9a3fde7cdb02591ca88243f3e31fb)
There, 'char' is unsigned, so once assigned '-1' from getopt, it gets
the value 255. Then, it compared to '-1' gives false.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add missing path release
Filipe David Borba Manana [Tue, 9 Jul 2013 11:45:23 +0000 (12:45 +0100)]
Btrfs-progs: add missing path release

Before repeating the search in extent-tree.c:lookup_inline_extent_backref(),
release the current path.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: Fix automatic prerequisite generation
Kusanagi Kouichi [Sat, 6 Jul 2013 08:22:31 +0000 (17:22 +0900)]
btrfs-progs: Fix automatic prerequisite generation

Some files don't compile because of insufficient prerequisite.

$ make btrfs
...
    [CC]     btrfs.o
btrfs.c:24:21: fatal error: version.h: No such file or directory
 #include "version.h"
                     ^
compilation terminated.
make: *** [btrfs.o] Error 1

Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: optimize function btrfs_read_chunk_tree
Filipe David Borba Manana [Tue, 30 Jul 2013 11:08:17 +0000 (12:08 +0100)]
Btrfs-progs: optimize function btrfs_read_chunk_tree

After reading all device items from the chunk tree, don't
exit the loop and then navigate down the tree again to find
the chunk items. Instead just read all device items and
chunk items with a single tree search. This is possible
because all device items are found before any chunk item in
the chunks tree.

This is a port of the corresponding kernel patch to keep both
kernel and btrfs-progs identical:

https://patchwork.kernel.org/patch/2835529/

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove incorrect slot decrement
Filipe David Borba Manana [Fri, 5 Jul 2013 13:35:07 +0000 (14:35 +0100)]
Btrfs-progs: remove incorrect slot decrement

In btrfs_set_block_flags() we want to check if the slot
in the leaf points to the first item in the leaf - if it
doesn't check if the previous item in the leaf is an extent
item. By removing this extra slot decrement we are indeed
checking the item right before the slot, and not the second
item before.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix optimization in btrfs_lookup_extent_info
Filipe David Borba Manana [Thu, 4 Jul 2013 15:48:39 +0000 (16:48 +0100)]
Btrfs-progs: fix optimization in btrfs_lookup_extent_info

If we did a tree search with the goal to find a metadata item
but the search failed with return value 1, we attempt to see
if in the same leaf there's a corresponding extent item, and if
there's one, just use it instead of doing another tree search
for this extent item. The check in the leaf was wrong because
it was seeking for a metadata item instead of an extent item.

This optimization was also being triggered incorrectly, as it
was evaluating path->slots which always evaluates to true. The
goal was to see if the leaf level slot was greater than zero
(i.e. not the first item in the leaf).

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: don't ignore errors in extent-tree.c
Filipe David Borba Manana [Mon, 8 Jul 2013 13:21:24 +0000 (14:21 +0100)]
Btrfs-progs: don't ignore errors in extent-tree.c

Several function return values were being completely
ignored.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: return error on write failure in make_btrfs()
Filipe David Borba Manana [Thu, 4 Jul 2013 09:48:39 +0000 (10:48 +0100)]
Btrfs-progs: return error on write failure in make_btrfs()

Instead of aborting with a BUG_ON() statement, return a
negated errno code. Also updated mkfs and convert tools
to print a nicer error message when make_btrfs() returns
an error.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add missing write check for mkfs
Filipe David Borba Manana [Wed, 3 Jul 2013 17:30:41 +0000 (18:30 +0100)]
Btrfs-progs: add missing write check for mkfs

Assert that the writes of the device and chunk tree
roots succeed. This verification is currently done
for all other tree roots, however it was missing for
those 2 trees.

Would these tree root writes fail, but all others succeed,
it would lead to a corrupted/incomplete btrfs filesystem,
or, more likely some weird failure later on in mkfs.btrfs
inside open_ctree().

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix possible memory leak related to subvolume/snapshot creation
Wang Shilong [Wed, 3 Jul 2013 16:07:38 +0000 (00:07 +0800)]
Btrfs-progs: fix possible memory leak related to subvolume/snapshot creation

The operation related qgroup inherit may fails, if it fails, we should
free memory allocated,otherwise, memory leak happens.

Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: per-thread, per-call pretty buffer
Zach Brown [Tue, 9 Jul 2013 20:24:43 +0000 (13:24 -0700)]
btrfs-progs: per-thread, per-call pretty buffer

We don't need callers to manage string storage for each pretty_sizes()
call.  We can use a macro to have per-thread and per-call static storage
so that pretty_sizes() can be used as many times as needed in printf()
arguments without requiring a bunch of supporting variables.

This lets us have a natural interface at the cost of requiring __thread
and TLS from gcc and a small amount of static storage.  This seems
better than the current code or doing something with illegible format
specifier macros.

Signed-off-by: Zach Brown <zab@redhat.com>
Acked-by: Wang Shilong <wangs.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove unused code
Filipe David Borba Manana [Wed, 3 Jul 2013 17:32:41 +0000 (18:32 +0100)]
Btrfs-progs: remove unused code

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add kstrdup() return value check
Filipe David Borba Manana [Sun, 30 Jun 2013 11:51:45 +0000 (12:51 +0100)]
Btrfs-progs: add kstrdup() return value check

When allocating a btrfs_device structure, device_list_add()
in volumes.c was not checking if the call to duplicate the
label string succeeded or not.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: Cleanup unneeded extra variant in btrfs_read_sys_array
Qu Wenruo [Thu, 27 Jun 2013 06:25:18 +0000 (14:25 +0800)]
btrfs-progs: Cleanup unneeded extra variant in btrfs_read_sys_array

The btrfs_read_sys_array function uses 3 variants to read data from
super block.

But the three variants are related to each other, so the patch removes
unneeded extra variants and make code a little simpler.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: Cleanup for using BTRFS_SETGET_STACK instead of raw convert
Qu Wenruo [Wed, 26 Jun 2013 05:27:08 +0000 (13:27 +0800)]
btrfs-progs: Cleanup for using BTRFS_SETGET_STACK instead of raw convert

Some codes still use the cpu_to_lexx instead of the
BTRFS_SETGET_STACK_FUNCS declared in ctree.h.

Also added some BTRFS_SETGET_STACK_FUNCS for btrfs_header and
btrfs_super.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: kerncompat.h: remove offsetof redefinition
Cristian Rodríguez [Wed, 26 Jun 2013 04:16:43 +0000 (00:16 -0400)]
btrfs-progs: kerncompat.h: remove offsetof redefinition

Must use the version provided by the compiler in stddef.h header

Signed-off-by: Cristian Rodríguez <crrodriguez@opensuse.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: fix the comment for extent_io.c/clear_extent_bits
Wang Sheng-Hui [Mon, 24 Jun 2013 07:09:15 +0000 (15:09 +0800)]
btrfs-progs: fix the comment for extent_io.c/clear_extent_bits

It should be 'clear', not 'set'.

Signed-off-by: Wang Sheng-Hui <shhuiw@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: update man page for btrfs filesystem label
Guangyu Sun [Mon, 17 Jun 2013 21:45:23 +0000 (14:45 -0700)]
btrfs-progs: update man page for btrfs filesystem label

btrfs filesystem label can work on a mounted filesystem, also on a
multi-devices filesystem. And the restriction of label name is
changed, too. The man page should be updated accordingly.

Signed-off-by: Guangyu Sun <guangyu.sun@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add option to btrfs-debug-tree to print uuid tree only
Stefan Behrens [Wed, 26 Jun 2013 15:17:59 +0000 (17:17 +0200)]
Btrfs-progs: add option to btrfs-debug-tree to print uuid tree only

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add uuid_tree_gen field to btrfs-show-super
Stefan Behrens [Wed, 26 Jun 2013 15:17:58 +0000 (17:17 +0200)]
Btrfs-progs: add uuid_tree_gen field to btrfs-show-super

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: use UUID tree for send/receive
Stefan Behrens [Wed, 26 Jun 2013 15:17:57 +0000 (17:17 +0200)]
Btrfs-progs: use UUID tree for send/receive

This commit changes the btrfs send/receive commands to use the
UUID tree to map UUIDs to subvolumes, and to use the root tree
to map subvolume IDs to paths. Now these tools start fast and are
independent on the number of subvolules/snapshot that exist.

Before this commit, mapping UUIDs to subvolume IDs was an operation
with a high effort. The algorithm even had quadratic effort (based
on the number of existing subvolumes). E.g. with 15,000 subvolumes
it took much more than 5 minutes on a state of the art XEON CPU to
start btrfs send or receive before these tools were able to send or
receive the first byte).
Even linear effort instead of the current quadratic effort would be
too much since it would be a waste. And these data structures to
allow mapping UUIDs to subvolume IDs had been created every time a
btrfs send/receive instance was started.

It is much more efficient to maintain a searchable persistent data
structure in the filesystem, one that is updated whenever a
subvolume/snapshot is created and deleted, and when the received
subvolume UUID is set by the btrfs-receive tool.

Therefore kernel code was added that is able to maintain data
structures in the filesystem that allow to quickly search for a
given UUID and to retrieve data that is assigned to this UUID, like
which subvolume ID is related to this UUID.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add UUID tree lookup methods
Stefan Behrens [Wed, 26 Jun 2013 15:17:56 +0000 (17:17 +0200)]
Btrfs-progs: add UUID tree lookup methods

This commit adds UUID tree lookup methods that make use of the search
ioctl. The code is based on the kernel code.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree
Stefan Behrens [Wed, 26 Jun 2013 15:17:55 +0000 (17:17 +0200)]
Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

Support printing these things.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: commit the csum_root if we do --init-csum-tree
Josef Bacik [Fri, 14 Jun 2013 18:25:54 +0000 (14:25 -0400)]
Btrfs-progs: commit the csum_root if we do --init-csum-tree

This is just an oddity with the commit stuff in btrfs-progs.  It will just
update the generation of the root you call with, which in btrfsck case would
have been the fs_root.  But because we didn't actually update the fs_root we
wouldn't have cow'ed the fs root and therefore the generation will not match the
node which will make the file system unmountable.  Fix this by calling with the
csum_root which is the one we're messing with.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: pretty print dir_item type
Filipe David Borba Manana [Mon, 10 Jun 2013 19:51:34 +0000 (20:51 +0100)]
Btrfs-progs: pretty print dir_item type

Instead of printing an integer, print a symbolic name which
is more human friendly. Particularly useful when using the
program btrfs-debug-tree.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Add missing close_ctree() calls to debug-tree
Filipe David Borba Manana [Mon, 10 Jun 2013 19:51:33 +0000 (20:51 +0100)]
Btrfs-progs: Add missing close_ctree() calls to debug-tree

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Add missing free_extent_buffer() call to debug-tree
Filipe David Borba Manana [Mon, 10 Jun 2013 19:51:32 +0000 (20:51 +0100)]
Btrfs-progs: Add missing free_extent_buffer() call to debug-tree

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix compile warning in btrfs_free_block_groups()
Wang Shilong [Sat, 18 May 2013 16:54:37 +0000 (00:54 +0800)]
Btrfs-progs: fix compile warning in btrfs_free_block_groups()

extent-tree.c: In function 'btrfs_free_block_groups':
extent-tree.c:3190:12: warning: cast to pointer from integer of
 different size [-Wint-to-pointer-cast]

Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: mkfs: add -O option to specify fs features
David Sterba [Thu, 16 May 2013 15:04:04 +0000 (17:04 +0200)]
btrfs-progs: mkfs: add -O option to specify fs features

Extend mkfs options to specify optional or potentially backwards
incompatible features.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: detect when scrub is started twice
Stefan Behrens [Wed, 15 May 2013 13:54:49 +0000 (15:54 +0200)]
Btrfs-progs: detect when scrub is started twice

Check whether any involved device is already busy running a
scrub. This would cause damaged status messages and the state
"aborted" without the explanation that a scrub was already
running. Therefore check it first, prevent it and give some
feedback to the user if scrub is already running.
Note that if scrub is started with a block device as the
parameter, only that particular block device is checked. It
is a normal mode of operation to start scrub on multiple
single devices, there is no reason to prevent this.

Here is an example:
/mnt2 is the mountpoint of a filesystem.
/dev/sdk and /dev/sdl are the block devices for that filesystem.

case 1:
btrfs scrub start /mnt2
btrfs scrub start /mnt2
-> complain

case 1:
btrfs scrub start /dev/sdk
btrfs scrub start /dev/sdk
-> complain

case 3:
btrfs scrub start /dev/sdk
btrfs scrub start /dev/sdl
-> don't complain

case 4:
btrfs scrub start /dev/sdk
btrfs scrub start /mnt2
-> complain

case 5:
btrfs scrub start /mnt2
btrfs scrub start /dev/sdk
-> complain if the scrub on /dev/sdk is still running.
-> don't complain if the scrub on /dev/sdk is finished, the
   status messages will be fine.

Reported-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: restore: use long option for the path regex
David Sterba [Wed, 15 May 2013 22:26:04 +0000 (00:26 +0200)]
btrfs-progs: restore: use long option for the path regex

Current way of specifying the path to match is not very comfortable, but
the feature itself is very useful. Let's save the short option -m for a
more user friendly syntax and keep a long option --path-regex with the
current syntax.

CC: Peter Stuge <peter@stuge.se>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: fix regexec to only work if we actually have a regexec
Josef Bacik [Fri, 2 Dec 2011 17:52:15 +0000 (12:52 -0500)]
btrfs-progs: fix regexec to only work if we actually have a regexec

We were unconditionally executing our regular expression, even though we may not
have one, so check to make sure mreg is not null before calling regexec.
Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agorestore: Add regex matching of paths and files to be restored
Peter Stuge [Fri, 25 Nov 2011 00:03:58 +0000 (01:03 +0100)]
restore: Add regex matching of paths and files to be restored

The option -m is used to specify the regex string. -c is used to
specify case insensitive matching. -i was already taken.

In order to restore only a single folder somewhere in the btrfs
tree, it is unfortunately neccessary to construct a slightly
nontrivial regex, e.g.:

restore -m '^/(|home(|/username(|/Desktop(|/.*))))$' /dev/sdb2 /output

This is needed in order to match each directory along the way to the
Desktop directory, as well as all contents below the Desktop directory.

Signed-off-by: Peter Stuge <peter@stuge.se>
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: pass up return value of walk_down_tree
Lin Ming [Fri, 3 May 2013 03:51:38 +0000 (23:51 -0400)]
btrfs-progs: pass up return value of walk_down_tree

Pass up return value of walk_down_tree, so the caller can handle it.
This also fixes a segfault when read_tree_block fails with NULL returned.

Signed-off-by: Lin Ming <mlin@kernel.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: update manpage to add filesystem show command in synopsis
Eryu Guan [Mon, 13 May 2013 15:23:39 +0000 (23:23 +0800)]
btrfs-progs: update manpage to add filesystem show command in synopsis

filesystem show was missing in SYNOPSIS section.

Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agolibbtrfs: Set SONAME to "libbtrfs.so.0" (instead of "libbtrfs.so").
Richard W.M. Jones [Sun, 12 May 2013 15:33:44 +0000 (16:33 +0100)]
libbtrfs: Set SONAME to "libbtrfs.so.0" (instead of "libbtrfs.so").

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add send option for using new end-cmd semantic
Stefan Behrens [Wed, 10 Apr 2013 17:10:56 +0000 (19:10 +0200)]
Btrfs-progs: add send option for using new end-cmd semantic

This commit adds a command line option to enable sending streams
which make use of the new end-cmd semantic if multiple snapshots are
sent back-to-back. The goal is to use the <end cmd> as an indication
to stop reading the input stream. So far, the receiver could only
use EOF to recognize the end.

If the new command line option '-e' is set, this commit requires a
kernel which is able to support the new flags in the send ioctl. New
bits in the flags of the send ioctl will be set which cause EINVAL
on old kernels. However, if the option '-e' is not set, it works
with old and new kernels without any errors or any changed behavior.

This used to be the encoding (with 2 snapshots in this example):
<stream header> + <sequence of commands> + <end cmd> +
<stream header> + <sequence of commands> + <end cmd> + EOF

The new format (if the two new flags are used) is this one:
<stream header> + <sequence of commands> +
                  <sequence of commands> + <end cmd>

Note that the currently existing receivers treat <end cmd> only as
an indication that a new <stream header> is following. This means,
you can just skip the sequence <end cmd> <stream header> without
loosing compatibility. As long as an EOF is following, the currently
existing receivers handle the new format (if the two new flags are
used) exactly as the old one.

Also note that the kernel interface was changed in a way that is
backward compatible to old btrfs-progs tools. You set one or two bits
in the flags field of the ioctl to enable the new behavior. Old tools
set these flags to zero, thus getting exactly the same as they got
with older kernels. And this is exactly what happens if the new '-e'
option is not set, the new bits in the flags are not set and thus
old kernels and new kernels are both supported.

So what is the benefit of this change? The goal is to be able to use
a single stream (one TCP connection) to multiplex a request/response
handshake plus Btrfs send streams, all in the same stream. In this
case you cannot evaluate an EOF condition as an end of the Btrfs send
stream. You need something else, and the <end cmd> is just perfect
for this purpose.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: make btrfsck a hardlink at install time
Eric Sandeen [Thu, 2 May 2013 16:20:22 +0000 (11:20 -0500)]
Btrfs-progs: make btrfsck a hardlink at install time

btrfsck gets hardlinked to btrfs during the build, but the
install phase simply copies them both to the destination without
preserving the link.

Just force-link btrfsck in the destination again during install
so that the installed btrfsck is a link as well.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove btrfs_init_path calls from ctree.c
Eric Sandeen [Fri, 26 Apr 2013 21:06:10 +0000 (16:06 -0500)]
Btrfs-progs: remove btrfs_init_path calls from ctree.c

btrfs_init_path was initially used when the path objects were on the
stack.  Now all the work is done by btrfs_alloc_path and btrfs_init_path
isn't required.

This patch removes it, and just uses kmem_cache_zalloc to zero out the object.

[Eric Sandeen: port kernel commit e00f730 to userspace]

(Note, the rest of userspace has an on-stack path, so the actual
function remains for now).

Signed-off-by: Chris Mason <chris.mason@oracle.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove the unnecessary 'return -1;' at the end of bin_search
Eric Sandeen [Fri, 26 Apr 2013 21:06:09 +0000 (16:06 -0500)]
Btrfs-progs: remove the unnecessary 'return -1;' at the end of bin_search

The code path should not reach there. Remove it.

[Eric Sandeen: port kernel commit 3fed40c to userspace]

Signed-off-by: Wang Sheng-Hui <shhuiw@gmail.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: make extent_ref_type() match kernelspace
Eric Sandeen [Fri, 26 Apr 2013 21:06:08 +0000 (16:06 -0500)]
Btrfs-progs: make extent_ref_type() match kernelspace

extent_ref_type() contains inconsequential differences between
kernelspace and userspace, and has since the initial commits
to each.  Just make userspace look like kernelspace.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: cleanup duplicated division functions
Eric Sandeen [Fri, 26 Apr 2013 21:06:07 +0000 (16:06 -0500)]
Btrfs-progs: cleanup duplicated division functions

div_factor has been implemented for two times, cleanup it.
And I move them into a independent file named math.h because they are
common math functions.

[Eric Sandeen: port kernel commit 3fed40c to userspace]

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove some dead/unbuilt code
Eric Sandeen [Fri, 26 Apr 2013 21:06:06 +0000 (16:06 -0500)]
Btrfs-progs: remove some dead/unbuilt code

Remove some commented-out & #if 0'd code:

 * close_blocks()
 * btrfs_drop_snapshot()
 * btrfs_realloc_node()
 * btrfs_find_dead_roots()

There are still some #if 0'd functions in there, but I'm hedging
on those for now, they have been copied to cmds-check.c and I want
to see if they can be brough back into ctree.c eventually.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: move btrfs_fsck_reinit_root to cmds-check.c
Eric Sandeen [Fri, 26 Apr 2013 21:06:05 +0000 (16:06 -0500)]
Btrfs-progs: move btrfs_fsck_reinit_root to cmds-check.c

cmds-check.c contains the only caller of btrfs_fsck_reinit_root;
moving it to the caller's source file gets ctree.c a little
closer to kernelspace, although it does require exporting
add_root_to_dirty_list(), which is not done in kernelspace.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix array bound checking
Eric Sandeen [Fri, 26 Apr 2013 21:06:04 +0000 (16:06 -0500)]
Btrfs-progs: fix array bound checking

Otherwise we can execced the array bound of path->slots[].

[Eric Sandeen: port kernel commit a05a9bb to userspace]

Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: remove dead btrfs-defrag.c
Eric Sandeen [Fri, 26 Apr 2013 21:06:02 +0000 (16:06 -0500)]
Btrfs-progs: remove dead btrfs-defrag.c

Files with only #include directives are boring. :)

This is just a leftover after the move to the btrfs tool.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-restore: deal with NULL returns from read_node_slot
Chris Mason [Fri, 5 Jul 2013 17:44:07 +0000 (13:44 -0400)]
btrfs-restore: deal with NULL returns from read_node_slot

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-restore: use the correct leafsize when reading the FS location
Chris Mason [Fri, 5 Jul 2013 16:30:43 +0000 (12:30 -0400)]
btrfs-restore: use the correct leafsize when reading the FS location

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoAdd options to btrfs-find-root to control generation and level
Chris Mason [Fri, 5 Jul 2013 16:11:22 +0000 (12:11 -0400)]
Add options to btrfs-find-root to control generation and level

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfsck: fix incorrect casting on items in the corrupt_blocks tree
Chris Mason [Fri, 5 Jul 2013 15:09:04 +0000 (11:09 -0400)]
btrfsck: fix incorrect casting on items in the corrupt_blocks tree

check_extent_refs is pinning down all the corrupt tree blocks it finds,
but it is incorrectly casting these to an extent_record first.

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix closing of devices
Filipe David Borba Manana [Wed, 26 Jun 2013 16:41:36 +0000 (17:41 +0100)]
Btrfs-progs: fix closing of devices

If a device could not be opened in volumes.c:read_one_dev(), a
btrfs_device instance was allocated and added to the list of
devices of the fs - however this device instance had its fd,
name and label fields not initialized. This is problematic in
disk-io.c:close_all_devices() as it tried to sync, fadvise and
close the (invalid) fd of the device, and kfree() its name and
label, which pointed to random memory locations.

  Thread 1 (Thread 0x7f0a3d2d1740 (LWP 23585)):
  #0  __GI___libc_free (mem=0xa5a5a5a5a5a5a5a5) at malloc.c:2970
  #1  0x000000000042054b in close_all_devices (fs_info=0x1e92bf0) at disk-io.c:1276
  #2  0x0000000000421dcd in close_ctree (root=<optimized out>) at disk-io.c:1336
  #3  0x0000000000418cfa in cmd_check (argc=<optimized out>, argv=<optimized out>) at cmds-check.c:4171
  #4  0x0000000000403ed4 in main (argc=2, argv=0x7fff9a583d28) at btrfs.c:295

v2: Added Liu Bo's review mention.

Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoMerge branch 'liubo-image-restore'
Chris Mason [Wed, 3 Jul 2013 18:24:43 +0000 (14:24 -0400)]
Merge branch 'liubo-image-restore'

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Conflicts:
disk-io.c
volumes.h

11 years agoBtrfs-progs: enhance btrfs-image to restore image onto multiple disks
Liu Bo [Sat, 22 Jun 2013 05:32:45 +0000 (13:32 +0800)]
Btrfs-progs: enhance btrfs-image to restore image onto multiple disks

This adds a 'btrfs-image -m' option, which let us restore an image that
is built from a btrfs of multiple disks onto several disks altogether.

This aims to address the following case,
$ mkfs.btrfs -m raid0 sda sdb
$ btrfs-image sda image.file
$ btrfs-image -r image.file sdc
---------
so we can only restore metadata onto sdc, and another thing is we can
only mount sdc with degraded mode as we don't provide informations of
another disk.  And, it's built as RAID0 and we have only one disk,
so after mount sdc we'll get into readonly mode.

This is just annoying for people(like me) who're trying to restore image
but turn to find they cannot make it work.

So this'll make your life easier, just tap
$ btrfs-image -m image.file sdc sdd
---------
then you get everything about metadata done, the same offset with that of
the originals(of course, you need offer enough disk size, at least the disk
size of the original disks).

Besides, this also works with raid5 and raid6 metadata image.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: cleanup btrfs-image usage
Liu Bo [Sat, 22 Jun 2013 05:32:44 +0000 (13:32 +0800)]
Btrfs-progs: cleanup btrfs-image usage

A '\n' is missing.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: delete fs_devices itself from fs_uuid list before freeing
Liu Bo [Sat, 22 Jun 2013 05:32:43 +0000 (13:32 +0800)]
Btrfs-progs: delete fs_devices itself from fs_uuid list before freeing

Otherwise we will access illegal addresses while searching on fs_uuid list.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: skip open devices which is missing
Liu Bo [Sat, 22 Jun 2013 05:32:42 +0000 (13:32 +0800)]
Btrfs-progs: skip open devices which is missing

A device can be added to the device list without getting a name, so we may
access to illegal addresses while opening devices with their name.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix misuse of skinny metadata in btrfs-image
Liu Bo [Sat, 22 Jun 2013 05:32:41 +0000 (13:32 +0800)]
Btrfs-progs: fix misuse of skinny metadata in btrfs-image

As for skinny metadata, key.offset stores levels rather than extent length.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: recover raid0/raid10/raid5/raid6 metadata chunk
Miao Xie [Wed, 3 Jul 2013 13:25:20 +0000 (21:25 +0800)]
Btrfs-progs: recover raid0/raid10/raid5/raid6 metadata chunk

According to the bytenr of the extent buffer record, we can calculate the index
of the stripes, and we also know which device and where we read out the extent
buffer record, that means we can know the relationship between the device extent
and the stripes in the chunk, by this relationship, we can recover the raid0/radi10/
raid5/raid6 metadata chunk.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Add chunk rebuild function for RAID1/SINGLE/DUP
Miao Xie [Wed, 3 Jul 2013 13:25:19 +0000 (21:25 +0800)]
Btrfs-progs: Add chunk rebuild function for RAID1/SINGLE/DUP

Add chunk rebuild for RAID1/SINGLE/DUP to chunk-recover command.

Before this patch chunk-recover can only scan and reuse the old chunk
data to recover. With this patch, chunk-recover can use the reference
between chunk/block group/dev extent to rebuild the whole chunk tree
even when old chunks are not available.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: introduce list_{first, next}_entry/list_splice_tail{_init}
Miao Xie [Wed, 3 Jul 2013 13:25:18 +0000 (21:25 +0800)]
Btrfs-progs: introduce list_{first, next}_entry/list_splice_tail{_init}

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Add chunk recover function - using old chunk items
Miao Xie [Wed, 3 Jul 2013 13:25:17 +0000 (21:25 +0800)]
Btrfs-progs: Add chunk recover function - using old chunk items

Add chunk-recover program to check or rebuild chunk tree when the system
chunk array or chunk tree is broken.

Due to the importance of the system chunk array and chunk tree, if one of
them is broken, the whole btrfs will be broken even other data are OK.

But we have some hint(fsid, checksum...) to salvage the old metadata.
So this function will first scan the whole file system and collect the
needed data(chunk/block group/dev extent), and check for the references
between them. If the references are OK, the chunk tree can be rebuilt and
luckily the file system will be mountable.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Add block group check funtion
Miao Xie [Wed, 3 Jul 2013 13:25:16 +0000 (21:25 +0800)]
Btrfs-progs: Add block group check funtion

This patch adds the function to check correspondence between block group,
chunk and device extent.

Original-signed-off-by: Cheng Yang <chenyang.fnst@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: extend the extent cache for the device extent
Miao Xie [Wed, 3 Jul 2013 13:25:15 +0000 (21:25 +0800)]
Btrfs-progs: extend the extent cache for the device extent

As we know, btrfs can manage several devices in the same fs, so [offset, size]
is not sufficient for unique identification of an device extent, we need the
device id to identify the device extents which have the same offset and size,
but are not in the same device. So, we added a member variant named objectid
into the extent cache, and introduced some functions to make the extent cache
be suitable to manage the device extent.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: use rb-tree instead of extent cache tree for fs/file roots
Miao Xie [Wed, 3 Jul 2013 13:25:14 +0000 (21:25 +0800)]
Btrfs-progs: use rb-tree instead of extent cache tree for fs/file roots

Because the fs/file roots are not extents, so it is better to use rb-tree
to manage them. Fix it.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: introduce common insert/search/delete functions for rb-tree
Miao Xie [Wed, 3 Jul 2013 13:25:13 +0000 (21:25 +0800)]
Btrfs-progs: introduce common insert/search/delete functions for rb-tree

In fact, the code of many rb-tree insert/search/delete functions is similar,
so we can abstract them, and implement common functions for rb-tree, and then
simplify them.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: cleanup similar code in open_ctree_* and close_ctree
Miao Xie [Wed, 3 Jul 2013 13:25:12 +0000 (21:25 +0800)]
Btrfs-progs: cleanup similar code in open_ctree_* and close_ctree

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: Don't free the devices when close the ctree
Miao Xie [Wed, 3 Jul 2013 13:25:11 +0000 (21:25 +0800)]
Btrfs-progs: Don't free the devices when close the ctree

Some commands(such as btrfs-convert) access the devices again after we close
the ctree, so it is better that we don't free the devices objects when the ctree
is closed, or we need re-allocate the memory for the devices. We needn't worry
the memory leak problem, because all the memory will be freed after the taskes
die.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: don't close the file descriptor 0 when closing a device
Miao Xie [Wed, 3 Jul 2013 13:25:10 +0000 (21:25 +0800)]
Btrfs-progs: don't close the file descriptor 0 when closing a device

As we know, the file descriptor 0 is a special number, so we shouldn't
use it to initialize the file descriptor of the devices, or we might
close this special file descriptor by mistake when we close the devices.
"-1" is a better choice.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix missing recow roots when making btrfs filesystem
Wang Shilong [Wed, 3 Jul 2013 13:25:09 +0000 (21:25 +0800)]
Btrfs-progs: fix missing recow roots when making btrfs filesystem

When making btrfs filesystem. we firstly write root leaf to
specified filed, and then we recow the root. If we don't recow,
some trees are not in the correct block group.

Steps to reproduce:
dd if=/dev/zero of=test.img bs=1M count=100
mkfs.btrfs -f test.img
btrfs-debug-tree test.img

extent tree key (EXTENT_TREE ROOT_ITEM 0)
leaf 4210688 items 10 free space 3349 generation 4 owner 2
fs uuid 2e08fd93-f24d-4f44-a226-e2116fcd544f
chunk uuid dc482988-6246-46ce-9329-68bcf6d3683c
item 0 key (0 BLOCK_GROUP_ITEM 4194304) itemoff 3971 itemsize 24
block group used 12288 chunk_objectid 256 flags 2
[..snip..]
item 3 key (1138688 EXTENT_ITEM 4096) itemoff 3827 itemsize 42
extent refs 1 gen 1 flags 2
tree block key (0 UNKNOWN.0 0) level 0
item 4 key (1138688 TREE_BLOCK_REF 7) itemoff 3827 itemsize 0
tree block backref
[..snip..]

checksum tree key (CSUM_TREE ROOT_ITEM 0)
leaf 1138688 items 0 free space 3995 generation 1 owner 7
fs uuid 2e08fd93-f24d-4f44-a226-e2116fcd544f
chunk uuid dc482988-6246-46ce-9329-68bcf6d3683c

For the above example, csum root leaf comes into system block group which
is wrong,csum root leaf should be in metadata block group.

Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Reviewed-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix reference check for roots in btrfsck
Josef Bacik [Wed, 5 Jun 2013 14:02:59 +0000 (10:02 -0400)]
Btrfs-progs: fix reference check for roots in btrfsck

I noticed that I was getting these errors on a bigger file system with more
snapshots that had been removed.  This check is bogus since we won't inc
rec->found_ref if we don't find a REF_KEY _and_ a DIR_ITEM, so we only have to
worry about there being no references to a root if it actually has a root item.
If it doesn't then it's just referenced by things that will go no where anyway.
With this patch fsck no longer incorrectly complains about this file system
image I have.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix incorrect root backref errors in fsck
Josef Bacik [Tue, 4 Jun 2013 20:54:52 +0000 (16:54 -0400)]
Btrfs-progs: fix incorrect root backref errors in fsck

A user reported that fsck was complaining about unresolved refs for some
snapshots.  You can reproduce this by doing

mkfs.btrfs /dev/sdb
mount /dev/sdb /mnt
btrfs subvol snap /mnt/ /mnt/a
btrfs subvol snap /mnt/ /mnt/b
btrfs subvol del /mnt/a
umount /mnt
btrfsck /dev/sdb

and you'd get this

unresolved ref root 258 dir 256 index 2 namelen 1 name a error 600

because snapshot b has a dir item that points to a.  Except we encode in our
root ref the dirid of the ref holder, and if it doesn't match we just give it
back a empty directory since we can't hardlink directories.  This makes the
check in btrfsck bogus, when we delete a we remove the ref key for it so any
lookups into /mnt/b/a will just give a blank directory as it's supposed to.  Fix
this by only saying the backref is reachable if there is both a DIR_ITEM and a
REF_KEY for the given root.  With this patch I no longer see errors when running
this reproducer.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix fsck dealing with finding backrefs first
Josef Bacik [Mon, 3 Jun 2013 19:15:28 +0000 (15:15 -0400)]
Btrfs-progs: fix fsck dealing with finding backrefs first

There is a problem where if we find a backref extent record first that doesn't
match a extent item we will delete some of the duplicates but not others.  In
order to deal with this we need to make sure we only pay attention to duplicates
that actually have duplicate extent items.  If a extent_rec has a duplicate but
the record itself doesn't have an associated extent item we promote the
duplicate to the extent record and just discard the original extent_rec since it
was just added by the backref.  We copy the backref onto the promoted extent
record and then continue processing.  This allowed me to fix a file system that
previously was not able to be fixed by fsck.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix the allocator
Josef Bacik [Mon, 3 Jun 2013 19:13:05 +0000 (15:13 -0400)]
Btrfs-progs: fix the allocator

The allocator looks for these hints when moving on to another block group which
will make it reset which block group it looks at, when we've already searched
that block group and didn't find any space to allocate, we need to fix this by
just letting the allocator make the determination if the block group is good
enough.  This also fixes a problem where if we couldn't find space in the block
group we were given we'd just error out instead of moving on to the next block
group.  Previously I couldn't fix some file systems that were relatively full,
but with this patch I can now run fsck on them with no allocation errors.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix free space cache checks
Josef Bacik [Wed, 29 May 2013 13:08:12 +0000 (09:08 -0400)]
Btrfs-progs: fix free space cache checks

This fixes two bugs with the free space cache checker.  First is we apparently
always use root->sectorsize for our unit in the kernel so we have to do that in
progs otherwise bitmaps turn out to not look right if we have leafsize !=
sectorsize.  The second is a small issue if we had skinny metadata extents set,
we wouldn't advance last properly because we unconditionally use key.offset
instead of root->leafsize.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add --init-extent-tree to btrfsck
Josef Bacik [Fri, 17 May 2013 13:34:23 +0000 (09:34 -0400)]
Btrfs-progs: add --init-extent-tree to btrfsck

In some cases the extent tree can just be so gone there is no point in trying to
figure out how to put it back together.  So add a --init-extent-tree mode which
will zero out the extent tree and then re-add extents for all of the blocks we
find.  This will also undo any balance that was going on at the time of the
crash, this is needed because the reloc tree seems to confuse fsck at the
moment.  With this patch I can put back together a users file system that was
completely gone.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add a newline to a free space cache message
Josef Bacik [Wed, 15 May 2013 17:56:42 +0000 (13:56 -0400)]
Btrfs-progs: add a newline to a free space cache message

Left out a newline in the generation check printf.

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: make image restore with the original device offsets
Josef Bacik [Wed, 8 May 2013 15:40:36 +0000 (11:40 -0400)]
Btrfs-progs: make image restore with the original device offsets

I noticed a slight problem with btrfs-image, since it was building a chunk tree
by setting the physical offset of the stripes to the same as the logical offset
it created this problem where the super block was now mapped into the file
system differently than it was before.  This isn't a huge deal except that we
also carry along the free space cache with us, which is setup with the idea that
super at physical X is at logical Y.  So this would make the free space checker
in fsck freak out because it would see that the cache says that the super block
is free space, and that the area where it thought the super block was located is
in fact used.  In the mount case we'd end up overwriting real metadata with
backup super blocks.  So we need to maintain the physical offsets in our
stripes.  This is a huge pain because we store the logical bytenrs of all of our
metadata.  This patch scans the entire image looking for chunk tree blocks and
builds an in memory chunk tree so we can write logical blocks to their physical
offsets.  With this patch we no longer have the problems I described above.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: pass properly formated key to read_fs_root
Josef Bacik [Thu, 2 May 2013 13:15:22 +0000 (09:15 -0400)]
Btrfs-progs: pass properly formated key to read_fs_root

We have a BUG_ON() in read_fs_root that expects key->offset == (u64)-1.  Restore
will just pass in the location it reads out of the inode if it has a subvol
reference and pass that in, which doesn't have offset == (u64)-1, so it causes
restore blow up.  This was reported in bugzilla and this patch fixed the
problem.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: make fsck fix certain file extent inconsistencies
Josef Bacik [Tue, 14 May 2013 18:44:22 +0000 (14:44 -0400)]
Btrfs-progs: make fsck fix certain file extent inconsistencies

The tree log bug I introduced could create inconsistent file extent entries in
the file system tree and in some worst cases even create multiple extent entries
for the same entry.  To fix this we need to do a few things

1) Keep track of extent items that overlap and then pick the one that covers the
largest area and delete the rest of the items.

2) Keep track of file extent items that land in extent items but don't match
disk_bytenr/disk_num_bytes exactly.  Once we find these we need to figure out
who is the right ref and then fix all of the other refs to agree.

Each of these cases require a complete rescan of all of the extents, so
unfortunately if you hit this particular problem the fsck is going to take quite
a while since it will likely rescan all the trees 2 or 3 times.  With this patch
the broken file system a user sent me is fixed and a broken file system that was
created by my reproducer is also fixed.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-image: add the ability to santize file names when making an image
Josef Bacik [Fri, 22 Mar 2013 14:52:07 +0000 (10:52 -0400)]
Btrfs-image: add the ability to santize file names when making an image

We've had a few users who wouldn't (or couldn't) provide us btrfs-images because
we maintain the file names when making an image.  So introduce a sanitize
option.  There are two uses, one that is fast and the other that is dog slow.
The fast way just generates garbage that's equal in length to the original name.
The slow way will try and find a crc32c collision for the file name that is also
the same length.  Finding a crc32c collision for the file name "btrfs-progs" on
my box without CPU crc32c support takes a little more than 3 minutes, and a
little less than 2 minutes for my box that has CPU crc32c support, so it's a
lengthy and CPU intensive process.

The idea is that we use -s for most cases, and then only use -ss when we need
the file system tree to be somewhat sane.  I could probably do a better job
about finding collisions, but I'll have to revist that later.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: add the ability to find mismmatching backrefs
Josef Bacik [Thu, 9 May 2013 14:09:55 +0000 (10:09 -0400)]
Btrfs-progs: add the ability to find mismmatching backrefs

An unfortunate side effect to my fsync bug means that anybody who didn't hit the
BUG_ON() during tree log replay would have ended up with a corrupted file
system.  Currently our fsck does not catch this because it just looks for
bytenrs for backrefs, it doesn't look at the num_bytes at all.  So this patch
makes us keep track of how big the backrefs are, since their disk_num_bytes
_have_ to match the number of bytes for the actual extent item.  With this patch
fsck now finds problems with a file system it previously thought was ok.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
11 years agoBtrfs-progs: sanity check the number of items in a leaf V2
Josef Bacik [Thu, 9 May 2013 13:56:19 +0000 (13:56 +0000)]
Btrfs-progs: sanity check the number of items in a leaf V2

I hit this while working on fsck, I got some weird corruption where the number
of items was way higher than what would fit in a leaf, which would make things
blow up.  This fixes the problem by catching it and returning an error so we
gracefully exit instead of segfaulting.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fix compile warnings in i386 machine
Wang Shilong [Tue, 7 May 2013 12:24:40 +0000 (12:24 +0000)]
Btrfs-progs: fix compile warnings in i386 machine

See the warnings below:

   [CC]     btrfs-list.o
btrfs-list.c: In function 'filter_by_parent':
btrfs-list.c:1183:34: warning: cast to pointer from integer of different size
[-Wint-to-pointer-cast]
     [CC]     cmds-subvolume.o
cmds-subvolume.c: In function 'cmd_subvol_show':
cmds-subvolume.c:917:5: warning: cast from pointer to integer of different size
[-Wpointer-to-int-cast]
     [CC]     cmds-restore.o
cmds-restore.c: In function 'decompress_lzo':
cmds-restore.c:116:10: warning: passing argument 4 of 'lzo1x_decompress_safe'
from incompatible pointer type [enabled by default]
In file included from cmds-restore.c:31:0:
/usr/include/lzo/lzo1x.h:77:1: note: expected 'lzo_uint *' but argument is of
type 'size_t *'

Reported-by: Russell Coker <russell@coker.com.au>
Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: fix typecast when printing csum value
David Sterba [Mon, 6 May 2013 22:01:51 +0000 (22:01 +0000)]
btrfs-progs: fix typecast when printing csum value

Only the first byte of the wanted csum is printed:

checksum verify failed on 65536 found DA97CF61 wanted 6B
checksum verify failed on 65536 found DA97CF61 wanted 6BC3870D

Also add leading zeros to the format.

Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: fixup: add flags to struct btrfs_ioctl_quota_rescan_args
Jan Schmidt [Mon, 6 May 2013 19:15:01 +0000 (19:15 +0000)]
Btrfs-progs: fixup: add flags to struct btrfs_ioctl_quota_rescan_args

The patch set previously sent was sent together with the kernel part, but
was not updated as I added some reserved bytes to the ioctl struct for
future compatibility. This fixes struct btrfs_ioctl_quota_rescan_args.

Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agoBtrfs-progs: init free space ctl with proper unit
Josef Bacik [Fri, 3 May 2013 21:00:52 +0000 (21:00 +0000)]
Btrfs-progs: init free space ctl with proper unit

btrfsck was blowing up when checking the free space cache when we ran xfstests
with -l 64k.  That is because I was init'ing the free space ctl to whatever the
leafsize was, which isn't right for data block groups.  With this patch btrfsck
no longer complains.  This also fixes a tiny little typo in free-space-cache.c I
noticed while figuring this problem out.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-image: fix reading the super block on big metadata blocks
Chris Mason [Fri, 3 May 2013 20:18:46 +0000 (16:18 -0400)]
btrfs-image: fix reading the super block on big metadata blocks

btrfs-image is assuming that every metadata block is the same size.  The
super is a special snowflake though.  It's 4K even on 64K filesystems.

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: mkfs seg fault for wrong free
Anand Jain [Fri, 26 Apr 2013 02:44:45 +0000 (10:44 +0800)]
btrfs-progs: mkfs seg fault for wrong free

With commit
        87c09f7 Btrfs-progs: fix memory leaks on cleanup

mkfs on multiple dev is ending with segfault at
close_all_devices() during kfree(device->name)

because mkfs calls btrfs_add_to_fsid, which does not initialize
name when dev is added to the list.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
11 years agobtrfs-progs: Add default rules to Makefile
Eric Sandeen [Sun, 21 Apr 2013 22:22:37 +0000 (17:22 -0500)]
btrfs-progs: Add default rules to Makefile

Add a default rule for any btrfs-$FOO or btrfs-$FOO.static
target, allowing it to be built from btrfs-$FOO.c along with
all the normal userspace objects.

This gets rid of a lot of the cut and pasted rules for
each individual command, and as an added bonus makes it
easy to build any btrfs-$FOO statically as well, i.e.

  # make btrfs-convert.static

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
11 years agobtrfs-progs: standardize tool source filenames
Eric Sandeen [Tue, 23 Apr 2013 20:57:25 +0000 (15:57 -0500)]
btrfs-progs: standardize tool source filenames

For any btrfs-$FOO executable, rename the main source file from
$FOO.c to to btrfs-$FOO.c

This makes it slightly more obvious what's building what,
and allows us to write a default rule in the Makefile for
these tools.

(also add btrfs-calc-size to the list of objects to remove
on make clean)

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
11 years agoBtrfs-progs: add function to map subvol ID to path
Stefan Behrens [Tue, 23 Apr 2013 10:18:55 +0000 (12:18 +0200)]
Btrfs-progs: add function to map subvol ID to path

Several tools like btrfs-send and btrfs-receive need to map a
subvolume ID to a filesystem path. The so far existing methods
in btrfs-list.c cause a horrible effort when performing this
operation (and the effort is dependent on the number of
existing subvolumes with quadratic effort). This commit adds a
function that is able to map a subvolume ID to a filesystem path
with an effort that is independent of the number of existing
subvolumes.

In addition to this function, a command line frontend is added as well:
btrfs inspect-internal subvolid-resolve <subvolid> <path>

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
11 years agoBtrfs-progs: in btrfs-debug-tree, print -1 in key for (u64)-1
Stefan Behrens [Tue, 23 Apr 2013 09:25:01 +0000 (11:25 +0200)]
Btrfs-progs: in btrfs-debug-tree, print -1 in key for (u64)-1

For the objectid and offset field of a key, print -1 instead of the
decimal representation of 0xffffffffffffffff. At least for me it is
more readable like this.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
11 years agoBtrfs-progs: btrfs-send: free used memory and close fds
Stefan Behrens [Tue, 23 Apr 2013 09:25:00 +0000 (11:25 +0200)]
Btrfs-progs: btrfs-send: free used memory and close fds

Not important at all since exit() is called afterwards and this is
not part of the library. It just makes valgrind happy and thus
allows to search for real flaws.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
11 years agoBtrfs-progs: btrfs-crc: support specifying checksum in hex
Stefan Behrens [Mon, 22 Apr 2013 10:17:51 +0000 (12:17 +0200)]
Btrfs-progs: btrfs-crc: support specifying checksum in hex

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
11 years agobtrfs-progs: update generation_v2 in btrfs_update_root
Eric Sandeen [Mon, 22 Apr 2013 05:16:41 +0000 (00:16 -0500)]
btrfs-progs: update generation_v2 in btrfs_update_root

This addresses the same issue as did:

2bd1169 btrfs-progs: root_item generation_v2 is out of sync after btrfsck

but rather than optionally updating generation_v2 based
on the size of the existing item, increase the size of the
item as needed, and unconditionally set generation_v2.
This matches the kernel code, and keeping things in sync is a
Good Thing.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>