platform/upstream/btrfs-progs.git
7 years agobtrfs-progs: set fixed with for printing uuids of subvolumes
David Sterba [Mon, 2 Oct 2017 14:00:44 +0000 (16:00 +0200)]
btrfs-progs: set fixed with for printing uuids of subvolumes

The tabular output format looks better if the empty uuids are aligned
with the other. In the list output (now default) it's not that nice but
the whole list format is not nice anyway.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: subvol: group options in help
David Sterba [Mon, 2 Oct 2017 13:00:43 +0000 (15:00 +0200)]
btrfs-progs: subvol: group options in help

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: docs: update btrfs-balance
David Sterba [Fri, 29 Sep 2017 20:29:38 +0000 (22:29 +0200)]
btrfs-progs: docs: update btrfs-balance

Copy unexpected edits from wiki so they do not get lost at next
git->wiki sync.

Author: Bill S.
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: backref: use separate list for indirect refs
Jeff Mahoney [Tue, 25 Jul 2017 20:51:38 +0000 (16:51 -0400)]
btrfs-progs: backref: use separate list for indirect refs

Rather than iterate over all outstanding backrefs to resolve indirect refs,
use a separate list that only contains indirect refs.

When we process missing keys, the ref moves to the indirect ref list.
Once the indirect ref is resolved, move the ref to the pending list.

Eventually these lists will be replaced by rbtrees.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ added assertion fix from Josef ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: backref: use separate list for missing keys
Jeff Mahoney [Tue, 25 Jul 2017 20:51:37 +0000 (16:51 -0400)]
btrfs-progs: backref: use separate list for missing keys

Rather than iterate over all outstanding backrefs to resolve missing keys,
use a separate list that only contains refs that need missing keys resolved.

Once the missing key is resolved, move the ref to the pending list.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: backref: add list_first_pref helper
Jeff Mahoney [Tue, 25 Jul 2017 20:51:36 +0000 (16:51 -0400)]
btrfs-progs: backref: add list_first_pref helper

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: backref: push state tracking into a helper structure
Jeff Mahoney [Tue, 25 Jul 2017 20:51:35 +0000 (16:51 -0400)]
btrfs-progs: backref: push state tracking into a helper structure

Eventually, we'll have several lists and trees, as well as some statistics.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: extent-cache: actually cache extent buffers
Jeff Mahoney [Tue, 25 Jul 2017 20:51:34 +0000 (16:51 -0400)]
btrfs-progs: extent-cache: actually cache extent buffers

We have the infrastructure to cache extent buffers but we don't actually
do the caching.  As soon as the last reference is dropped, the buffer
is dropped.  This patch keeps the extent buffers around until the max
cache size is reached (defaults to 25% of memory) and then it drops
the last 10% of the LRU to free up cache space for reallocation.  The
cache size is configurable (for use by e.g. lowmem) when the cache is
initialized.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ update codingstyle, switch total_memory to bytes ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: switch to iterating over the backref_tree
Jeff Mahoney [Tue, 25 Jul 2017 20:51:33 +0000 (16:51 -0400)]
btrfs-progs: check: switch to iterating over the backref_tree

We now have two data structures that can be used to iterate the same data
set, and there may be quite a few of them in memory.  Eliminating the
list_head member will reduce memory consumption while iterating over
the extent backrefs.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: supplement extent backref list with rbtree
Jeff Mahoney [Tue, 25 Jul 2017 20:51:32 +0000 (16:51 -0400)]
btrfs-progs: check: supplement extent backref list with rbtree

For the pathlogical case, like xfstests generic/297 that creates a
large file consisting of one, repeating reflinked extent, fsck can
take hours.  The root cause is that calling find_data_backref while
iterating the extent records is an O(n^2) algorithm.  For my
example test run, n was 2*2^20 and fsck was at 8 hours and counting.

This patch supplements the list with an rbtree and drops the runtime
of that testcase to about 20 seconds.

A previous version of this patch introduced a regression that would
have corrupted file systems during repair.  It was traced to the
compare algorithm honoring ->bytes regardless of whether the
reference had been found and a failure to reinsert nodes after
the target reference was found.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: remove unused function
David Sterba [Fri, 29 Sep 2017 13:24:54 +0000 (15:24 +0200)]
btrfs-progs: mkfs: remove unused function

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: use existing helper for path concatenation
David Sterba [Fri, 29 Sep 2017 13:23:35 +0000 (15:23 +0200)]
btrfs-progs: mkfs: use existing helper for path concatenation

Mkfs uses make_path that is duplicate of path_cat* functions, so we can
switch to them and add the error handling.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: add a parameter to btrfs_mksubvol
Yingyi Luo [Fri, 15 Sep 2017 18:17:31 +0000 (11:17 -0700)]
btrfs-progs: add a parameter to btrfs_mksubvol

A convert parameter is added as a flag to indicate if btrfs_mksubvol()
is used for btrfs-convert. The change cascades down to the callchain.

Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: convert: move link_subvol out of main
Yingyi Luo [Fri, 15 Sep 2017 17:49:21 +0000 (10:49 -0700)]
btrfs-progs: convert: move link_subvol out of main

link_subvol() is moved to inode.c and renamed as btrfs_mksubvol().
The change cascades down to the callchain.

Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: subvol: fix subvol del --commit-after
Misono, Tomohiro [Wed, 27 Sep 2017 02:03:48 +0000 (11:03 +0900)]
btrfs-progs: subvol: fix subvol del --commit-after

Fix 'subvolume delete --commit-after' to work properly:
- SYNC ioctl will be issued even when last delete fails
- SYNC ioctl will be issued on each file system only once in the end

To achieve this, get_fsid() and add_seen_fsid() are called after each
delete to keep only one fd for each fs.

In the end, seen_fsid_hash will be traversed and SYNC is issued on each
fs.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: change seen_fsid to hold fd and DIR
Misono, Tomohiro [Wed, 27 Sep 2017 02:02:47 +0000 (11:02 +0900)]
btrfs-progs: change seen_fsid to hold fd and DIR

Change seen_fsid to hold fd and DIR in order to keep access to each fs.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: move seen_fsid to utils.c
Misono, Tomohiro [Wed, 27 Sep 2017 02:02:19 +0000 (11:02 +0900)]
btrfs-progs: move seen_fsid to utils.c

Move is_seen_fsid()/add_seen_fsid()/free_seen_fsid() to common functions.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: move get_fsid() to utils.c
Misono, Tomohiro [Wed, 27 Sep 2017 02:01:43 +0000 (11:01 +0900)]
btrfs-progs: move get_fsid() to utils.c

Make get_fsid() to a common functions.
This will be used for 'subvol delete --commit-after'.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: subvol: exchange subvol del --commit-after and --commit-each
Misono, Tomohiro [Wed, 27 Sep 2017 02:01:09 +0000 (11:01 +0900)]
btrfs-progs: subvol: exchange subvol del --commit-after and --commit-each

Current code is reversed in --commit-after and --commit-each operation,
i.e. --commit-after means --commit-each actually. This patch fixes this
and also introduces enum type for more readable code.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs progs v4.13.1
David Sterba [Mon, 25 Sep 2017 16:25:18 +0000 (18:25 +0200)]
Btrfs progs v4.13.1

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update CHANGES for v4.13.1
David Sterba [Fri, 28 Jul 2017 13:29:46 +0000 (15:29 +0200)]
btrfs-progs: update CHANGES for v4.13.1

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: check there are no unprintable characters in btrfs-image -ss...
David Sterba [Mon, 25 Sep 2017 13:09:21 +0000 (15:09 +0200)]
btrfs-progs: tests: check there are no unprintable characters in btrfs-image -ss output

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: add more sparse warning checks
David Sterba [Sat, 23 Sep 2017 20:15:42 +0000 (22:15 +0200)]
btrfs-progs: build: add more sparse warning checks

- declarations after statements (style issue)
- bitfields without defined signedness

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: use -std=gnu89 for sparse
David Sterba [Sat, 23 Sep 2017 19:53:26 +0000 (21:53 +0200)]
btrfs-progs: build: use -std=gnu89 for sparse

Sparse does not seem to support gnu90 that we'd like to keep for the
regular build. Use gnu89 for C=1 build.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: add missing defines for the C=1 build
David Sterba [Sat, 23 Sep 2017 19:49:11 +0000 (21:49 +0200)]
btrfs-progs: build: add missing defines for the C=1 build

Sparse complains that BTRFSCONVERT_EXT2 and _RESIERFS are not defined.
The per-target options are missing from the default CFLAGS and only
added in the specific rule.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: updated and add missing function attributes to the definition
David Sterba [Sat, 23 Sep 2017 19:08:43 +0000 (21:08 +0200)]
btrfs-progs: updated and add missing function attributes to the definition

Reported by sparse. Make the prototype match the definition and put the
attributes before the function.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: free-space-cache: fix endianity when reading from disk_key
David Sterba [Sat, 23 Sep 2017 19:00:41 +0000 (21:00 +0200)]
btrfs-progs: free-space-cache: fix endianity when reading from disk_key

Sparse reports:

free-space-cache.c:921:13: warning: incorrect type in assignment (different base types)
free-space-cache.c:921:13:    expected unsigned long long [unsigned] [usertype] ino
free-space-cache.c:921:13:    got restricted __le64 [addressable] [usertype] objectid

we need to use the helper to read objectid, as read_eb_member just
copies the data without disk->cpu transformation. This could lead to
bogus results on bigendian machines.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: use proper helper for reading offset
David Sterba [Sat, 23 Sep 2017 18:45:24 +0000 (20:45 +0200)]
btrfs-progs: print-tree: use proper helper for reading offset

Sparse warns

print-tree.c:1261:49: warning: incorrect type in argument 4 (different base types)
print-tree.c:1261:49:    expected unsigned long long [unsigned] [usertype] start
print-tree.c:1261:49:    got restricted __le64 [addressable] [usertype] offset

which means we have to use the helper instead of directly accessing the offset
member. This could print bogus number on bigendian arch.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: fix debugging macro checks
David Sterba [Sat, 23 Sep 2017 18:44:02 +0000 (20:44 +0200)]
btrfs-progs: fix debugging macro checks

Sparse warns that the DEBUG_*_ERROR macros are not defined when checked.
Use the ifdef check rather then relying on the implicit behaviour.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: Remove misleading BCP 78 boilerplate from SHA implementation
Nicholas D Steeves [Sun, 17 Sep 2017 23:52:27 +0000 (19:52 -0400)]
btrfs-progs: tests: Remove misleading BCP 78 boilerplate from SHA implementation

BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD.

This causes the following lintian error when building on Debian and
Debian derivatives:

    E: btrfs-progs source: license-problem-non-free-RFC-BCP78
       tests/sha224-256.c

Please consult the following email from debian-legal@lists.debian.org
for more information:

https://lists.debian.org/debian-legal/2017/08/msg00004.html
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
[ I've copied too much from the RFC that's not related to the code,
  covered by the explicit copyright notice in the file ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: Add required IETF Trust copyright to SHA implementation
Nicholas D Steeves [Sun, 17 Sep 2017 23:52:26 +0000 (19:52 -0400)]
btrfs-progs: tests: Add required IETF Trust copyright to SHA implementation

Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: docs: start ioctl documentation manual page
David Sterba [Mon, 11 Sep 2017 18:26:04 +0000 (20:26 +0200)]
btrfs-progs: docs: start ioctl documentation manual page

Start documenting the ioctl interface to btrfs. The overall structure
should be settled, the formatting of the ioctl description may change in
the future, newly added ioctl descriptions should follow the examples of
BTRFS_IOC_SUBVOL_CREATE.

The document is not finished yet and will not be installed until most of
ioctls' details are filled in.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: make sure _is_file_or_command does not get confused
David Sterba [Fri, 15 Sep 2017 14:18:00 +0000 (16:18 +0200)]
btrfs-progs: tests: make sure _is_file_or_command does not get confused

The test cli/007-check-force reports something like:

$ type -p '--string that starts with dashes'
bash: type: --: invalid option

Add the option/argument separator.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Fix one-byte overlap bug in free_block_group_cache
Qu Wenruo [Mon, 11 Sep 2017 06:36:07 +0000 (15:36 +0900)]
btrfs-progs: Fix one-byte overlap bug in free_block_group_cache

free_block_group_cache() calls clear_extent_bits() with wrong end, which
is one byte larger than the correct range.

This will cause the next adjacent cache state to be split.  And due to
the split, private pointer (which points to block group cache) will be
reset to NULL.

This is very hard to detect as this function only gets called in
cleanup_temp_chunks() which is just before mkfs finishes.  This bug only
gets exposed when reworking --rootdir option.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Refactor find_next_chunk to get rid of parameter root and objectid
Qu Wenruo [Mon, 11 Sep 2017 06:36:06 +0000 (15:36 +0900)]
btrfs-progs: Refactor find_next_chunk to get rid of parameter root and objectid

Function find_next_chunk() is used to find next chunk start position,
which should only do search on chunk tree and objectid is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

So refactor the parameter list to get rid of @root, which should be
obtained from fs_info->chunk_root, and @objectid, which is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: omit unnecessary -MD flag
Naohiro Aota [Thu, 14 Sep 2017 10:10:56 +0000 (19:10 +0900)]
btrfs-progs: build: omit unnecessary -MD flag

According to gcc(1), "-MD is equivalent to -M -MF file, except that -E is not
implied." Since the rule in the Makefile is just generating dependency file
and not building object file, it is no use to have "-MD" here. Also, it's
overridden and conflicting with the following "-MM" flag. I guess we can drop
it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: generate all dependency files
Naohiro Aota [Thu, 14 Sep 2017 10:10:46 +0000 (19:10 +0900)]
btrfs-progs: build: generate all dependency files

We're missing several dependency files like:

$ diff -u <(find -name '*.o'|cut -d. -f2|sort) <(find -name '*.o.d'|cut -d. -f2|sort)
@@ -3,7 +3,6 @@
 /btrfs-corrupt-block
 /btrfs-debug-tree
 /btrfs-find-root
-/btrfs-list
 /btrfs-map-logical
 /btrfs-select-super
 /btrfstune
@@ -29,11 +28,6 @@
 /cmds-scrub
 /cmds-send
 /cmds-subvolume
-/convert/common
-/convert/main
-/convert/source-ext2
-/convert/source-fs
-/convert/source-reiserfs
 /ctree
 /dir-item
 /disk-io
<snip>

This is due to moving things out of objects and cmds_objects variables. Such
missing dependency files cause mis-building of some source files (try touch
utils.h; make mkfs/main.o).

This patch introduce a new variable "all_objects" to keep all the objects and
use the variable to generate proper dependency file building rules.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: refactor create_data_reloc_tree
yingyil [Wed, 30 Aug 2017 17:24:49 +0000 (10:24 -0700)]
btrfs-progs: mkfs: refactor create_data_reloc_tree

Add an objectid parameter to make the function a general one for
inserting root items and rename it to create_tree. The change cascades
down to the callchain.

Signed-off-by: yingyil <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: whitespace adjustment of [LD] fssum
David Sterba [Tue, 12 Sep 2017 17:09:29 +0000 (19:09 +0200)]
btrfs-progs: build: whitespace adjustment of [LD] fssum

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: remove temporary images in mkfs/005 and mkfs/006
David Sterba [Tue, 12 Sep 2017 16:22:24 +0000 (18:22 +0200)]
btrfs-progs: tests: remove temporary images in mkfs/005 and mkfs/006

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: remove temporary loopdev files
David Sterba [Tue, 12 Sep 2017 16:17:51 +0000 (18:17 +0200)]
btrfs-progs: tests: remove temporary loopdev files

Remove files for testing loop devices when using the helper.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: convert: use correct string printing for errcode_t
David Sterba [Tue, 12 Sep 2017 13:23:05 +0000 (15:23 +0200)]
btrfs-progs: convert: use correct string printing for errcode_t

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: cleanup whitespaces of btrfs-completion
Misono, Tomohiro [Tue, 12 Sep 2017 01:04:10 +0000 (10:04 +0900)]
btrfs-progs: cleanup whitespaces of btrfs-completion

Clean up spaces and use only tabs. No functional change.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add testing image for zstd for btrfs-restore
Nick Terrell [Mon, 11 Sep 2017 19:27:52 +0000 (21:27 +0200)]
btrfs-progs: tests: add testing image for zstd for btrfs-restore

Adds tests for the new features based on a prebuilt btrfs image with a
zstd compressed file.

Split from the previous patch.

Signed-off-by: Nick Terrell <terrelln@fb.com>
[ with some cleanups ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Add zstd support
Nick Terrell [Fri, 8 Sep 2017 20:29:23 +0000 (13:29 -0700)]
btrfs-progs: Add zstd support

Adds zstd support to the btrfs program. An optional dependency on libzstd
>= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and
defaults to detecting if libzstd is present using `pkg-config'.

The patch is also available in my fork of btrfs-progs [1], which passes
Travis-CI with the new tests. The prebuilt binary is available there.

I haven't updated Android.mk.

[1] https://github.com/terrelln/btrfs-progs/tree/devel

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update btrfs-completion
Misono, Tomohiro [Mon, 11 Sep 2017 05:22:43 +0000 (14:22 +0900)]
btrfs-progs: update btrfs-completion

This patch updates btrfs-completion:
 - add "filesystem du" and "rescure zero-log"
 - restrict _btrfs_mnts to show btrfs type only
 - add more completion in last case statements

(This file contains both spaces/tabs and may need cleanup.)

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: build: sync Makefile to Android.mk
David Sterba [Mon, 11 Sep 2017 18:56:18 +0000 (20:56 +0200)]
btrfs-progs: build: sync Makefile to Android.mk

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: fix run_mustfail in cli-tests/007-check-force
David Sterba [Mon, 11 Sep 2017 15:29:03 +0000 (17:29 +0200)]
btrfs-progs: tests: fix run_mustfail in cli-tests/007-check-force

The sanity check in run_mustfail does not work as expected and allowed
the command to continue without the missing parameter

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=196893
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: README: update links, enhance contribution section
David Sterba [Mon, 11 Sep 2017 15:17:26 +0000 (17:17 +0200)]
btrfs-progs: README: update links, enhance contribution section

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: use CRC32C reversing instead of brute force to find collisions
Piotr Pawlow [Fri, 11 Aug 2017 13:47:35 +0000 (15:47 +0200)]
btrfs-progs: image: use CRC32C reversing instead of brute force to find collisions

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: add a function to check if generated filename suffix is valid
Piotr Pawlow [Fri, 11 Aug 2017 13:31:56 +0000 (15:31 +0200)]
btrfs-progs: image: add a function to check if generated filename suffix is valid

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: add a function to calculate CRC32C collisions
Piotr Pawlow [Fri, 11 Aug 2017 13:06:17 +0000 (15:06 +0200)]
btrfs-progs: image: add a function to calculate CRC32C collisions

The function uses the reverse CRC32C table to quickly calculate a
4-byte suffix, that when added to the original data will make it
match desired checksum.

Author: Piotr Pawlow <pp@siedziba.pl>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: add reverse CRC32C table
Piotr Pawlow [Fri, 11 Aug 2017 12:36:32 +0000 (14:36 +0200)]
btrfs-progs: image: add reverse CRC32C table

The table will be used to speed up calculations of CRC32C collisions.

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: move core find_collision code to a separate function
Piotr Pawlow [Fri, 11 Aug 2017 11:46:32 +0000 (13:46 +0200)]
btrfs-progs: image: move core find_collision code to a separate function

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: image: fix non-printable characters in generated file names
Piotr Pawlow [Fri, 11 Aug 2017 15:11:44 +0000 (17:11 +0200)]
btrfs-progs: image: fix non-printable characters in generated file names

Function find_collision sometimes generated file names with non-printable
DEL characters (code 127), for example file name "|5gp!" would be changed
to "U'2<DEL>y" when using "crc-collisions" sanitize mode.

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: Removed missing header file, fixes compilation error
Uli Heller [Sat, 9 Sep 2017 06:02:51 +0000 (08:02 +0200)]
btrfs-progs: Removed missing header file, fixes compilation error

(From pull request #61.)

Author: Uli Heller <uli.heller@daemons-point.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs progs v4.13
David Sterba [Fri, 8 Sep 2017 14:20:05 +0000 (16:20 +0200)]
Btrfs progs v4.13

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: update CHANGES for v4.13
David Sterba [Fri, 28 Jul 2017 13:29:46 +0000 (15:29 +0200)]
btrfs-progs: update CHANGES for v4.13

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: add new test for inspect-internal rootid
Misono, Tomohiro [Thu, 7 Sep 2017 01:48:27 +0000 (10:48 +0900)]
btrfs-progs: test: add new test for inspect-internal rootid

This new test checks inspect-internal rootid
 - handle path to subvolume/directory/file as an argument
 - get different id for each subvolume
 - get the expected id for each file/directory (i.e. the same as
 containing subvolume)

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: check for kernel support for reiserfs
David Sterba [Thu, 7 Sep 2017 15:29:51 +0000 (17:29 +0200)]
btrfs-progs: tests: check for kernel support for reiserfs

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: enhance post-rollback fsck tests
David Sterba [Thu, 7 Sep 2017 12:44:45 +0000 (14:44 +0200)]
btrfs-progs: tests: enhance post-rollback fsck tests

The post-rollback helper still assumes just extN, we need an extra
argument that'll get passed to fsck. Change all callsites at once so the
tests do not fail temporarily.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agoBtrfs-progs: print-tree: check num_stripes in print_chunk
Zhang Yu [Thu, 24 Aug 2017 10:38:41 +0000 (18:38 +0800)]
Btrfs-progs: print-tree: check num_stripes in print_chunk

[TEST/fuzz] case: 004-simple-dump-tree

Since the wrong key(DATA_RELOC_TREE CHUNK_ITEM 0) in root tree,
error calling print_chunk(), resulting in num_stripes == 0.

ERROR:
     [TEST/fuzz]   004-simple-dump-tree
ctree.h:317: btrfs_chunk_item_size: BUG_ON `num_stripes == 0`
        triggered, value 1

failed (ignored, ret=134): /myproject/btrfs-progs/btrfs
inspect-internal dump-tree
/myproject/btrfs-progs/tests/fuzz-tests/images/
bko-155201-wrong-chunk-item-in-root-tree.raw.restored

test failed for case 004-simple-dump-tree
Makefile:288: recipe for target 'test-fuzz' failed
make: *** [test-fuzz] Error 1

So, check on num_stripes in print_chunk

Signed-off-by: Zhang Yu <zhangyu-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: fix name generation not to contain trailing spaces
Misono, Tomohiro [Tue, 5 Sep 2017 05:50:03 +0000 (14:50 +0900)]
btrfs-progs: test: fix name generation not to contain trailing spaces

First patch causes test-convert fails.  This is because
generate_dataset() creates a name containing trailing spaces for
"slow_symlink" type, and cause getfacl error in convert_test_perm().
(This is not noticed since original run_check_stdout() throws away the
error.)

Fix this by use space for delimiter for cut.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: test: fix run_check_stdout() call _fail()
Misono, Tomohiro [Tue, 5 Sep 2017 05:49:02 +0000 (14:49 +0900)]
btrfs-progs: test: fix run_check_stdout() call _fail()

run_check_stdout() uses "... | tee ... || _fail".  However, since tee
won't fail, _fail() is not called even if first command fails.

Fix this by checking PIPESTATUS in the end.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: Add test case for mkfs --rootdir parameter
Qu Wenruo [Tue, 5 Sep 2017 01:24:15 +0000 (10:24 +0900)]
btrfs-progs: tests: Add test case for mkfs --rootdir parameter

Add test case which checks if -r|--rootdir mkfs option can handle
symlink/char/block/fifo files.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: Fix wrong file type for dir items and indexes when specifying...
Qu Wenruo [Mon, 4 Sep 2017 03:43:19 +0000 (12:43 +0900)]
btrfs-progs: mkfs: Fix wrong file type for dir items and indexes when specifying root directory

[Bug]
If using mkfs.btrfs with "-r" parameter and specified directory has
fifo/socket/char/block special file, then created filesystem can't pass
fsck:

------
checking fs roots
unresolved ref dir 241158 index 3 namelen 9 name S.dirmngr filetype 0 errors 80, filetype mismatch
ERROR: errors found in fs roots
------

[Reason]
Btrfs dir items/indexes records inode type, while "-r" only handles
directories, regular files and symlink, it makes such special files type
to be regular file and caused the problem.

[Fix]
Add missing types for add_directory_items(), so that result of
"mkfs.btrfs -r" can pass mkfs.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: mkfs: Move the tree root creation to own function
Gu Jinxiang [Fri, 25 Aug 2017 07:23:36 +0000 (15:23 +0800)]
btrfs-progs: mkfs: Move the tree root creation to own function

make_btrfs is too long to understand, make creatation of root tree
in a function.

Some of the tree roots are now created in a loop, where the code is just
copypasted. We now make use of the reference_root_table to translate
block index to root objectid.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ updated changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: missing device and slack space report
David Sterba [Mon, 4 Sep 2017 16:33:48 +0000 (18:33 +0200)]
btrfs-progs: tests: missing device and slack space report

Verify that a missing device will not result in reporting a negative
value interpreted as 16EiB.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: device usage: don't calculate slack on missing device
Patrik Lundquist [Thu, 31 Aug 2017 11:00:24 +0000 (13:00 +0200)]
btrfs-progs: device usage: don't calculate slack on missing device

Print      Device slack:              0.00B
instead of Device slack:           16.00EiB

Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: inspect rootid: Allow a file to be specified
Misono, Tomohiro [Mon, 4 Sep 2017 05:05:34 +0000 (14:05 +0900)]
btrfs-progs: inspect rootid: Allow a file to be specified

Since cmd_inspect_rootid() calls btrfs_open_dir(), it rejects a file to
be specified. But as the document says, a file should be supported.

This patch introduces btrfs_open_file_or_dir(), which is a counterpart
of btrfs_open_dir(), to safely check and open btrfs file or directory.
The original btrfs_open_dir() content is moved to btrfs_open() and shared
by both function.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: convert misc/011-delete-missing-device to loopdevs
David Sterba [Fri, 1 Sep 2017 18:18:20 +0000 (20:18 +0200)]
btrfs-progs: tests: convert misc/011-delete-missing-device to loopdevs

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: convert misc/006-image-on-missing-device to loopdevs
David Sterba [Fri, 1 Sep 2017 18:14:55 +0000 (20:14 +0200)]
btrfs-progs: tests: convert misc/006-image-on-missing-device to loopdevs

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: move loopdev helpers out of the testcase to common
David Sterba [Fri, 1 Sep 2017 18:14:03 +0000 (20:14 +0200)]
btrfs-progs: tests: move loopdev helpers out of the testcase to common

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: cleanup loop device helpers
David Sterba [Fri, 1 Sep 2017 17:32:16 +0000 (19:32 +0200)]
btrfs-progs: tests: cleanup loop device helpers

Make the loop device helpers a bit more generic before moving them to
the common helpers.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out extent_csum dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out extent_csum dump

Factor out code to own helper and tweak the format so it matches the
rest.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print the csum length in debug-tree
Josef Bacik [Fri, 25 Aug 2017 15:11:46 +0000 (11:11 -0400)]
btrfs-progs: print the csum length in debug-tree

While looking at a log of a corrupted fs I needed to verify we were
missing csums for a given range.  Make this easier by printing out the
range of bytes a csum item covers.

Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add testcase for 'fi du' and empty subvol
David Sterba [Fri, 1 Sep 2017 14:45:34 +0000 (16:45 +0200)]
btrfs-progs: tests: add testcase for 'fi du' and empty subvol

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: fi du: don't call lookup_path_rootid for BTRFS_EMPTY_SUBVOL_DIR_OBJECTID
Goffredo Baroncelli [Fri, 18 Aug 2017 07:04:21 +0000 (09:04 +0200)]
btrfs-progs: fi du: don't call lookup_path_rootid for BTRFS_EMPTY_SUBVOL_DIR_OBJECTID

When ino is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID, the item is not referred to
any file-tree. So lookup_path_rootid() doesn't return any meaningful
value.

As was reported, this can be triggered by

$ btrfs sub create test1
$ btrfs sub create test1/test2
$ btrfs sub snap test1 test1.snap
$ btrfs fi du -s test1
  Total   Exclusive  Set shared  Filename
  0.00B       0.00B       0.00B  test1
$ btrfs fi du -s test1.snap
  Total   Exclusive  Set shared  Filename
ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: reset the ret value when ignoring an error from du_add_file
Goffredo Baroncelli [Fri, 18 Aug 2017 07:04:20 +0000 (09:04 +0200)]
btrfs-progs: reset the ret value when ignoring an error from du_add_file

In du_walk_dir(), when du_add_file() returns an error it is usually
ignored. However if the error is returned querying the last item, the
error is returned to the caller.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: tests: add test for check --force
David Sterba [Thu, 31 Aug 2017 16:49:38 +0000 (18:49 +0200)]
btrfs-progs: tests: add test for check --force

Basic test of the --force functionality, on an empty filesystem.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: add option to skip mount checks
David Sterba [Thu, 31 Aug 2017 16:34:37 +0000 (18:34 +0200)]
btrfs-progs: check: add option to skip mount checks

Sometimes it's needed to do a check on a mounted filesystem. This should
work fine on a quiescent filesystem or a read-only mount. Changes on the
block device done by kernel might confuse the userspace checker and it
might crash when it reads some stale data.

Repair without mount checks is not supported right now.

Signed-off-by: David Sterba <dsterba@suse.cz>
7 years agobtrfs-progs: print-tree: factor out temporary_item dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out temporary_item dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out persistent_item dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out persistent_item dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out qgroup_limit dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out qgroup_limit dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out qgroup_info dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out qgroup_info dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out qgroup_status dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out qgroup_status dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out dev_extent dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out dev_extent dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out free_space_info dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out free_space_info dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out shared_data_ref dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out shared_data_ref dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out extent_data_ref dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out extent_data_ref dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: factor out block_group_item dump
David Sterba [Thu, 31 Aug 2017 15:07:00 +0000 (17:07 +0200)]
btrfs-progs: print-tree: factor out block_group_item dump

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: don't print the item type for all items
David Sterba [Thu, 31 Aug 2017 14:57:47 +0000 (16:57 +0200)]
btrfs-progs: print-tree: don't print the item type for all items

In some cases it's clear from the context which item is being printed,
so we can remove them. If the item has no data, some description is
still desired (eg. orphan or various backrefs).

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: print-tree: rename item callbacks to match the key name
David Sterba [Thu, 31 Aug 2017 14:34:15 +0000 (16:34 +0200)]
btrfs-progs: print-tree: rename item callbacks to match the key name

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: move fs roots check mode switch to a helper
David Sterba [Wed, 30 Aug 2017 17:47:50 +0000 (19:47 +0200)]
btrfs-progs: check: move fs roots check mode switch to a helper

Simplify main a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: cleanup use of fs_info in check_fs_roots
David Sterba [Wed, 30 Aug 2017 17:52:44 +0000 (19:52 +0200)]
btrfs-progs: check: cleanup use of fs_info in check_fs_roots

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: pass fs_info to check_fs_roots
David Sterba [Wed, 30 Aug 2017 17:50:31 +0000 (19:50 +0200)]
btrfs-progs: check: pass fs_info to check_fs_roots

The root pointer is not used anyway, will be cleaned up next.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: move chunk and extent check mode switch to a helper
David Sterba [Wed, 30 Aug 2017 17:47:50 +0000 (19:47 +0200)]
btrfs-progs: check: move chunk and extent check mode switch to a helper

Simplify main a bit.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: pass fs_info to check_chunks_and_extents_v2
David Sterba [Wed, 30 Aug 2017 17:38:02 +0000 (19:38 +0200)]
btrfs-progs: check: pass fs_info to check_chunks_and_extents_v2

The root pointer is set to fs_root as was originally.

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: use local fs_info in check_chunks_and_extents
David Sterba [Wed, 30 Aug 2017 17:41:15 +0000 (19:41 +0200)]
btrfs-progs: check: use local fs_info in check_chunks_and_extents

Signed-off-by: David Sterba <dsterba@suse.com>
7 years agobtrfs-progs: check: pass fs_info to check_chunks_and_extents
David Sterba [Wed, 30 Aug 2017 17:38:02 +0000 (19:38 +0200)]
btrfs-progs: check: pass fs_info to check_chunks_and_extents

The root pointer is set to fs_root as was originally.

Signed-off-by: David Sterba <dsterba@suse.com>