David Sterba [Thu, 8 Feb 2018 17:34:40 +0000 (18:34 +0100)]
btrfs-progs: tests: unify test drivers, make ready for extenral testsuite
Make the TOP variable more configurable, allow to set it to any path
where to find binaries when the testsuite is exported, or fallback to
system binaries.
There's now more code duplication, the logic is now more complex so it's
left open coded for clarity. Further cleanups are possible.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 16:40:34 +0000 (17:40 +0100)]
btrfs-progs: tests: update README.md
Irregular proofreading with adjustments and enhancements.
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Thu, 8 Feb 2018 06:34:19 +0000 (14:34 +0800)]
btrfs-progs: introduce TEST_TOP and INTERNAL_BIN for tests
Use TEST_TOP as base for tests to reference any files, this will be used
for git and external testsuite.
INTERNAL_BIN is needed for referencing binaries that could reside in
different paths in git vs external testsuite.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ add quotes around sourced files, update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 14:34:04 +0000 (15:34 +0100)]
btrfs-progs: rework testsuite export
Move the testsuite to tests/ and make the tarball generation more
deterministic. As there could be many random temporary files left in the
test directories, we can't just copy them. Use 'git ls-tree' to
filter just what we want, this needs a slight extension of the file list
specification.
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Thu, 8 Feb 2018 06:34:18 +0000 (14:34 +0800)]
btrfs-progs: Add make testsuite command for export tests
Export the testsuite files to a separate tar. Since fsck tests depend
on btrfs-corrupt-block, and misc tests depends on both
btrfs-corrupt-block and fssum, so set it as prerequisites for package
commad.
Because, althougth fssum can be generated by source that are all in
tests directory, and has no rely on the btrfs's structure. But
btrfs-corrupt-block deeply relys on btrfs's structure. For consistency,
at the present stage, generete the two when create test tar.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ applied without changes, the generated tarball will be different from
the one after the follow up commits ]
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Thu, 8 Feb 2018 05:08:58 +0000 (13:08 +0800)]
btrfs-progs: add prerequisite btrfs-convert for test-misc
Since tests/misc-tests/005-convert-progress-thread-crash/test.sh need
the btrfs-convert for prerequisite. So add the dependency in Makefile.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Thu, 8 Feb 2018 05:08:57 +0000 (13:08 +0800)]
btrfs-progs: add prerequisite btrfs-image for test-fuzz
Since tests/fuzz-tests/002-simple-image/test.sh need the btrfs-image for
prerequisite. So add the dependency in Makefile.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Thu, 8 Feb 2018 05:08:56 +0000 (13:08 +0800)]
btrfs-progs: add prerequisite mkfs.btrfs for test-cli
Since tests/cli-tests/002-balance-full-no-filters/test.sh need
the mkfs.btrfs for prerequisite.
So add the dependency in Makefile.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:49:11 +0000 (01:49 +0100)]
btrfs-progs: ci: replace inline shell commands with scripts
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:41:40 +0000 (01:41 +0100)]
btrfs-progs: ci: use helper script for default build commands
There's a script for the make command combo, let's use it.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:36:50 +0000 (01:36 +0100)]
btrfs-progs: ci: enable musl build tests in docker
Enable docker in travis and start build test(s) before our testsuite.
Now for musl libc, so we don't accidentally break the build anymore.
Issue: #29
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:26:55 +0000 (01:26 +0100)]
btrfs-progs: ci: add dockerfile for a musl build test
Build environment with musl as the libc, based on Alpine Linux.
To build the docker image:
- cd travis/images
- cp test-build $dir
- cd $dir
- docker build -t $tag .
- docker push $tag
We're going to have more images that will share the prep script so it
has to be copied to the directory before making the image. No other way
around that.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:24:25 +0000 (01:24 +0100)]
btrfs-progs: ci: add support scripts for docker build
The test-build resides inside the docker image and is supposed to be
called from outside, downloads the branch and calls build-default.
Buid-default will run up to make.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 8 Feb 2018 00:00:12 +0000 (01:00 +0100)]
btrfs-progs: ci: change clone depth to 1
Check only the top commit of a given branch.
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 7 Feb 2018 09:57:43 +0000 (17:57 +0800)]
btrfs-progs: tests common: remove meaningless colon in extract_image()
The colon is meaningless so remove it.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Tue, 6 Feb 2018 07:31:42 +0000 (15:31 +0800)]
btrfs-progs: fsck-tests: Cleanup the restored image for 028
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Sat, 3 Feb 2018 00:15:42 +0000 (01:15 +0100)]
btrfs-progs: mkfs: fix build on musl
Another build failure on musl.
Issue: #90
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Sat, 3 Feb 2018 00:06:21 +0000 (01:06 +0100)]
btrfs-progs: convert: fix build on musl
Voidlinux has a patch to fix build on musl.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 2 Feb 2018 19:51:25 +0000 (20:51 +0100)]
btrfs-progs: build: add support for asciidoctor doc generator
We've been using asciidoc that's written in python2, which is going to
be phased out and deprecated next year. There's a replacement,
asciidoctor. Add a configure-time detection which tool is available,
update Documentation/Makefile.in.
The original asciidoc tool is still preferred as it produces slightly
better output. The file asciidoc.conf does not have a direct equivalten
in asciidoct and would need to be replaced by extension written in ruby.
The differences:
- the <literal> are not automatically underlined and are less visible in
the generated manual page, but it's still acceptable
- the inline CSS for the html output looks subjectively worse, is less
compact and colourful
Issue: #89
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 2 Feb 2018 19:49:00 +0000 (20:49 +0100)]
btrfs-progs: docs: fix manual page title format
Asciidoctor is more strict about the formatting for the manual pages and
needs exact number of =.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 1 Feb 2018 16:14:42 +0000 (17:14 +0100)]
btrfs-progs: check: rename files after moving code
Add prefixes to the files so it's clear they belong to the mode group.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 1 Feb 2018 06:34:42 +0000 (14:34 +0800)]
btrfs-progs: check: Cleanup all checkpatch error and warning
Since we're moving tons of code, it's a good idea to fix all errors and
warnings from checkpatch.
Note: we don't rely on checkpatch, take its warnings and errors as a
hint and do not blindly fix anything that gets reported. The code
follows some style that should be kept and is not in 100% with
what checkpatch finds. Always apply your common sense and decide
if the change makes sense or if it is really making the code
better.
Additional fixes from dsterba:
- %Lu -> %llu
- lowercase first word or messages
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ added note ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 19 Jan 2018 05:26:49 +0000 (13:26 +0800)]
btrfs-progs: check/lowmem: Cleanup unnecessary _v2 suffixes
There used to be some functions with _v2 suffix to distinguish them from
original mode that had similar functions.
However now when the lowmem mode is moved to own check/lowmem.[ch],
cleanup the _v2 suffixes, and for functions that really need to be
distinguished from original mode (exported functions), change the _v2
suffix to _lowmem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 1 Feb 2018 05:37:31 +0000 (13:37 +0800)]
btrfs-progs: check: Move lowmem check code to its own check/lowmem.[ch]
Since lowmem mode code is highly internally connected, it's pretty hard to
move them piece by piece.
In theory it's possible to move part of the functions and temporarily
export them, but it will just cause extra temporarily modifications.
So this patch moves the whole lowmem check part into its own
check/lowmem.[ch].
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:38:23 +0000 (16:38 +0800)]
btrfs-progs: check: Move reset_cached_block_groups to check/common.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:30:10 +0000 (16:30 +0800)]
btrfs-progs: check: Move check_child_node to check/common.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:25:11 +0000 (16:25 +0800)]
btrfs-progs: check: move reada_walk_down to check/common.c
Both original and lowmem modes share this function to do readahead.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:19:53 +0000 (16:19 +0800)]
btrfs-progs: check: Move check_dev_size_alignment to check/common.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:07:19 +0000 (16:07 +0800)]
btrfs-progs: check: Move link_inode_to_lostfound function to common.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 08:03:09 +0000 (16:03 +0800)]
btrfs-progs: check: Move __create_inode_item function to check/common.c
Move __create_inode_item() function to check/common.c and rename it to
insert_inode_item(), with comment added.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 07:49:39 +0000 (15:49 +0800)]
btrfs-progs: check: Move count_csum_range function to check/common.c
Despite of moving it to check/common.c, also:
1) Add extra comment of the function
2) Change @root parameter to @fs_info
Since @root is never used, csum_root is picked from fs_info anyway.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 07:41:15 +0000 (15:41 +0800)]
btrfs-progs: check: Move fs_root_objectid function to check/common.h
Just another small wrapper shared between original and lowmem mode.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 07:38:45 +0000 (15:38 +0800)]
btrfs-progs: check: Move imode_to_type function to check/common.h
This function is shared between original and lowmem mode, and it's small
enough, so move it to check/common.h.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 07:31:53 +0000 (15:31 +0800)]
btrfs-progs: check: Export check global variables to check/common.h
There are a dozen of variables which are used as "check global"
variables, like @total_csum_bytes or @no_holes.
These variables are used freely across the check code, however since
we're splitting check code, they need to be exported so they can be used
in other files.
This patch just export them and add declarations for them in
check/common.h.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 18 Jan 2018 05:52:16 +0000 (13:52 +0800)]
btrfs-progs: check: Move node_refs structure to check/common.h
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 17 Jan 2018 07:24:05 +0000 (15:24 +0800)]
btrfs-progs: check: Move definitions of lowmem mode to check/lowmem.h
Unlike original mode, lowmem mode mostly uses normal tree operations, so
no structure definitions, only a lot of random error bits.
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 17 Jan 2018 07:16:51 +0000 (15:16 +0800)]
btrfs-progs: check: Move original mode definitions to check/original.h
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 17 Jan 2018 07:01:56 +0000 (15:01 +0800)]
btrfs-progs: Move cmds-check.c to check/main.c
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 31 Jan 2018 02:40:36 +0000 (10:40 +0800)]
btrfs-progs: tests: chang tree-reloc-tree test number from 027 to 015
There are 2 fsck tests with the same number 027:
tree-reloc-tree
bad-extent-inline-ref-type
And we also have a hole in 015, so just rename tree-reloc-tree to 015,
to get rid of the duplicated test number and fill in the hole.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Wed, 31 Jan 2018 03:09:19 +0000 (11:09 +0800)]
btrfs-progs: Cleanup use of root in leaf_data_end
In function leaf_data_end, root is just used to get fs_info, so change
the parameter of this function from btrfs_root to btrfs_fs_info. And
also make it consistent with kernel.
Add const to parameter leaf of function btrfs_item_offset_nr to keep
type consistent with leaf_data_end.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:05 +0000 (15:26 +0800)]
btrfs-progs: remove no longer used btrfs_alloc_extent
Function btrfs_alloc_extent is no longer be used. So let's remove it.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:04 +0000 (15:26 +0800)]
btrfs-progs: do clean up for redundancy value assignment
Although skinny_metadata's type is int, its value just can be 0/1. And
if condition be true only when skinny_metadata equals 1, so in if's
executive part, set skinny_metadata to 1 is redundancy. Remove it.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:03 +0000 (15:26 +0800)]
btrfs-progs: Use fs_info instead of root for BTRFS_MAX_XATTR_SIZE
Do a cleanup. Also make it consistent with kernel. Use fs_info instead
of root for BTRFS_MAX_XATTR_SIZE, since maybe in some situation we do
not know root, but just know fs_info.
To be consistent with kernel, change macro to inline function.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:02 +0000 (15:26 +0800)]
btrfs-progs: Sync code with kernel for BTRFS_MAX_INLINE_DATA_SIZE
Do a cleanup. Also make it consistent with kernel. Use fs_info instead
of root for BTRFS_MAX_INLINE_DATA_SIZE, since maybe in some situation we
do not know root, but just know fs_info.
Change macro to inline function to be consistent with kernel. And
change the function body to match kernel.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:01 +0000 (15:26 +0800)]
btrfs-progs: Use fs_info instead of root for BTRFS_NODEPTRS_PER_BLOCK
Do a cleanup. Also make it consistent with kernel. Use fs_info instead
of root for BTRFS_NODEPTRS_PER_BLOCK, since maybe in some situation we
do not know root, but just know fs_info.
To be consistent with kernel, change macro to inline function.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Fri, 26 Jan 2018 07:26:00 +0000 (15:26 +0800)]
btrfs-progs: Use fs_info instead of root for BTRFS_LEAF_DATA_SIZE
Do a cleanup. Also make it consistent with kernel. Use fs_info instead
of root for BTRFS_LEAF_DATA_SIZE, since maybe in some situation we do
not know root, but just know fs_info.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Fri, 2 Feb 2018 14:51:18 +0000 (15:51 +0100)]
Btrfs progs v4.15
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 14 Nov 2017 14:52:26 +0000 (15:52 +0100)]
btrfs-progs: update CHANGES for v4.15
Signed-off-by: David Sterba <dsterba@suse.com>
William Giokas [Mon, 22 Jan 2018 18:25:49 +0000 (11:25 -0700)]
btrfs-progs: docs: fix typo in btrfs-filesystem manual page
Pull-request: #86
Signed-off-by: William Giokas <1007380@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 30 Jan 2018 17:26:46 +0000 (18:26 +0100)]
btrfs-progs: tests: fixup mount tests of fsck/028-unaligned-super-dev-sizes
This test was broken because it tried to mount a different image than
what it had repaired.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 30 Jan 2018 17:51:25 +0000 (18:51 +0100)]
btrfs-progs: tests: enhance common umount helper to take optional paths
The run_check_umount_test_dev umounts the TEST_DEV and also optionally
uses the arguments but this would not work as expected if the TEST_DEV
is not a vald path for umount (eg. a restored image).
Update the helper so it tries to umount all paths, or fallback to
TEST_DEV to keep the current behaviour.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 30 Jan 2018 14:46:05 +0000 (15:46 +0100)]
btrfs-progs: tests: disable some mkfs/010 testcases inside travis
Node sizes larger than 16k will fail due to enospc in the mount test.
This is likely caused by the kernel. Keep the condition only local to
travis so any other testing environment could see the failure
eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 29 Jan 2018 17:26:30 +0000 (18:26 +0100)]
btrfs-progs: docs: clean all generated files
The section 3 manual page btrfs-ioctl was not deleted.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 29 Jan 2018 16:46:42 +0000 (17:46 +0100)]
btrfs-progs: build: update help text for zstd
The enable/disable does not take an argument and we don't autodetect
zstd anymore.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 24 Jan 2018 02:30:28 +0000 (10:30 +0800)]
btrfs-progs: Remove unnecessary parameter for btrfs_add_block_group
@chunk_objectid of btrfs_make_block_group() function is always fixed to
BTRFS_FIRST_FREE_OBJECTID, so there is no need to pass it as parameter
explicitly.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Rosen Penev [Sun, 7 Jan 2018 21:54:21 +0000 (13:54 -0800)]
btrfs-progs: treewide: Replace strerror(errno) with %m.
As btrfs is specific to Linux, %m can be used instead of strerror(errno)
in format strings. This has some size reduction benefits for embedded
systems.
glibc, musl, and uclibc-ng all support %m as a modifier to printf.
A quick glance at the BIONIC libc source indicates that it has
support for %m as well. BSDs and Windows do not but I do believe
them to be beyond the scope of btrfs-progs.
Compiled sizes on Ubuntu 16.04:
Before:
3916512 btrfs
233688 libbtrfs.so.0.1
4899 bcp
2367672 btrfs-convert
2208488 btrfs-corrupt-block
13302 btrfs-debugfs
2152160 btrfs-debug-tree
2136024 btrfs-find-root
2287592 btrfs-image
2144600 btrfs-map-logical
2130760 btrfs-select-super
2152608 btrfstune
2131760 btrfs-zero-log
2277752 mkfs.btrfs
9166 show-blocks
After:
3908744 btrfs
233256 libbtrfs.so.0.1
4899 bcp
2366560 btrfs-convert
2207432 btrfs-corrupt-block
13302 btrfs-debugfs
2151104 btrfs-debug-tree
2134968 btrfs-find-root
2281864 btrfs-image
2143536 btrfs-map-logical
2129704 btrfs-select-super
2151552 btrfstune
2130696 btrfs-zero-log
2276272 mkfs.btrfs
9166 show-blocks
Total savings: 23928 (24 kilo)bytes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 25 Jan 2018 12:37:13 +0000 (13:37 +0100)]
btrfs-progs: don't clobber errno in close_file_or_dir
Preserve the errno value for the caller in case closing happens in the
middle of eg. an ioctl and reporing the failure. The errors that could
happen in close/closedir do not bother us.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 Jan 2018 07:13:05 +0000 (15:13 +0800)]
btrfs-progs: volumes: Remove unnecessary parameters when allocating device extent
@chunk_tree and @chunk_objectid of device extent is fixed to
BTRFS_CHUNK_TREE_OBJECTID and BTRFS_FIRST_CHUNK_TREE_OBJECTID
respectively.
There is no need to pass them as parameter explicitly.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 Jan 2018 07:13:04 +0000 (15:13 +0800)]
btrfs-progs: volumes: Remove unnecessary trans parameter
Remove @trans parameter for find_free_dev_extent_start() and its
callers.
The function itself is doing read-only tree search, no use of
transaction.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 Jan 2018 07:13:03 +0000 (15:13 +0800)]
btrfs-progs: volumes: Make find_free_dev_extent_start static
The function is not used by anyone else outside of volumes.c, make it
static.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 3 Jan 2018 07:13:02 +0000 (15:13 +0800)]
btrfs-progs: Use bool parameter to determine if we're allocating data extent
btrfs_reserve_extent() uses int @data to determine if we're allocating
data extent, while reuse the parameter later to pass it as profile
(data/meta/sys).
It's a little confusing, this patch will follow kernel parameter to use
bool @is_data to replace it.
And in btrfs_reserve_extent(), use dedicated u64 @profile.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 23 Jan 2018 16:22:30 +0000 (17:22 +0100)]
btrfs-progs: tests: truncate test image to 0 first
We use the prepare_test_dev helper to make sure the image has at least
this size. The "at least" part is not desired by some tests as the
device might be larger than the test expects.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 23 Jan 2018 15:57:57 +0000 (16:57 +0100)]
btrfs-progs: tests: add more coverage to mkfs-tests/013-reserved-1M-for-single
Though the newly added mkfs profiles should not be affected, let's add
the remaining valid single device profiles for better coverage.
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 10 Jan 2018 04:56:48 +0000 (12:56 +0800)]
btrfs-progs: tests: mkfs: don't overwrite first 1M for single
Add test case to check if the first device extent is occupying reserved
0~1M range.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 24 Jan 2018 02:38:30 +0000 (10:38 +0800)]
btrfs-progs: tests:mkfs/010: Output minimal device size
To make debugging a little easier.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 10 Jan 2018 04:56:47 +0000 (12:56 +0800)]
btrfs-progs: mkfs: Prevent temporary system chunk to use space in reserved 1M range
When creating btrfs, mkfs.btrfs will firstly create a temporary system
chunk as basis, and then created needed trees or new devices.
However the layout temporary system chunk is hard-coded and uses
reserved [0, 1M) range of devid 1.
Change the temporary chunk layout from old:
0 1M 4M 5M
|<----------- temp chunk -------------->|
And it's 1:1 mapped, which means it's a SINGLE chunk,
and stripe offset is also 0.
to new layout:
0 1M 4M 5M
|<----------- temp chunk -------------->|
And still keeps the 1:1 mapping.
However this also affects btrfs_min_dev_size() which still assume
temporary chunks starts at device offset 0.
The problem can only be exposed by "-m single" or "-M" where we reuse the
temporary chunk.
With other meta profiles, system and meta chunks are allocated by later
btrfs_alloc_chunk() call, and old SINGLE chunks are removed, so it will
be no such problem for other meta profiles.
Reported-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
[ folded fix for the minimal device size calculation ]
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 23 Jan 2018 15:25:11 +0000 (16:25 +0100)]
btrfs-progs: tests: 029-super-recovery: cleanup the test
Transform the test to the common helpers and don't manage the loop
devices here. The test category changes from check to misc.
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:48 +0000 (10:39 +0200)]
btrfs-progs: tests: Add test for super block recovery
This functionality regressed some time ago and it was never caught. Seems no
one complained of that, but to be sure add a regression test to prevent future
regressions.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:49 +0000 (10:39 +0200)]
btrfs-progs: super-recover: fix the broken sb detection
Commit
3296d058b7ce ("btrfs-progs: super-recover: Reuse
btrfs_read_dev_super function") changed the logic when a superblock
is added to the bad block list to depend on -EIO. However currently
btrfs_read_dev_super doesn't return -EIO when the fist super block
is broken. Instead it returns -1. This causes the super-recovery
logic to miss the fact that the first super block is completely broken.
Fix this by considering any error code from btrfs_read_dev_super other
than -ENOENT to mean that the super block is corrupted. -ENOENT
means that the superblock copy is not part of the fs i.e. it's smaller
than the offset of the block. This can only occur for the 2nd copy at
256gb mark.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:47 +0000 (10:39 +0200)]
btrfs-progs: Document logic of btrfs_read_dev_super
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Thu, 7 Dec 2017 09:10:05 +0000 (11:10 +0200)]
btrfs-progs: Replace usage of list_for_each with list_for_each_entry
There are a couple of places where instead of the more succinct
list_for_each_entry the code uses list_for_each. This results in
slightly more code with no additional benefit as well as no
coherent pattern. This patch makes the code uniform. No functional
changes.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
[ remove unused variable in uuid_search ]
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:45 +0000 (10:39 +0200)]
btrfs-progs: Remove recover_get_good_super
Currently getting the good super really consists of just getting the
first entry on the linked list, since it's the one with the highest
transid. So remove the function and just use list_first_entry directly.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:44 +0000 (10:39 +0200)]
btrfs-progs: Factor out common print_device_info
This function has been copied twice in chunk-recover and super-recover. Factor
it out into utils.c/h and use it. No functional changes.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Tue, 5 Dec 2017 08:39:43 +0000 (10:39 +0200)]
btrfs-progs: tests: Explictly state test.sh must be executable
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 23 Jan 2018 13:38:30 +0000 (14:38 +0100)]
btrfs-progs: build: simplify version tracking
There will be a plain file tracking the last released version.
The rest will be simplified to print it where needed. The version
augmented by the current git status was not working anyway since we've
switched to autoconf. The result of version.h with the potential git
status was generated at configure time, which does not mean it's
accurate regarding the git status.
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Tue, 23 Jan 2018 13:16:45 +0000 (14:16 +0100)]
btrfs-progs: tests: bump zstd version in CI to 1.3.3
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov [Fri, 29 Dec 2017 09:01:44 +0000 (11:01 +0200)]
btrfs-progs: Print error on invalid extent item format during check
While performing normal mode check if the code comes across an invalid
extent format it will just BUG() and exit without printing any useful
information for debugging. Improve the situation by outputting the
key/leaf bytenr/slot which will enable to quickly inspect the tree and
see what the corruption is.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Su Yue [Wed, 22 Nov 2017 02:17:16 +0000 (10:17 +0800)]
btrfs-progs: check: report more specific info about invalid location
Previously, it was so useless to print message like
"invalid location %d".
Let it print objectid and offset of the dir_item too.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Mon, 8 Jan 2018 18:48:51 +0000 (19:48 +0100)]
btrfs-progs: tests: fix typo in error message
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Tue, 31 Oct 2017 09:23:04 +0000 (17:23 +0800)]
btrfs-progs: build: Remove unused variable TESTS
Variable is never used.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang [Tue, 31 Oct 2017 09:23:03 +0000 (17:23 +0800)]
btrfs-progs: Remove unused parameter trans
Some parameter of trans is not used indeed.
Let's remove them.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Mon, 13 Nov 2017 05:33:16 +0000 (13:33 +0800)]
btrfs-progs: qgroup: cleanup __qgroup_search
Replace the if statement with the switch statement, and return the
appropriate value for the future use rather than directly exit.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lu Fengqi [Mon, 13 Nov 2017 05:33:15 +0000 (13:33 +0800)]
btrfs-progs: qgroup: move btrfs_show_qgroups's error handler to __qgroup_search
We have to process the return value of BTRFS_IOC_TREE_SEARCH ioctl in
advance, so that we can distinguish between the two case where quota
is not enabled (ioctl return -ENOENT) and either parent qgroup or child
qgroup does not exist (update_qgroup_relation return -ENOENT). Besides
this, any error in this routine has been reported, so we don't need to
report again in cmd_qgroup_show.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Sat, 23 Dec 2017 20:52:38 +0000 (21:52 +0100)]
btrfs-progs: Fix build of btrfs-calc-size
Build would fail because it couldn't find the usage function.
Signed-off-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Hans van Kranenburg [Sat, 23 Dec 2017 20:52:37 +0000 (21:52 +0100)]
btrfs-progs: Fix progs_extra build dependencies
The Makefile does not have a dependency path that builds dependencies
for tools listed in progs_extra.
E.g. doing make btrfs-show-super in a clean build environment results in:
gcc: error: cmds-inspect-dump-super.o: No such file or directory
Makefile:389: recipe for target 'btrfs-show-super' failed
Signed-off-by: Hans van Kranenburg <hans@knorrie.org>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.com>
David Sterba [Thu, 7 Dec 2017 18:56:34 +0000 (19:56 +0100)]
btrfs-progs: docs: update manual for mkfs --shrink
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 29 Nov 2017 08:58:43 +0000 (16:58 +0800)]
btrfs-progs: tests/mkfs: verify that mkfs.btrfs rootdir+shrink behaves correctly
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 29 Nov 2017 08:42:05 +0000 (16:42 +0800)]
btrfs-progs: mkfs: Use the whole file or block device to mkfs for rootdir
For --rootdir, even for large existing file or block device, it will
always shrink the resulting filesystem.
The problem is, mkfs.btrfs will try to calculate the dir size, and use
it as @block_count to mkfs, which makes the filesystem shrunk.
Fix it by trying to get the original block device or file size as
@block_count, so mkfs.btrfs can use the full file/block device for
--rootdir option.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 29 Nov 2017 08:26:02 +0000 (16:26 +0800)]
btrfs-progs: tests/mkfs: Introduce test case to check if mkfs rootdir can create a new file
To test regression
460e93f25754 ("btrfs-progs: mkfs: check the status of
file at mkfs").
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update test to create a out of /tmp ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 29 Nov 2017 08:07:34 +0000 (16:07 +0800)]
btrfs-progs: mkfs: fix regression preventing --rootdir to create file
Commit
460e93f25754 ("btrfs-progs: mkfs: check the status of file at mkfs")
will try to check the file state before creating fs on it.
The check is mostly fine for normal mkfs case, while for --rootdir
option, it's allowed to create a new file if the destination file
doesn't exist.
Fix it by allowing non-existent file if --rootdir is specified.
Fixes:
460e93f25754 ("btrfs-progs: mkfs: check the status of file at mkfs")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 09:13:55 +0000 (17:13 +0800)]
btrfs-progs: mkfs: Separate shrink from rootdir
Make --shrink a separate option for --rootdir, and change the default to
off.
The shrinking behaviour is not a commonly used feature but can be useful
for creating minimal pre-filled images, in one step, without requiring
to mount.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog and error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 07:12:58 +0000 (15:12 +0800)]
btrfs-progs: mkfs/rootdir: Shrink fs for rootdir option
Use the new dev extent based shrink method for rootdir option. This
restores the original behaviour when --rootdir will create a minimal
filesystem size.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 02:11:07 +0000 (10:11 +0800)]
btrfs-progs: mkfs/rootdir: Use over-reserve method to make size estimate easier
Use an easier method to calculate the estimate device size for
mkfs.btrfs --rootdir.
The new method will over-estimate, but should ensure we won't encounter
ENOSPC.
It relies on the following data:
1) number of inodes -- for metadata chunk size
2) rounded up data size of each regular inode -- for data chunk size
Total meta chunk size = round_up(nr_inode * (PATH_MAX * 3 + sectorsize),
min_chunk_size) * profile_multiplier
PATH_MAX is the maximum size possible for INODE_REF/DIR_INDEX/DIR_ITEM.
Sectorsize is the maximum size possible for inline extent.
min_chunk_size is 8M for SINGLE, and 32M for DUP, get from
btrfs_alloc_chunk().
profile_multiplier is 1 for Single, 2 for DUP.
Total data chunk size is much easier.
Total data chunk size = round_up(total_data_usage, min_chunk_size) *
profile_multiplier
Total_data_usage is the sum of *rounded up* size of each regular inode
use.
min_chunk_size is 8M for SINGLE, 64M for DUP, get from btrfS_alloc_chunk().
Same profile_multiplier for meta.
This over-estimate calculate is, of course inacurrate, but since we will
later shrink the fs to its real usage, it doesn't matter much now.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update comments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 07:36:57 +0000 (15:36 +0800)]
btrfs-progs: mkfs: Don't use custom chunk allocator for rootdir
Remove the custom chunk allocator for mkfs. It is buggy in connection to
the --rootdir option and puts file data to the reerved 1M area. The
feature of the custom allocator was to reserve only minimal amount of
blockgroup space. This will temporarily stop working and will need an
explicit request by option, added by following patches.
Use the generic chunk allocator.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Wed, 29 Nov 2017 06:31:39 +0000 (14:31 +0800)]
btrfs-progs: mkfs: Cleanup temporary chunks before filling rootdir
Cleanup of temporary chunks should be done as soon as possible, and it
should be especially before doing large tree operations, like filling
the filesystem when using --rootdir.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 20 Oct 2017 01:59:06 +0000 (09:59 +0800)]
btrfs-progs: mkfs: Update allocation info before verbose output
Since new --rootdir can allocate chunk, it will modify the chunk
allocation result.
This patch will update allocation info before verbose output to reflect
such info.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Fri, 20 Oct 2017 01:59:04 +0000 (09:59 +0800)]
btrfs-progs: mkfs/rootdir: Introduce function to get end position of last device extent
Useful for later 'mkfs.btrfs --rootdir' shrink support.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 12 Oct 2017 07:11:33 +0000 (15:11 +0800)]
btrfs-progs: test/mkfs: Test if the minimal device size is valid
New test case to test if the minimal device size given by "mkfs.btrfs"
failure case is valid.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ renamed script ]
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:38 +0000 (13:41 +0800)]
btrfs-progs: mkfs: move source dir size calculation to its own files
Also rename the function from size_sourcedir() to mkfs_size_dir().
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo [Thu, 19 Oct 2017 05:41:37 +0000 (13:41 +0800)]
btrfs-progs: mkfs: move image creation of rootdir to its own files
In fact, --rootdir option is getting more and more independent from
normal mkfs code.
So move image creation function, make_image() and its related code to
mkfs/rootdir.[ch], and rename the function to btrfs_mkfs_fill_dir().
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>